CodeSonar GitLab CI/CD Integration

NOTICE: An internal or private access GitLab repository is required for this CodeSonar integration.

The CodeSonar GitLab Integration package provides tools, examples, and documentation to help you add CodeSonar to your GitLab Continuous Integration (CI) pipelines. Using CodeSonar in your GitLab pipelines will allow you to automatically analyze code changes when they are pushed to your GitLab project. Basic analysis results can be viewed directly in GitLab. Detailed analysis results can be viewed in the CodeSonar hub.

Prerequisites

• Access to a GitLab server. GitLab Ultimate edition is strongly recommended.
• Access to a CodeSonar hub.
• A CodeSonar hub user account to be used in automated pipeline jobs.
• A GitLab runner host where you can compile your code and execute a CodeSonar analysis.
• A mechanism for preserving your analysis files after the pipeline has finished: one of the following.
  • You are using CodeSonar SaaS.
  • Your hub already has one or more associated remote analysis launch daemons that you can use to perform remote-managed analyses.
  • You have available resources to set up an Analysis Data Server to manage the files.
• CodeSonar software installation package.
• CodeSonar GitLab integration tools package.
• Familiarity with Git, YAML, and shell programming (bash, powershell, etc.).
• A Developer workstation where you can check out and modify code using the Git command line.
• An internal or private access GitLab repository project that you want to analyze using CodeSonar.

IMPORTANT: An internal or private access GitLab repository is required. Limitations imposed by the CodeSonar End User License Agreement (EULA) prohibit the use of CodeSonar for code analysis on a public repository.

You will need to prepare one or more GitLab pipeline job runner hosts.

• A host that will run the CodeSonar build/analysis.

  If you already have a runner host that can compile your code, you can add CodeSonar to it. Otherwise, you will need to create a new runner host for this purpose.

• A host that will arrange for analysis file management after the pipeline has completed.

  If you are running CodeSonar SaaS, or if your CodeSonar hub is already set up with remote analysis launch daemons for use with -remote and -remote-archive, you can use your existing infrastructure and do not need to set up a separate host.

  Otherwise, you will need to set up an Analysis Data Server host to store large analysis artifacts from your pipeline jobs and shares them with the CodeSonar hub.

GitLab Static Analysis features (Ultimate Edition)

GitLab provides several Static Application Security Testing (SAST) features, including a "security report" which can display the results of a CodeSonar analysis in GitLab. The GitLab SAST features are only available in GitLab Ultimate edition. If you do not have GitLab Ultimate edition, then you will not have access to the security report in GitLab and you may need to adjust your pipeline job definitions so that they do not make use of the advanced SAST features.

Security scan report (Ultimate Edition)

A GitLab security scan report is available for each merge request. When you perform a static analysis in a GitLab pipeline associated with a merge request, the results of the analysis are uploaded as a pipeline "artifact" and are presented in a security scan report web page. After a successful pipeline analysis, you can find a link to "View full report" near the top of your merge request page. Since a security scan report is associated with each pipeline you run, you can also find a link to the report in the "Security" tab of each pipeline page.

The report shows a list of all results found by the analysis. Clicking on a result item will show a popup containing item details.

Project Security dashboard (Ultimate Edition)

GitLab provides a "Security Dashboard" and a "Vulnerability Report" that can display aggregate security scan information for your repository project's default branch. The security dashboard provides information about vulnerabilities found in your project over time. The vulnerability report is similar to a merge request security scan report, but it is intended to apply to your project as a whole.

This document will not describe how to configure the GitLab security dashboard or the project vulnerability report, however, make note of the following.

• The dashboard and project vulnerability report apply only to your default branch.
• Your analysis pipeline job name must be suffixed with the string "-sast" and it must get executed in the "test" stage.
• Only the scan results from pipelines triggered by a direct "push" to your default branch will apply to these project-level reports. Results from merge request pipelines will not apply.

GitLab provides some documentation for using custom scanners with the security dashboard: https://docs.gitlab.com/ee/user/application_security/sast/#customizing-the-sast-settings.

Pipeline Setup

There are many benefits to setting up a continuous integration (CI) pipeline. However, the setup process can be complicated for a variety of reasons. The information below is intended to help you navigate the complexities of introducing CodeSonar static analysis into a pipeline and managing the code analysis data produced when the pipeline is executed.

Detailed instructions for preparing a pipeline for analysis with CodeSonar 7.3 or later are provided in the following platform-specific documents:

• Linux and Docker
• Linux (without Docker)
• Windows (without Docker)

More instruction document links are provided further below.

The remaining sections of this document provide a platform-neutral summary of the pipeline configuration steps.

General pipeline configuration procedure

The following general steps are required for preparing a pipeline for CodeSonar analysis:

1. Create and install an analysis data server host.
2. Create a basic pipeline that can build your code.
3. Install CodeSonar and integration tools in build runner environment.
4. Update pipeline job definition to perform CodeSonar analysis.

Several systems and services will be involved, including:

• GitLab Server
• CodeSonar hub
• CodeSonar analysis data server
• GitLab runner
• Developer workstation

The following diagram depicts the typical relationships between these systems and services.

1. Create and install an analysis data server host

If you are using CodeSonar SaaS, or if your CodeSonar hub is already set up with remote analysis launch daemons for use with -remote and -remote-archive, you can use your existing infrastructure and do not need to set up a separate host: go on to 2. Create a basic pipeline that can build your code.

A CodeSonar analysis produces several analysis data artifacts which contain information about the source code as it was interpreted by your compilers. The analysis data artifacts are created on the host machine that does the analysis, and can be quite large. The CodeSonar hub needs access to the analysis data artifacts in order to enable many advanced features of code navigation, etc.

A typical pipeline consists of several jobs, each of which generates artifacts and transfers them to subsequent jobs upon completion. Artifacts that are not transfered will be deleted by the pipeline job runner process. The CodeSonar analysis data created in a pipeline job must therefore be transferred (i.e. "relocated") to a persistent storage location outside the pipeline runner environment so that they will remain available after the pipeline runs.

We will refer to the host system for the analysis data persistent storage as an "analysis data server".

• The analysis data server must have enough storage space to accommodate the analysis data produced by the execution of many pipeline jobs.
• The analysis data server must run a CodeSonar launch daemon ("launchd") process so that the analysis data can be shared with your hub. Therefore, you must install CodeSonar on the analysis data server and you must run a CodeSonar launch daemon process there.

2. Create a basic pipeline that can build your code

In order to analyze your code using CodeSonar in a GitLab pipeline, you must first create a working pipeline that can compile and build your code. This requires the following.

• A GitLab repository project containing the source code that you want to analyze.
• A GitLab pipeline job definition file (.gitlab-ci.yaml) which includes instructions for building your code.
• A GitLab pipeline job runner environment which contains all of the necessary build tools and compilers required to build your code.

If you do not already have a working build pipeline, you must create one specific to your code before introducing CodeSonar.

• For instructions, see https://docs.gitlab.com/ee/ci/quick_start/index.html.
• If you already have a working build pipeline, you do not need to create another one: you can add CodeSonar to the existing pipeline at step 4.

3. Install CodeSonar and integration tools in build runner environment

In order to initiate the CodeSonar analysis, you must install CodeSonar in your GitLab runner environment. For Docker-based runners, this involves adding CodeSonar to the Docker image used when building your code. Without Docker, this involves installing CodeSonar on the runner host itself.

Install the following in your GitLab build runner environment

• CodeSonar.
• The CodeSonar GitLab integration package.
• Any compilers or other tools required to build your software project.

CodeSonar and the integration tools may not automatically trust the HTTPS certificates provided by your hub or your GitLab server. You may need to take steps to ensure that the CodeSonar installation in your runner environment will trust the certificates: see the Troubleshooting section for details.

4. Update pipeline job definition to perform CodeSonar analysis

You will need to modify your GitLab pipeline job definition to integrate the CodeSonar build/analysis.

CodeSonar works by creating an internal representation (IR) of your software, then performing a suite of analyses on that representation. The IR creation stage works differently for different source languages:

• C and C++: CodeSonar observes the compiler invocations in your regular software build.

  You must therefore build the CodeSonar project in the same job as your regular software build.

• Java and C#: CodeSonar collects the source code and compiled object files ("binaries": JARs, DLLs, etc.) after your regular software build has taken place: it does not need to observe the build directly.

  You can build the CodeSonar project in the same job as your regular software build, but this is not required: it is sufficient for CodeSonar to have access to the source code and binaries produced by the build.

• If your software project contains multiple programming languages, build the CodeSonar project in the same job as your regular software build.

Modify the GitLab pipeline job you identified in step 2 so that it also builds and analyzes a CodeSonar project based on your code.

You will do this by adding zero or more codesonar build commands, plus a final codesonar analyze command. For full details of the CodeSonar build/analysis command line, see the manual: Using CodeSonar > Building and Analyzing Projects > Command Line Build/Analysis

Make sure your codesonar build and codesonar analyze commands account for the following.

• Authentication credentials for a hub user account that can authorize the analysis. With factory settings, it is sufficient for the account to have the built-in User role. (See Authentication Notes, below.)

• For CodeSonar SaaS, your codesonar analyze command must include -wait and -remote "./saas/*". It must not include -foregound.

• For non-SaaS CodeSonar, your codesonar analyze command must include one of the following.

  • -wait and -remote <analysis-launchd>, where <analysis-launchd> is a suitable remote analysis launch daemon: either one you set up on your analysis data server host, or one that existed previously. <analysis-launchd> will manage both the analysis of your project and the hub interaction with analysis files when the analysis has completed. The command must not include -foregound.

  • -wait and -remote-archive <archive-launchd>, where <archive-launchd> is a suitable remote analysis launch daemon: either one you set up on your analysis data server host, or one that existed previously. The analysis files will be transferred to <archive-launchd> after the analysis completes, and <archive-launchd> will manage all subsequent hub interaction with analysis files. The command must not include -foregound.

  If you are using CodeSonar 7.2 or earlier, -remote-archive is not available. See the note below for further options.

• You may wish to specify -name <analysis-name> to provide a custom analysis name: for example, to relate the analysis to a specific GitLab merge request record.

Authentication Notes

There are two main options for authenticating a codesonar command line.

• -auth password -hubuser <user-name> -hubpwfile <pwfilename>

  Authenticates hub user <user-name> using the password stored in <pwfilename>.

• -auth certificate -hubcert <certfile> -hubkey <privkeyfile>

  Presents the certificate in <certfile> to the hub, using the private key stored in <privkeyfile> to sign its response to the hub's challenge. The authenticated hub user will be the one whose username matches the Subject Common Name specified in the certificate.

You can use GitLab "CI/CD Variables" to provide credentials for authentication. CI/CD Variables can be created and updated on the GitLab project CI/CD Settings page.

For more information about authentication in CodeSonar, see the manual: How CodeSonar Works > CodeSonar Structure > Hub > Authentication and Access Control

Other Notes

The CodeSonar GitLab integration package includes a few script programs which are used to process analysis results and share the results with GitLab. These scripts must be executed using the cspython Python interpreter included with CodeSonar.

• sarif2sast.py

  Converts analysis results from the SARIF format retrieved by codesonar get into GitLab SAST format. It also generates a summary report of warning results and evaluates warning results to conditionally trigger a pipeline failure.

• upload_gitlab_mr_notes.py

  Posts a summary of warning results to your Merge Request discussion.

Execute the pipeline

The typical way to trigger a pipeline is by pushing new code commits to a GitLab merge request. After a successful analysis in the pipeline, you can view your analysis results in GitLab and on the CodeSonar hub. This allows you to automatically scan for newly introduced issues in your code and to verify that known issues have been fixed.

Note for CodeSonar 7.2 and earlier

The -remote-archive option was introduced in CodeSonar 7.3. If you are using an earlier CodeSonar version, or if you do not want to use -remote-archive, you will need to do one of the following.

• Use the -remote option (introduced in CodeSonar 7.1) to specify a persistent remote analysis launch daemon located outside your pipeline. This launch daemon will continue to manage your analysis files and communicate with the hub after the analysis phase is finished.

  or

• Relocate the analysis artifacts to an external analysis data server after the analysis has completed.

  1. Install the GitLab runner software on your analysis data server.

     Configure the runner with a distinctive "tag" so that it can be targeted by the "relocation" pipeline job that you are about to create.

  2. Adjust the build/analysis pipeline job to send the analysis data to your analysis data server.

     Move all of the following (they will all be in the same parent directory).

     • entire project analysis directory (<project_name>.prj_files/)
     • project file (<project_name>.prj)
     • general project configuration file (<project_name>.conf)

  3. Establish a second, "relocation" pipeline job on the analysis data server to receive the analysis artifacts and inform the CodeSonar hub of their new location.

     The relocation job must do the following.

     • Receive the analysis data from the build/analysis pipeline: <project_name>.prj_files/, <project_name>.prj, and <project_name>.conf.

       Use job definition keyword needs:artifacts to receive the data.

     • Execute the codesonar relocate command to inform the hub of the new location.

       See the CodeSonar manual for details: Using CodeSonar > Typical CodeSonar Tasks > Analysis Tasks > Relocate Analysis Files

Further Information

• CodeSonar 7.3 and later (with -remote-archive option), detailed instructions:

  • Analysis data server using -remote-archive option: [Linux and Docker] [Linux] [Windows]

  • Pipeline configuration using -remote or -remote-archive option: [Linux and Docker] [Linux] [Windows]

• CodeSonar 7.2 or earlier (-remote-archive not available):

  • Analysis data server using codesonar relocate: [Linux and Docker] [Linux] [Windows]

  • Pipeline configuration using codesonar relocate: follow instructions for CodeSonar 7.3 and later (-remote-archive option) above, and supplement instrucions with: [Linux and Docker] [Linux] [Windows]

• See also: troubleshooting and tips.

---