codesonar_gerrit_citool.py

codesonar_gerrit_citool.py is a wrapper for running the CodeSonar build/analysis in Gerrit pipelines and performing specified checks on the analysis results.

For some commented usage examples, see the example Jenkinsfiles in the CodeSonar-Gerrit integration package.

There are four general forms.

Command Form Description
build Accumulate different components toward a CodeSonar project without analyzing the project.
analyze Build a CodeSonar project, run the analysis and output the analysis results to a hub, evaluate the analysis results.
check Check the results of a previously completed analysis.
update Update analysis properties and warning annotations for a previously completed analysis.

Example command lines in the remainder of this page are written like the following.

$CSONAR/codesonar/bin/cspython $CSONAR_GERRIT/codesonar_gerrit_citool.py <...>

where:

If you want to use these environment variables you will need to set them to appropriate values: they are not set automatically.

Table of Contents

build : Accumulate Project Components Without Analyzing

Use the build form of the command to accumulate components toward a CodeSonar project, but not finalize or analyze the project.

This form has similar functionality to the regular codesonar build command, with some key differences:

The general form of the command is:

$CSONAR/codesonar/bin/cspython $CODESONAR_GERRIT/tools/codesonar_gerrit_citool.py build <project_file> \
   [<GLOBAL_OPTS>] [<CS_BUILD_OPTS>] <hub_address> <command>

Where:

For more information about the <project_file>, <hub_address>, <CS_BUILD_OPTS>, and <command> elements, see:
MANUAL: Using CodeSonar > Building and Analyzing Projects > Building and Analyzing a CodeSonar Project

analyze : Build and Analyze Project, Evaluate Analysis Results

Use the analyze form of the command to build and analyze a CodeSonar project, then run a specified set of checks with respect to a specified subset of the analysis results.

In most respects this form extends the functionality of the regular codesonar analyze command. However, interactive functionality, such as prompting for a password, is not available.

The general form of the command is:

$CSONAR/codesonar/bin/cspython $CODESONAR_GERRIT/tools/codesonar_gerrit_citool.py analyze <project_file> \
   [<GLOBAL_OPTS>] [<CS_ANALYZE_OPTS>] [<SEARCH_OPTS>] [<CHECK_OPTS>] <hub_address> [<command>]

Where:

For more information about the <project_file>, <hub_address>, <CS_ANALYZE_OPTS>, and <command> elements, see:
MANUAL: Using CodeSonar > Building and Analyzing Projects > Building and Analyzing a CodeSonar Project

check : Check Results of an Existing Analysis

Use the check form of the command to perform checks on an existing analysis. This can be an analysis performed with $CSONAR/codesonar/bin/cspython $CODESONAR_GERRIT/tools/codesonar_gerrit_citool.py analyze, or one performed with the standard codesonar analyze.

The general form of the command is:

$CSONAR/codesonar/bin/cspython $CODESONAR_GERRIT/tools/codesonar_gerrit_citool.py check <analysis_ref> \
     [<GLOBAL_OPTS>] [<SEARCH_OPTS>] [<CHECK_OPTS>] <hub_address>

Where:

Specifying <analysis_ref> for check and update Commands

The codesonar_gerrit_citool.py check and codesonar_gerrit_citool.py update commands take an <analysis_ref> argument that specifies the analysis of interest. The <analysis_ref> can have any of the following forms.

For more information, see the CodeSonar manual.

update : Update an Existing Analysis

Use the update form of the command to update information about an existing analysis and its warnings.

The general form of the command is:

$CSONAR/codesonar/bin/cspython $CODESONAR_GERRIT/tools/codesonar_gerrit_citool.py update *<analysis_ref> \
     [<GLOBAL_OPTS>] [<SEARCH_OPTS>] [<UPDATE_OPTS>] <hub_address>

Where:


Command Options

The various forms of the codesonar_gerrit_citool.py command have a large number of options.


Global Options <GLOBAL_OPTS>

The global options can be used with all forms of the codesonar_gerrit_citool.py command.

-tmpdir TEMP

Save temporary files, including temporary files used for authentication, to directory TEMP.

standard authentication options

You can authenticate using the standard CodeSonar hub authentication options:

For details, see MANUAL: How CodeSonar Works > Hub > Authentication and Access Control

-hubbearervar VARIABLE

Get the hub bearer token from the environment variable whose name is VARIABLE. This will implicitly create a temporary file containing the token and provide it to the standard CodeSonar -hubbearerfile option. The temporary file will be deleted after program execution.

-hubcertvar VARIABLE

Get the hub user certificate as a value from the environment variable whose name is VARIABLE. This will implicitly create a temporary file containing the certificate and provide it to the standard CodeSonar -hubcert option. The temporary file will be deleted after program execution.

-hubkeyvar VARIABLE

Get the hub user certificate private key as a value from the environment variable whose name is VARIABLE. This will implicitly create a temporary file containing the key and provide it to the standard CodeSonar -hubkey option. The temporary file will be deleted after program execution.

-hubpwvar VARIABLE

Get the hub user password from the environment variable whose name is VARIABLE. This will implicitly create a temporary file containing the password and provide it to the standard CodeSonar -hubpwfile option. The temporary file will be deleted after program execution.

-cs-environment-format VARTEMPLATE

Specifies an alternate environment variable format for recognized CodeSonar variables when -use-environment is enabled. VARTEMPLATE defines a variable name pattern containing one ? character. The default template is "CODESONAR_?", which will recognize variable names like CODESONAR_HUBUSER, etc.

-use-environment

When this is specified, codesonar_gerrit_citool.py will make use of variables and system information from the local environment.

Note: If you do not specify -use-environment, ensure that your analyze command explicitly sets the important analysis properties to appropriate values.

Options that take a VARIABLE argument, such as -hubpwvar, do not depend on this option.


codesonar build Options <CS_BUILD_OPTS>

Any option that can be used with the standard codesonar build command can also be used with the build form of the codesonar_gerrit_citool.py command.

For more information, see MANUAL: Using CodeSonar > Building and Analyzing Projects > Command Line Build/Analysis


codesonar analyze Options <CS_ANALYZE_OPTS>

Any option that can be used with the standard codesonar analyze command can also be used with the analyze form of the codesonar_gerrit_citool.py command.

For more information, see MANUAL: Using CodeSonar > Building and Analyzing Projects > Command Line Build/Analysis


Search Options <SEARCH_OPTS>

The search options can be used with the analyze, check, and update forms of the codesonar_gerrit_citool.py command.

Use these options to restrict the following to a particular subset of the warnings produced by the analysis.

If no search options are specified, checks/updates are applied to all warnings produced by the analysis, and output files include information about all warnings.

-comparison-analysis COMP_ANALYSIS

Apply checks/updates only to warnings that are issued by the current analysis but not by COMP_ANALYSIS. You can specify COMP_ANALYSIS in any of the following ways.

Cannot be specified multiple times.

-warning-visibility FILTER

Apply checks/updates only to warnings that satisfy FILTER: an existing saved warning search. The available searches can be viewed on the Warnings tab of the hub Saved Searches page.

Cannot be specified multiple times.

-warning-search WSEARCH_EXP

Apply checks/updates only to warnings that satisfy WSEARCH_EXP: a Warning Search Language expression. If <asearch_exp> contains spaces or other special characters, it must be appropriately quoted or escaped.

This expression can use parameters that you define with -search-param.

MANUAL: Using CodeSonar > GUI Reference > Searching > Warning Search Language

Cannot be specified multiple times.

-search-param PARAM VALUE

Define a parameter to use in analysis or warning search expressions for the -comparison-analysis and -warning-search options.

Can be specified multiple times.


Check Options <CHECK_OPTS>

The check options can be used with the analyze and check forms of the codesonar_gerrit_citool.py command.

-check CHECK_METHOD [+CHECK_ARG...]

Check the analysis results using CHECK_METHOD; return a nonzero exit code (as specified by -fail-code) if the check fails. The check will be applied only to the warnings that satisfy all conditions imposed by the specified search options.

Note that the default alert check will be applied if you do not explicitly specify any -check alert checks.

See codesonar_gerrit_ci.py: Check Methods for check for full details.

-dump-sarif SARIF_FILE

Save analysis results in SARIF format to SARIF_FILE. Only the warnings that satisfy all conditions imposed by the specified search options will be saved.

-fail-code EXIT_CODE

If an analysis check fails, return EXIT_CODE. Defaults to 1.

-polling-timeout INACTIVITY_SECONDS

An inactivity timeout of INACTIVITY_SECONDS will be applied to the hub status polling that is performed when the analysis is executed without -wait or -foreground. If there is no analysis activity for longer than INACTIVITY_SECONDS, the program will exit with a nonzero code.

Defaults to 14400 (4 hours).

If -wait or -foreground is specified, this option has no effect.

-summary REPORT_FILE

Save a summary of analysis results to REPORT_FILE.

The contents of the summary report are determined by the specified search options and checks.

The format of the summary report is determined by the file name:


Gerrit-Specific Options <GERRIT_OPTS>

The Gerrit-specific options can be used with the analyze and check forms of the codesonar_gerrit_citool.py command.

<GERRIT_OPTS>: Details


Update Options <UPDATE_OPTS>

The update options can be used with the update form of the codesonar_gerrit_citool.py command.

For all of these updates, the authorizing hub user must have ANALYSIS_ANNOTATE permission for the specified analysis. Any additional permission requirements for individual updates are noted in the option details.

Note on eligible warnings. If one or more search options are specified, warning updates are only applied to warnings that were issued by the specified analysis and meet all the specified constraints. Otherwise, warning updates are applied to all warnings issued by the specified analysis.
Search options do not affect the behavior of -set-property.

For more information, see the CodeSonar manual.

-set-property NAME VALUE

Set user-assigned analysis property NAME to VALUE. If the analysis already has a user-assigned analysis property named NAME, its value will be updated.

-set-priority PRIORITY

For each eligible warning, set the warning priority to PRIORITY.

PRIORITY is the priority name. For a list of available priority names, see the hub "Manage Priorities" page: http://<hub_location>/priorities.html.

If the hub has no priority whose name matches PRIORITY the command will attempt to create one, failing if the authorizing hub user does not have G_PRIORITY_ADD permission.

-set-state STATE

For each eligible warning, set the warning state to STATE.

The authorizing hub user must have ANALYSIS_ANNOTATE permission for the specified analysis.

STATE is the state name. For a list of available state names, see the hub "Manage States" page: http://<hub_location>/states.html.

If the hub has no state whose name matches STATE the command will attempt to create one, failing if the authorizing hub user does not have G_STATE_ADD permission.

-set-finding FINDING

For each eligible warning, set the warning finding to FINDING.

FINDING is the finding name. For a list of available finding names, see the hub "Manage Findings" page: http://<hub_location>/findings.html.

If the hub has no finding whose name matches FINDING the command will attempt to create one, failing if the authorizing hub user does not have G_FINDING_ADD permission.

-set-owner OWNER

For each eligible warning, set the warning owner to the hub user whose username is OWNER.

The OWNER must have ANALYSIS_OWN_WARNINGS permission for the specified analysis.

-add-note COMMENT

For each eligible warning, add COMMENT to the warning notes.


Appendix

Environment Variables for -use-environment

When the -use-environment option is specified, various environment variables are recognized and used as command defaults or stored as user-defined analysis properties. These environment variables can be divided into three groups: those set by the user, those set by the Gerrit Trigger Plugin for Jenkins, and those set by Jenkins Pipeline.

CodeSonar Environment Variables Set by the User

The table below shows the default CodeSonar environment variable names that will be recognized and used when -use-environment is specified. If you want to use these variables, you will need to define them to appropriate values.

If -cs-environment-format is also specified, codesonar_gerrit_citool.py will instead recognize the variable names produced by replacing the CODESONAR_ according to the specified format. Variables with other prefixes will not be affected.

For example, if you specify -cs-environment-format MYCS_? then the recognized environment variables will be MYCS_HUBUSER, MYCS_HUBPWFILE, and so on.

Environment Variable Usage if -use-environment is specified
CODESONAR_HUBUSER Default value for -hubuser : hub user name.
CODESONAR_HUBPWFILE Default value for -hubpwfile : hub user password file path.
CODESONAR_HUBPWVAL Hub user password. This will implicitly create a temporary file containing the key and provide it to the standard CodeSonar -hubpwfile option. The temporary file will be deleted after program execution. A password provided on the command line with -hubpwfile or -hubpwvar will take precedence over this value.
CODESONAR_HUBCERTFILE Default value for -hubcert : hub user certificate file path.
CODESONAR_HUBKEYFILE Default value for -hubkey : hub user certificate private key file path.
CODESONAR_HUBCERTVAL CodeSonar hub user certificate contents. When this value is used, the command will implicitly create a temporary file containing the token and provide it to the standard CodeSonar -hubcert option. The temporary file will be deleted after program execution.
CODESONAR_HUBKEYVAL Hub user certificate private key. This will implicitly create a temporary file containing the key and provide it to the standard CodeSonar -hubkey option. The temporary file will be deleted after program execution.
CODESONAR_HUBBEARERFILE Default value for hubbearerfile: hub user bearer token file path.
CODESONAR_HUBBEARERVAL CodeSonar hub bearer token. This will implicitly create a temporary file containing the token and provide it to the standard CodeSonar -hubbearerfile option. The temporary file will be deleted after program execution.
CODESONAR_PROJECT_PATH Default value for standard CodeSonar -project option: a string /[<ancestors>]/<project-name>.
CODESONAR_REMOTE_ANALYSIS Default value for standard CodeSonar -remote option: a positive integer (ldgroup ID), negative integer (launch daemon ID), or string (ldgroup path).
CODESONAR_REMOTE_ARCHIVE Default value for standard CodeSonar -remote-archive option: a positive integer (ldgroup ID), negative integer (launch daemon ID), or string (ldgroup path).
CODESONAR_WARNING_VISIBILITY Default value for -warning-visibility : a saved search name or ID.

See the CodeSonar manual for more information on standard build/analysis options:
MANUAL: Using CodeSonar > Command Line Build/Analysis

Useful Environment Variables Set by the Gerrit Trigger Plugin for Jenkins

These variables are set by the Gerrit Trigger Plugin for Jenkins: you should not need to modify them.

Some of these variables are recognized and automatically used when -use-environment is specified.

Environment Variable Default value for option ...
(when -use-environment is specified)
Stored as user-defined analysis property named ...
(when -use-environment is specified)
GERRIT_BRANCH -gerrit-target-branch change_target, gerrit_target_branch
GERRIT_CACERT -gerrit-cacert (none)
GERRIT_CHANGE_ID -gerrit-change change_id, gerrit_change, gerrit_change_id
GERRIT_CHANGE_NUMBER (none) gerrit_change_num
GERRIT_CHANGE_URL backup -gerrit-url default if GERRIT_URL is not defined change_url, gerrit_change_url
GERRIT_EVENT_TYPE (none) gerrit_event_type
GERRIT_PATCHSET_REVISION -gerrit-patchset change_revision, gerrit_patchset, gerrit_patchset_commit
GERRIT_PATCHSET_NUMBER (none) gerrit_patchset_num
GERRIT_PROJECT -gerrit-project gerrit_project
GERRIT_PWFILE -gerrit-pwfile (none)
GERRIT_REFSPEC (none) gerrit_refspec
GERRIT_REVIEW_LABEL -gerrit-label (none)
GERRIT_URL -gerrit-url gerrit_url
GERRIT_USERNAME -gerrit-user (none)

Useful Environment Variables Set by Jenkins Pipeline

If you are using Jenkins Pipeline, it will set a number of useful environment variables. Some of these variables are recognized and their values automatically recorded -use-environment is specified.

Environment Variable Stored as user-defined analysis property named ...
(when -use-environment is specified)
BUILD_NUMBER ci_job, jenkins_build, jenkins_build_num
EXECUTOR_NUMBER jenkins_agent_executor_num
NODE_NAME jenkins_agent

For more information about these and other environment variables made available by Jenkins Pipeline, see: https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#using-environment-variables

User-Defined Analysis Properties

As shown in the table above, many environment variables are stored as user-defined analysis properties when -use-environment is specified.

These properties can be used in analysis search language expressions to find the CodeSonar analysis that corresponds to a particular Gerrit change or Jenkins build. For example, the example Jenkinsfile at csonar-ci/cso-gerrit/examples/IncrExample/Jenkinsfile includes the following fragment, in which an update command specifies the analysis of interest using an analysis search language expression. This expression will match an analysis whose gerrit_change_id and gerrit_patchset_commit properties have values matching those recorded in the specified environment variables.

    // ...
    script {
        // Find the last analysis for this change and update it with warning annotations and analysis properties:
        sh '''
            "$CSONAR/codesonar/bin/cspython" "$CSONAR_GERRIT/tools/codesonar_gerrit_citool.py" \
            update \
           "?((gerrit_change_id=$GERRIT_CHANGE_ID) & (gerrit_patchset_commit=$GERRIT_PATCHSET_REVISION))" \
            // ...

If your analyze command does not specify -use-environment, these properties are not set automatically. In this case, you will need to explicitly set any properties that you will later rely on to coordinate CodeSonar analyses, Jenkins builds, and Gerrit changes. You do not need to use the same property names as those discussed here, although you may find it convenient to do so.

It is generally most convenient to do this in your analyze command, using the standard CodeSonar -property option. This ensures that the properties are available for any subsequent analysis searches or other uses. For example:

$CSONAR/codesonar/bin/cspython $CODESONAR_GERRIT/tools/codesonar_gerrit_citool.py analyze \
   <...> \
   -property change_id "$GERRIT_CHANGE_ID" \
   -property change_revision "$GERRIT_PATCHSET_REVISION" \
   -property change_target "$GERRIT_BRANCH" \
   <...> 

If you need to add or modify an analysis property after the analyze command has run, you can use an update command with the -set-property option. Note that if you are updating a historical analysis the environment variables may not have the values you require, so you may need to obtain the property values from some other source.

$CSONAR/codesonar/bin/cspython $CODESONAR_GERRIT/tools/codesonar_gerrit_citool.py update \
   <...> \
   -set-property change_id "$GERRIT_CHANGE_ID" \
   -set-property change_revision "$GERRIT_PATCHSET_REVISION" \
   -set-property change_target "$GERRIT_BRANCH" \
   <...>