Create and Install a Relocating Analysis Data Server on Linux (CodeSonar 7.2 and earlier)

This page describes how to set up a relocating analysis data server to manage your CodeSonar analysis files after the analysis has completed. The analysis data server will store these files and use them to service hub requests for information such as source file listings and procedure-granularity metrics.

These instructions are designed for use with CodeSonar 7.2 and earlier. If you are using CodeSonar 7.3 or later, we recommend that you use a standard analysis data server in combination with codesonar analyze -remote-archive instead.

If you are using CodeSonar SaaS, or if your hub already has suitable associated remote analysis launch daemons, you do not need to set up a separate analysis data server.

We also provide instructions to create and install a relocating analysis data server on other systems:

• Linux with Docker
• Microsoft Windows

Prerequisites

• A CodeSonar hub.
• A suitable machine for running the analysis data server.

Overview

There are three stages:

• A. Prepare the hub
• B. Prepare the analysis data server host machine
• C. Start a remote analysis launch daemon and a GitLab Runner instance

A. Prepare the hub

1. Log in to your hub as Administrator, or as another user with administrative privileges.

2. Create a new launchd group to contain the launch daemons that you will create for your analysis data server.

   For details, see the manual: Settings > Other Links > Analysis Cloud > Create New Launchd Group.

   The steps below will assume that the launchd group path name is '/analysis-data-server'.

3. Create a hub user account that you can use to run the launch daemons and associated pipeline jobs. Make sure the user has sufficient permissions: with factory settings, it is sufficient to assign the User and Enabled roles

B. Prepare the analysis data server host machine

1. If you have not already done so, identify a suitable host machine for the analysis data server.

   • This can be a physical machine, a virtual machine, or a container.
   • It will not need to listen on any ports, and does not need a stable host name.
   • It will make outgoing network connections to your CodeSonar hub and to your GitLab server.
   • It will need plenty of disk space where you can store analysis data files.

2. Log in to the host machine as root.

   The next few steps assume that you have super user privileges. Depending on your system you may need to use sudo to execute some of the commands.

3. Create a new CI user account. This account will run the CodeSonar launch daemon process.

   These instructions will assume that the CI user name is csonar_ci.

   CI_USER_UID=1001
   CI_USER=csonar_ci
   CI_USER_GID=1001
   CI_USER_GROUP=csonar_ci
   CI_USER_HOME=/home/$CI_USER
   
   groupadd -g $CI_USER_GID $CI_USER_GROUP || true
   useradd -g $CI_USER_GROUP -u $CI_USER_UID -d $CI_USER_HOME -ms /bin/bash $CI_USER
   

4. Remove the .bash_logout file for the new user: it tries to clear the console and causes the GitLab Runner shell process to fail. See https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information.

   rm -fv $CI_USER_HOME/.bash_logout
   

5. Create a base directory for saving analysis data.

   Analysis data can take plenty of space depending on the size of your code. You will need to arrange to clean up old data periodically using a cron job or similar.

   CI_USER=csonar_ci
   CI_USER_GROUP=csonar_ci
   PIPELINE_ANALYSIS_DIR=/srv/codesonar/analysis_data/pipelines
   
   mkdir -p "$PIPELINE_ANALYSIS_DIR"
   chown $CI_USER:$CI_USER_GROUP "$PIPELINE_ANALYSIS_DIR"
   

6. Install CodeSonar.

   1. Download the CodeSonar install archive (e.g. codesonar-7.3p0.20230330-x86_64-pc-linux.tar.gz).

   2. Extract the archive.

      cd /opt
      tar -xzf /path/to/codesonar-7.3p0.20230330-x86_64-pc-linux.tar.gz
      

   3. Activate the installation.

      codesonar-7.3p0/codesonar/bin/codesonar activate
      

      You will be prompted to accept the CodeSonar license.

7. Install Git.

   Although you won't use Git for analysis data server tasks, it is required by the GitLab Runner.

   On Debian-based Linux:

   apt-get install git
   

8. Install GitLab Runner.

   CI_RUNNER_DIR=/usr/local/bin
   CI_RUNNER_BINARY=gitlab-runner-linux-amd64
   CI_RUNNER=$CI_RUNNER_DIR/gitlab-runner
   
   mkdir -p $CI_RUNNER_DIR
   curl -L --output $CI_RUNNER "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/$CI_RUNNER_BINARY"
   chmod +x $CI_RUNNER
   

   For more information: https://docs.gitlab.com/runner/install/.

C. Start a remote analysis launch daemon and a GitLab Runner instance

1. Log in to the host machine as your CI user (csonar_ci).

2. Register the GitLab Runner instance.

   The runner instance must be registered exactly once. If the host machine is restarted, the runner will not need to be re-registered. Note that you can have many runners on a single machine, but each runner must have a separate home directory.

   You will need to modify some or all of the variable settings.

   Variable	Setting
   CI_RUNNER	Your GitLab Runner binary.
   CI_SERVER_URL	GitLab server URL.
   CI_RUNNER_REG_TOKEN	Provided by your GitLab site under the "CI/CD" Runner settings.
   CI_RUNNER_DESCRIPTION	Name of runner to display in GitLab UI.
   CI_RUNNER_TAGS	Used by the pipeline to identifiy the runner. In this example, the tags "Linux" and "GCC" indicate that this runner has GCC installed on a Linux platform.
   CI_RUNNER_HOMEDIR	Base directory where runner files should be stored on the local machine.
   CI_RUNNER_WORKDIR	Directory where runner will execute jobs.
   CI_RUNNER_CONFIGDIR	Directory where runner configuration files will be saved.
   CI_RUNNER_CONFIG	Path to runner configuration file. This file stores the runner instance settings.

   CI_RUNNER=/usr/local/bin/gitlab-runner
   CI_SERVER_URL=https://gitlab.example.com
   CI_RUNNER_REG_TOKEN=abc123def
   CI_RUNNER_DESCRIPTION=linux_ci_runner_01
   CI_RUNNER_TAGS=Linux,codesonar_analysis_server
   CI_RUNNER_HOMEDIR=$HOME/gitlab-runner_01
   CI_RUNNER_WORKDIR=$CI_RUNNER_HOMEDIR/work
   CI_RUNNER_CONFIGDIR=$CI_RUNNER_HOMEDIR/config
   CI_RUNNER_CONFIG="$CI_RUNNER_CONFIGDIR/config.toml"
   
   mkdir -p $CI_RUNNER_CONFIGDIR && mkdir -p $CI_RUNNER_WORKDIR
   
   $CI_RUNNER register \
    --config "$CI_RUNNER_CONFIG" \
    --non-interactive \
    --url "$CI_SERVER_URL" \
    --registration-token "$CI_RUNNER_REG_TOKEN" \
    --description "$CI_RUNNER_DESCRIPTION" \
    --tag-list "$CI_RUNNER_TAGS" \
    --executor shell \
    --shell bash \
    --output-limit 10000
   

3. Start the CodeSonar launch daemon.

   You will need to modify some or all of the variable settings.

   Variable	Setting
   CSONAR	The path to your CodeSonar installation.
   CSONAR_HUB	The location of your CodeSonar hub (protocol://host:port).
   CSONAR_HUBUSER	The hub user account you created in part A.
   CSLAUNCHD_GROUP	The launch daemon group you created in part A.
   CSLAUNCHD_KEY	A unique identifier to distinguish this launch daemon from any others you may wish to start on the analysis data server.

   CSONAR=/opt/codesonar-7.3p0 
   CSONAR_HUB=https://codesonar.example.com:7340
   CSONAR_HUBUSER=cshub_ci
   CSLAUNCHD_GROUP=/analysis-data-server
   CSLAUNCHD_KEY=remote-archive
   
   # This command may prompt you to trust the hub's HTTPS certificate the first time you run it:
   $CSONAR/codesonar/bin/codesonar install-launchd \
      -auth password -hubuser $CSONAR_HUBUSER \
      -launchd-group $CSLAUNCHD_GROUP \
      -launchd-key $CSLAUNCHD_KEY \
      $CSONAR_HUB
   

   When prompted, enter the hub user account password.

4. Start the GitLab Runner instance.

   You will need to arrange for the GitLab Runner to start whenever the system starts.

   CI_RUNNER=/usr/local/bin/gitlab-runner
   CI_RUNNER_HOMEDIR=$HOME/gitlab-runner_01
   CI_RUNNER_WORKDIR=$CI_RUNNER_HOMEDIR/work
   CI_RUNNER_CONFIGDIR=$CI_RUNNER_HOMEDIR/config
   CI_RUNNER_CONFIG="$CI_RUNNER_CONFIGDIR/config.toml"
   
   $CI_RUNNER run --config "$CI_RUNNER_CONFIG" --working-directory "$CI_RUNNER_WORKDIR"
   

Next Steps

If you are setting up a GitLab pipeline to perform CodeSonar analysis, go on to step C to create and install a pipeline build runner.

Notes

Modifications for certificate authentication (HTTPS hubs only)

If you need to authenticate your launch daemon with a certificate instead of with a password in part C, make the following changes.

1. Before you start the launch daemon, generate a hub user certificate and private key.

   • If you already have a suitable hub user certificate and private key, you do not need to generate new ones. Make sure that CSONAR_HUBCERT and CSONAR_HUBKEY are set to the locations of the certificate and key, respectively, and then go on to the next step.

   Make any necessary changes to variable settings before executing.

   Variable	Setting
   CSONAR	Your CodeSonar installation.
   CSONAR_HUB	Your hub location.
   CODESONAR_DATA	The analysis data directory you created in part B.
   CSONAR_CERTDIR	The directory where your generated certificates will be saved. This directory must already exist.
   CSONAR_HUBUSER	Your hub user account. Note that the command below uses this as both the username of the account that is authorizing certificate generation and the username of the account that is the subject of the certificate.
   CSONAR_HUBCERT, CSONAR_HUBKEY	Output files for the user authentication certificate and private key, respectively. These files will become inputs to codesonar install-launchd in the following step.

   CSONAR=/opt/codesonar-7.3p0
   CSONAR_HUB=https://codesonar.example.com:7340
   CODESONAR_DATA=/srv/codesonar
   CSONAR_CERTDIR=$CODESONAR_DATA/certificates
   CSONAR_HUBUSER=cshub_ci
   CSONAR_HUBCERT=$CSONAR_HUBUSER.cert
   CSONAR_HUBKEY=$CSONAR_HUBUSER.key
   
   $CSONAR/codesonar/bin/codesonar generate-hub-cert \
      -foruser "$CSONAR_HUBUSER" \
      -auth password \
      -hubuser "$CSONAR_HUBUSER" \
      -out "$CSONAR_HUBCERT" \
      -outkey "$CSONAR_HUBKEY" \
      "$CSONAR_HUB"
   

   When prompted, enter the hub user account password.

2. When you run the codesonar install-launchd command to start the launch daemon, replace

   -auth password -hubuser $CSONAR_HUBUSER \
   

   with

   -auth certificate -hubcert "$CSONAR_HUBCERT" -hubkey "CSONAR_HUBKEY" \
   

---