Create and Install an Analysis Data Server on Linux with Docker

This page describes how to set up an analysis data server to manage your CodeSonar analysis files after the analysis has completed. The analysis data server will store these files and use them to service hub requests for information such as source file listings and procedure-granularity metrics.

If you are using CodeSonar SaaS, or if your hub already has suitable associated remote analysis launch daemons, you do not need a separate analysis data server.

Note that you do not necessarily need to use a Docker container for your analysis data server, even if your pipeline is running in a Docker container. The most important aim is to have a "persistent" analysis data server host that will not be destroyed when your pipeline finishes.

We also provide instructions to create and install an analysis data server on other systems:

• Linux (without Docker)
• Microsoft Windows

Prerequisites

• A CodeSonar hub.
• A suitable host machine for the Docker container.
  • This can be a physical machine or a virtual machine.
  • It must have Docker installed.
  • It will not need to listen on any ports, and does not need a stable host name.
  • It will make outgoing network connections to your CodeSonar hub.
  • It will need plenty of disk space where you can store analysis data files.

Overview

There are three stages:

• A. Prepare the hub
• B. Prepare the host machine
• C. Create Docker images

A. Prepare the hub

1. Log in to your hub as Administrator, or as another user with administrative privileges.

2. Create a new launchd group to contain the launch daemons that you will create for your analysis data server.
   In the hub GUI:

   1. Click Settings in the quick navigation menu at top right.
      The hub Settings page will open.
   2. Select the Other Links tab.
   3. Click the Analysis Cloud link.
      The hub Analysis Cloud page will open.
   4. Click the Create New Launchd Group link (under the page heading) and use the form to create a new launchd group.
      • These instructions will assume that the launchd group path name is /analysis-data-server.

3. Create a hub user account that you can use to run the launch daemons and associated pipeline jobs.
   In the hub GUI:

   1. Go back to the hub Settings page.
   2. Select the User Administration tab.
   3. Click the Create Account link.
      The hub Create Account page will open.
   4. Use the Create Account form to create a new hub user account.
      • Make sure the user has sufficient permissions: with factory settings, it is sufficient to assign the User and Enabled roles.
      • These instructions will assume that the hub user name is cshub_ci.

B. Prepare the host machine

1. If you have not already done so, identify a suitable host machine for the analysis data server container.

2. Log in to the host machine as a user with root access. Depending on your local system, you may need to use sudo with some or all of the commands in this section.

3. If Docker Engine is not already installed, use your package manager to install it.

   See https://docs.docker.com/engine/install/#server for more information

4. Create a new analysis data service user account.

   These instructions will assume that the CI user name is csonar_ci.

5. Ensure you have a user that can build and run Docker containers (don't run the Docker containers as root).

   If necessary, create another new user that has permission to execute Docker commands. You will need to follow instructions provided by Docker for how to configure this Docker user account. Docker user account configuration is described in the Docker setup "Linux post-installation steps". https://docs.docker.com/engine/install/linux-postinstall/

   These instructions will assume that the Docker user name is docker_ci.

6. Create a base directory for saving analysis data.

   Analysis data can take plenty of space depending on the size of your code. When you create the CodeSonar launch daemon you will provide a "quota" to help manage disk space use.

   The directory should be owned by the analysis data user account you previously created (csonar_ci).

   CI_USER=csonar_ci
   CI_USER_GROUP=csonar_ci
   HOST_ANALYSIS_DATA=/srv/codesonar/analysis_data
   
   mkdir -p "$HOST_ANALYSIS_DATA"
   chown $CI_USER:$CI_USER_GROUP "$HOST_ANALYSIS_DATA"
   

7. Log in to the host machine as the Docker user (docker_ci).

8. Download and extract the CodeSonar-GitHub integration tools package.

   1. Download the CodeSonar-GitHub integration tools package (e.g. codesonar-github-integration-1.3p0.tar.gz).

   2. Change to a suitable user directory.

      The integration tools will be used for creating a Docker container, but will not be used directly by service processes, so they do not need to be installed in a system directory.

   3. Extract the package.

      tar -xzf /path/to/codesonar-github-integration-1.3p0.tar.gz
      

   Some example scripts are found in the examples/analysis-server.remote.docker/ subdirectory of the integration tools package. We will use these scripts further below.

9. [HTTPS hubs only] Download a copy of your hub's hub server certificate. We will use this copy to ensure that the CodeSonar command line tools in your Docker image will trust your hub.

   • If your CodeSonar hub uses plain HTTP (and not secure HTTPS), skip this step.

   • Otherwise:

     1. Download the hub server certificate from the hub Configure HTTPS page.

        See the Troubleshooting document for additional information about how to download this file.

     2. Save the certificate to your local machine in Base-64 ASCII text format (often called "PEM" format). A typical name for saving the certificate file is "cacert.pem".

C. Create Docker images

We will create a sequence of Docker image layers for our container. The first Docker image will be a base Linux system with CodeSonar installed. The second Docker image will contain a dedicated user account for running the launch daemon, and it will be configured to run the launch daemon upon startup.

1. Log in to the host machine as the Docker user (docker_ci).

2. Create a base Docker image containing CodeSonar.

   1. Change directory to <extract-dir>/distro-image, where <extract-dir> is the directory where you extracted the CodeSonar-GitHub integration tools package.

   2. Save a copy of the CodeSonar installer archive into <extract-dir>/distro-image. The name of the installer archive be something like codesonar-8.0p0.20231117-x86_64-pc-linux.tar.gz (version number and datestamp will vary).

   3. [HTTPS hubs only] Copy your downloaded hub server certificate to this directory.

      cp <path-to>/cacert.pem .
      

   4. Build the base Docker image.

      Script variables. You may need to adjust one or more variable settings before executing the script.

      Variable	Setting
      CODESONAR_PACKAGE	The name of the CodeSonar installer archive file located in <extract-dir>/distro-image.
      CODESONAR_HUB_CACERT	For HTTPS hubs, the path to the hub server certificate that you downloaded. For HTTP hubs, leave the setting unchanged.

      Script. Adjust variable settings as described in the table above, then execute this script to build the image.

      CODESONAR_PACKAGE=codesonar-8.0p0.20231117-x86_64-pc-linux.tar.gz
      CODESONAR_IMAGE=codesonar_launchd
      CODESONAR_IMAGE_VERSION=latest
      CODESONAR_HUB_CACERT=cacert.pem
      BASE_IMAGE=ubuntu:18.04
      
      docker build --tag $CODESONAR_IMAGE:$CODESONAR_IMAGE_VERSION \
            --build-arg BASE_IMAGE=$BASE_IMAGE \
            --build-arg CODESONAR_PACKAGE=$CODESONAR_PACKAGE \
            --build-arg CODESONAR_HUB_CACERT=$CODESONAR_HUB_CACERT \
            --build-arg TELEMETRY=1 \
            --build-arg HUB=0 \
            --build-arg JAVA_ANALYSIS=0 \
            --build-arg PYTHON_ANALYSIS=0 \
            --build-arg ECLIPSE=0 \
            .
      

3. Create a Docker image configured for a CodeSonar launch daemon.

   We will use the CodeSonar base image and add launch daemon configuration.

   1. Change directory to <extract-dir>/examples/analysis-server.remote.docker, where is the directory where you installed the CodeSonar-GitHub integration tools package.

   2. Check the directory contents. The following files should be present.

      • Dockerfile: an image built from this Dockerfile will execute the run-cslaunchd.sh script.
      • run-cslaunchd.sh

   3. Build the image.

      Script variables. You may need to adjust one or more variable settings before executing the script.

      Variable	Setting
      CI_USER	The username of the analysis data service user account that you created in part B.

      Script. Adjust variable settings as described in the table above, then execute this script to build the image.

      ANALYSIS_SERVER_IMAGE=codesonar_remote_analysis_server
      ANALYSIS_SERVER_IMAGE_VERSION=latest
      CI_USER=csonar_ci
      CI_USER_HOME=/home/$CI_USER
      CI_USER_UID=$(id -u $CI_USER)
      CI_USER_GID=$(id -g $CI_USER)
      CI_USER_GROUP=$(id -ng $CI_USER)
      CODESONAR_IMAGE=codesonar_launchd
      CODESONAR_IMAGE_VERSION=latest
      
      docker image build \
          --build-arg BASE_IMAGE=$CODESONAR_IMAGE:$CODESONAR_IMAGE_VERSION \
          --build-arg CI_USER=$CI_USER \
          --build-arg CI_USER_HOME=$CI_USER_HOME \
          --build-arg CI_USER_UID=$CI_USER_UID \
          --build-arg CI_USER_GID=$CI_USER_GID \
          --build-arg CI_USER_GROUP=$CI_USER_GROUP \
          -t $ANALYSIS_SERVER_IMAGE:$ANALYSIS_SERVER_IMAGE_VERSION \
          .
      

4. Set up hub credentials.

   If your hub has HTTPS enabled, we recommend using certificate-based authentication as described here. If your hub is HTTP-only, or does not permit certificate-based authentication, you will need to modify this step to set up password-based authentication instead.

   Execute the following to start a temporary Docker container that generates a hub certificate and saves it to a Docker volume. This volume will be used when starting the launch daemon container later.

   1. Set up variables to use in the subsequent steps.

      Script variables. You may need to adjust one or more variable settings before executing the script.

      Variable	Setting
      CODESONAR_HUB	The location of your CodeSonar hub (protocol://host:port).
      CODESONAR_HUBUSER	The username of the analysis data service user account that you created in part A.

      Script. Adjust variable settings as described in the table above, then execute this script to set the variable values.

      ANALYSIS_SERVER_IMAGE=codesonar_remote_analysis_server
      ANALYSIS_SERVER_IMAGE_VERSION=latest
      ANALYSIS_SERVER_USER_VOL=$ANALYSIS_SERVER_IMAGE.config
      CODESONAR_HUB=https://example.codesonar.com:7340
      CODESONAR_HUBUSER=cshub_ci
      CI_USER_HOME=/home/csonar_ci
      CS_USER_HOME=$CI_USER_HOME/cs_home
      CSONAR=/opt/codesonar
      

   2. Create the volume that will contain the hub user certificate files.

      docker volume create $ANALYSIS_SERVER_USER_VOL
      

   3. Start the temporary container.

      docker container run \
          -v $ANALYSIS_SERVER_USER_VOL:$CS_USER_HOME \
          --env CS_USER_HOME=$CS_USER_HOME \
          --rm \
          -it \
          $ANALYSIS_SERVER_IMAGE:$ANALYSIS_SERVER_IMAGE_VERSION \
          $CSONAR/codesonar/bin/codesonar generate-hub-cert -foruser "$CODESONAR_HUBUSER" "$CODESONAR_HUB"
      

      You will be prompted to enter a hub username and password to authorize certificate creation. This does not have to be the same as the CODESONAR_HUBUSER hub user account.

5. Start the analysis data server container.

   Script variables. You may need to adjust one or more variable settings before executing the script.

   Variable	Setting
   HOST_ANALYSIS_DATA	A path to a directory on your Docker host machine. This directory will be "bind mounted" into the container.
   CODESONAR_ANALYSIS_DATA	The home directory for your launch daemon: should be a subdirectory of CODESONAR_DATA. Data for each analysis managed by the launch daemon will be stored under a subdirectory of this directory. The launch daemon will be responsible for creating this directory.
   CODESONAR_ANALYSIS_DATA_MAX_MB	The maximum permitted size of CODESONAR_ANALYSIS_DATA.
   CODESONAR_HUB	The location of your CodeSonar hub (protocol://host:port).
   CODESONAR_HUBUSER	The username of the hub user account that you created in part A.
   CODESONAR_LAUNCHD_GROUP	The launchd group you created in part A.
   CODESONAR_LAUNCHD_KEY	A unique identifier to distinguish this launch daemon from any others you may wish to start on the analysis data server.
   CS_USER_HOME	This variable is recognized by CodeSonar and influences hub authentication commands. Do not use the same directory as CI_USER_HOME.
   ANALYSIS_SERVER_USER_VOL	The Docker volume where your credentials are saved.

   Script. Adjust variable settings as described in the table above, then execute this script to start the container.

   ANALYSIS_SERVER_IMAGE=codesonar_remote_analysis_server
   ANALYSIS_SERVER_IMAGE_VERSION=latest
   ANALYSIS_SERVER_CONTAINER=$ANALYSIS_SERVER_IMAGE
   ANALYSIS_SERVER_USER_VOL=$ANALYSIS_SERVER_IMAGE.config
   ANALYSIS_SERVER_HOSTNAME=csonar-analysis
   HOST_ANALYSIS_DATA=/srv/codesonar/analysis_data
   CODESONAR_ANALYSIS_DATA=/srv/codesonar/analysis_data
   CODESONAR_ANALYSIS_DATA_MAX_MB=100000
   CODESONAR_HUB=https://codesonar.example.com:7341
   CODESONAR_HUBUSER=cshub_ci
   CODESONAR_LAUNCHD_GROUP=/analysis-data-server
   CODESONAR_LAUNCHD_KEY=remote-archive
   CI_USER=csonar_ci
   CI_USER_HOME=/home/$CI_USER
   CS_USER_HOME=$CI_USER_HOME/cs_home
   
   docker container run \
       --env "CODESONAR_HUB=$CODESONAR_HUB" \
       --env "CODESONAR_HUBUSER=$CODESONAR_HUBUSER" \
       --env "CODESONAR_ANALYSIS_DATA=$CODESONAR_ANALYSIS_DATA" \
       --env "CODESONAR_ANALYSIS_DATA_MAX_MB=$CODESONAR_ANALYSIS_DATA_MAX_MB" \
       --env "CODESONAR_LAUNCHD_GROUP=$CODESONAR_LAUNCHD_GROUP" \
       --env "CODESONAR_LAUNCHD_KEY=$CODESONAR_LAUNCHD_KEY" \
       --env "CODESONAR_HUBUSER=$CODESONAR_HUBUSER" \
       --env "CS_USER_HOME=$CS_USER_HOME" \
       -v "$ANALYSIS_SERVER_USER_VOL:$CS_USER_HOME" \
       -v "$HOST_ANALYSIS_DATA:$CODESONAR_ANALYSIS_DATA" \
       --hostname $ANALYSIS_SERVER_HOSTNAME \
       --name $ANALYSIS_SERVER_CONTAINER \
       --init \
       --detach \
       --restart unless-stopped \
       $ANALYSIS_SERVER_IMAGE:$ANALYSIS_SERVER_IMAGE_VERSION
   

6. Verify that your analysis data server container is running

   Use docker container logs to show the console output from the Docker container. You should see that the cslaunchd process was started successfully, and you should not see errors.

   For example:

   $ docker container logs $ANALYSIS_SERVER_CONTAINER
   codesonar: Authenticating with https://codesonar.example.com:443 (password)...ok
   codesonar: Running cslaunchd in foreground...
   

Next Steps

If you are setting up a GitHub pipeline to perform CodeSonar analysis, go on to step C to create and install a build workflow runner.

Notes

Disk management

The data stored on your analysis data server may reach the maximum permitted size you specified when you started the server.

You can delete unwanted analysis data manually from the CodeSonar GUI. You can also configure your hub to automatically delete data from older analyses, using custom criteria to define "older".

See the CodeSonar manual for more information:

• Using CodeSonar > GUI Reference > Analysis Auto-Deletion and Analysis Log Auto-Deletion
• Using CodeSonar > GUI Reference > Analysis

Modifications for password authentication

If you need to authenticate your launch daemon with a password instead of with a certificate, make the following changes.

1. When you set up the temporary container to produce and store credentials, replace the line that begins

   $CSONAR/codesonar/bin/codesonar generate-hub-cert [...]
   

   with the following command:

   $CSONAR/codesonar/bin/codesonar generate_hubpwfile.py "$CS_USER_HOME/hubpwfile"
   

2. When you start the temporary container provide the hub user account password when prompted. The password will be saved in file $CS_USER_HOME/hubpwfile in the Docker volume. Make sure only your Docker user (docker_ci) has access to this file.