Analyzer Development Guide · Sec · Development · Help · GitLab (2023)

Analyzers are available as Docker images to run as part of a CI pipeline. This guide describes development and testing practices in analyzers.

common unity

For common behaviors and interfaces, there are a number of common Go modules shared between parsers:

• thisSeriesThe Go package implements the CLI interface.
• thisUsualThe project provides several common modules for logging, certificate management, and directory search functionality.
• thisreportloosereportITo discoverStructure for JSON reports.
• thisrole modelA new analyzer for project scaffolding.

How to use the analyzer

The analyzer is available as a Docker image. For example, runningSamGripScan your working directory for Docker images:

1. CDChange to the directory of the source code you want to scan.

2. joggingDocker login registry.gitlab.comand give username pluspersonaltheWorkThe access token is at leastread the recordfield of application.

3. Run the Docker image:

   docker run\ - interactive --tty - R M \ - Sound volume "$password":/tmp/applications\ --Surrounding area project directory=/tmp/applications\ -w/tmp/applications\registry.gitlab.com/gitlab-org/security-products/analyzers/semgrep:latest /launch analyser

   (Video) Agile Management - GitLab-Jira Development Panel Integration

4. The Docker container creates a reference to the attached project directory with the reference filename corresponding to the parser class. For example,Senkocreate a file namedgl-sast-report.json.

Analyst development

To update the analyst:

1. Modify the Go source code.
2. Create a new Docker image.
3. Run the profiler in its test project.
4. Compare the generated report with the expected one.

Here's how you can create oneanalyst:

construction of the ghat-tanalyst.

For example, to check stealth detection, run the following command:

wget https://gitlab.com/gitlab-org/security-products/ci-templates/-/raw/master/scripts/compare_reports.shsh ./compare_reports.sh sdtest/fixtures/gl-secret-detection-report.jsontest/expect/gl-secret-detection-report.json\| fix it-NP1 test/expect/gl-secret-detection-report.json&&Git commit-M "Expectation Update" test/expect/gl-secret-detection-report.jsonR Mcomparative report.sh

You can also compile the binary for your environment and run it locally, butanalyzesIjoggingIt probably won't work because the runtime dependencies for the parser are missing.

This is based onSpotBugs:

build-Theanalyst./search parsertest/Program./parser-transformtest/fixtures/app/spotbugsXml.Xml>./gl-sast-report.json

Executive template

activate the first oneIt is necessary to include the defaultsrole modelin the GitLab CI/CD configuration.

The following independent criteria determine which parser should be run on a project:

1. Using the SAST templaterule: existsDetermines which parser to run based on the existence of certain files. For example, analyst Brakemanrun me .rbfile and agame file.
2. Each analyzer performs a customizableappropriate interfacebefore performing the actual analysis. For example:Flawfinder checks C/C++ files.
3. For some parsers that work on common file extensions, there is checking based on CI/CD variants. For example: Kubernetes events are written in YAML, soto su secundecode onlySCAN_KUBERNETES_MANIFESTS set to true.

Step 1 helps avoid wasted CI/CD time that will be spent running analysts that are not right for the project. However, becausetechnical limitations, cannot be used for large projects. Therefore, step 2 acts as a final check to ensure that mismatched parsers are exited early.

How to test the analyzer

Video tutorial on how to use Dependency Scan Analyzerdownstream pipelineCheck the functionality of the parser using the test project:

Try local changes

Test local changes to shared drives (egSeriesthereport) for the parser you can usemodify and replacecommand to loadSeriesUse your local changes instead of issuing the command with a remote tag. For example:

edited for change-replacegitlab.com/gitlab-org/security-products/analyzers/command/v3=/local/path/to/command

Alternatively, you can update manually viago.moddocument:

Module gitlab.com/gitlab-org/security-products/analyzers/awesome-analyzer/v2Zamijenite gitlab.com/gitlab-org/security-products/analyzers/command/v3 => /path/to/commandrequire (...gitlab.com/gitlab-org/security-products/analyzers/command/v3 v2.19.0)

Test the local changes in Docker

use dockerreplacefrom the insidego.moddocument:

1. copy contentSeriesChange to the parser directory.cp -r /put/do/naredbe path/do/analizatora/naredbe.
2. Add a copy of the statement to the analystdocument:command/copy command.
3. renewreplacestatement to make sure it matches the destinationcopyThe statement in the above step:Replace gitlab.com/gitlab-org/security-products/analyzers/command/v3 => /command

Scenario Analyzer

thisScenario AnalyzerThe repository contains scripts that can be used to interact with most parsers. They allow you to build, run, and debug parsers in a GitLab CI environment and are especially useful for validating parser changes locally.

For more information seeproject readme.

Version control and version process

Analyzers are separate projects that follow their own version control.repairIssue bumps tend to matchUnderageDeprecating versions of key tools (egrobber), which allows us more flexibility in maintenanceUnderageA number of important changes to our scanners. If the packet scanner forces critical changes, you should consider creating a new analyzer in a separate repository.

Analyzers are published as Docker images according to the following scheme:

• push each timeMr; Mrbranch will overriderubbingimage label
• any pressure on anyexcellent featurebranch will create a matchexcellent featureimage label
• Each Git tag creates a counterpartmain.small.patchimage label. Manual tasks allow you to override themMrILatestThe image label indicates thismain.small.patch.

There are two different options for publishing a new Docker analyzer image:

• Manual release procedure
• Automatic publishing process

Manual release procedure

1. be sureChangelog.mdThe entry for the new analyst is correct.
2. Be sure to post the source (usuallyMr; MrtheMrbranch) has a delivery pipeline.
3. with the optionto organizemenu on the left side of the project window and selectexemptSubmenu.
4. choosenew product launchOpennew product launchPage.
   1. from the insidetag namedrop-down menu, enter the same versionChangelog.md, For examplev2.4.2, then set the create bookmark option (Create tags v2.4.2here).
   2. from the insidePost titletext box type the same version used above eg.v2.4.2.
   3. from the insiderelease notestext box, copy and paste the comments of the appropriate version into theChangelog.md.
   4. Leave all other settings as default.
   5. choosecreate a release.

After following the above process to create a new release, a new Git tag is created containing ittag namementioned above. This will start a new pipeline with the given tag version and create a new Docker image of the analyzer.

If the analyst usesparser.ymlrole model, and then vaccinated asnew product launchThe above process automatically adds tags and installs a new version of the Docker analyzer image.

If the parser is not usedparser.ymltemplate, you must manually point and deploy a new version of the Docker parser image:

1. chooseContinuous integration / Continuous integrationmenu on the left side of the project window and selectconveyor beltSubmenu.
2. For example, a new funnel should be created with the same tags used beforev2.4.2.
3. Once the pipeline is complete, it will be importedexclusioncondition.
4. chooseHandmade workplay button on the right side of the window, then selectlabel versionCheck and deploy a new version of the Docker analyzer image.

Use your best judgment to decide when to create a Git tag that will then trigger the release task. If you can't decide, seek the opinion of others.

Automatic publishing process

Before using the auto-publish process, you must do the following:

1. configurationCREATE_GIT_TAG:真asCI/CD environment variables.
2. examine, examinevariablein the CI/CD project settings. unless the work is legacyGITLAB_TOKENenvironment variables from the project team, create aproject access tokenIFull access to read/write APIand adjustGITLAB_TOKENasCI/CD environment variablesThis refers to this token.

After the above steps are completed, the automatic publishing process is performed as follows:

1. The project maintainer merges the MR into the default branch.
2. the default pipeline starts andenter a git tagJob done.
   • If the latest version is availableChangelog.mdCorresponds to one of the Git tags, the task is not a function.
   • Otherwise, the business benefitsPublish API.The version and message are taken from the most recent entryChangelog.mdProject files.
3. Automatically start pipelines for new Git tags. The pipeline publishesLatest,Mr,UnderageIrepairDocker analyzer image.

Steps to perform after publishing the analyzer

1. Once the new version of the Docker analyzer image is tagged and deployed, test it with the appropriate test project.

2. Announce the release in the relevant team Slack channel. Example message:

   (Video) Seven ways GitLab Premium will accelerate your development

   FYI, I just postedANALYZER_NAME ANALYZER_VERSION version.LINK_TO_RELEASE

Never delete a forwarded Git tagSince tags will likely be used and/or cached in the Go package registry.