Jenkins

You can integrate Jenkins CI pipelines with Testmo and submit your automated test results to Testmo from your builds to implement Jenkins test management. This usually requires only a few lines of code and is easy to set up. Any test automation tool, framework and platform can be used.

We support both simple CI pipelines (e.g. running just a single test automation suite) as well as complex workflows with multiple test suites (e.g. frontend, backend, API etc.) and even parallel test jobs with multiple threads.

Regardless of the complexity of your Jenkins CI pipelines and workflow, the basic concept of how to integrate with Testmo is always the same:

1. You execute your automated tests as part of your Jenkins CI pipeline. It doesn't matter if you run a single test suite, multiple test suites or even parallel test jobs.

2. You configure your automated tests to generate a result file using the standard JUnit XML file format. This file format has become the de facto standard format to exchange test results between testing tools. So practically any test automation tool and framework either supports this format directly or can convert to this format.

3. You then use the Testmo CLI tool to easily submit the results to Testmo, enabling you to track your test suites over time, identify problematic test cases and improve testing performance.

Further links, references and guides

• Product: Testmo Jenkins integration features

• Guide: Example Jenkins CI Test Automation Pipeline

• Code: Jenkins CI test automation example

• Docs: Testmo CI/CD integration

• Docs: Testmo CLI tool

• Docs: Testmo test automation

Basic Jenkins & Testmo pipeline

The following Jenkinsfile pipeline configuration shows how to run a simple test automation run as part of your CI workflow and submit the test results to Testmo. In this example we are using a test suite with the Mocha/Chai (Node.js/Javascript) testing framework. Remember that you can use any test automation tool and framework with Testmo and we are just using this framework for our example.

We are using the Testmo CLI tool here as it makes it very easy to run our tests and submit all results with a single command line call (using the automation:run:submit command).

We are also sending a few additional fields to Testmo here, such as a link back to the Jenkins build as well as the Git commit hash. These details are then displayed in Testmo along with the test results, console output, exit code etc.

You can find the full code for this project on GitHub in the example project repository. You can also find a detailed explanation of this project along with more code samples and screenshots in our reference guide:

Guide: Jenkins CI Test Automation Pipeline & Reporting ➞

Important: For this code to work, it is required to define the credentials TESTMO_TOKEN and TESTMO_URL in CircleCI with your Testmo address and API token as explained below.

# Jenkinsfile
pipeline {
    agent {
        docker { image 'node' }
    }
    environment {
        TESTMO_URL = credentials('TESTMO_URL')
        TESTMO_TOKEN = credentials('TESTMO_TOKEN')
    }
    stages {
        stage('Build') {
            steps {
                echo 'Building ..'
            }
        }
        stage('Test') {
            environment {
                GIT_SHORT_COMMIT = "${GIT_COMMIT[0..7]}"
            }
            steps {
                // Install your project packages & dependencies here
                // [..]

                // Install Testmo CLI tool locally (then use npx testmo .. to run it)
                sh 'npm install --no-save @testmo/testmo-cli'

                // Optionally add a couple of fields such as the
                // git hash and link to the build
                sh '''
                  npx testmo automation:resources:add-field --name git --type string \
                    --value $GIT_SHORT_COMMIT --resources resources.json
                  npx testmo automation:resources:add-link --name build \
                    --url $BUILD_URL --resources resources.json
                '''

                // Run automated tests and report results to Testmo
                sh '''
                  npx testmo automation:run:submit \
                    --instance "$TESTMO_URL" \
                    --project-id 1 \
                    --name "Test run" \
                    --source "frontend" \
                    --resources resources.json \
                    --results results/*.xml \
                    -- <your-test-command> # Run your automation tool; note space after --
                '''
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying ..'
            }
        }
    }
    post {
        always {
            junit 'results/*.xml'
        }
    }
}

Full example repository on GitHub

Advanced Jenkins workflows

You an also easily integrate advanced Jenkins CI pipelines and workflows with Testmo. Consider the following scenarios:

• Running multiple test suites in a build If you have multiple test suites for your project (such as frontend, backend, API, mobile tests etc.) you can simply submit these as separate test runs from the same pipeline. You would just use multiple calls to automation:run:submit to submit each test suite separately. Important: if you have different test suites, you need to use different source names for the test runs in Testmo. This way Testmo can differentiate where the test results are coming from. Learn more about test automation sources.

• Using parallel test jobs with Jenkins Testmo also fully supports tracking test runs with multiple parallel testing jobs. So if you have configured your Jenkins pipeline to run multiple parallel jobs to speed up your test execution, you can also submit each job as a separate thread of the same run to Testmo. You can learn more about how to implement this in the Testmo CLI: Parallel test jobs in CI/CD pipelines article.

Example parallel testing workflow

Important security considerations

When working with the Testmo API and CLI to submit test automation results to Testmo, you need to specify an API key for the authentication. You can learn more about generating API keys in Testmo here.

When implementing Testmo in your Jenkins CI pipeline, it is important not to store the API key in your code repository. If you store the API key in your code repository (e.g. in Git), everyone with access to the repository would be able to access the key. You should never add and store your Testmo API key as part of your code repository.

Instead, use the Jenkins credentials feature to securely store your secrets. You can then make the credentials available as environment variables in your pipeline. You can see this in the example Jenkinsfile above, as well as in our detailed Jenkins guide. You can also learn more about Jenkins credentials in the official documentation.