Issue Trackers
Enterprise auth
GitLab CI/CD
Testmo features extensive GitLab CI/CD integration to submit test automation results from your CI pipelines to implement GitLab test management. This only requires a few lines of code and is easy to set up. Any test automation tool, framework and platform can be used.
Also works with GitLab Issues
You can integrate simple GitLab CI/CD pipelines (such as running just a single test automation suite) as well as complex workflows with multiple test suites (e.g. frontend, backend, API etc.) and even parallel testing jobs with multiple threads.
Regardless of your specific configuration and workflow, the basic concept of integrating your GitLab CI/CD pipeline with Testmo is always the same:
- 1.You execute your automated tests as part of your GitLab CI/CD workflow. It doesn't matter if you run a single test suite, multiple test suites or even parallel testing jobs.
- 2.You configure your automated tests to generate a result file using the standard JUnit XML file format. This file format has become the de facto standard format to exchange test results between testing tools. So practically any test automation tool and framework either supports this format directly or can convert to this format.
- 3.You then use the Testmo CLI tool to easily submit the results to Testmo, enabling you to track your test suites over time, identify problematic test cases and improve testing performance.
Further links, references and guides
Basic GitLab CI/CD & Testmo workflow
The following GitLab CI/CD workflow shows how to run a simple test automation run as part of your CI workflow and submit the test results to Testmo. In this example we are using a test suite with the Mocha/Chai (NodeJS/Javascript) testing framework. Remember that you can use any test automation tool and framework with Testmo and we are just using this framework for our example.
We are using the Testmo CLI tool here as it makes it very easy to run our tests and submit all results with a single command line call (using the
automation:run:submit command).We are also sending a few additional fields to Testmo here, such as a link back to the GitLab CI/CD pipeline run as well as the Git commit hash. These details are then displayed in Testmo along with the test results, console output, exit code etc.
You can find the full code for this project on GitLab in the example project repository. You can also find a detailed explanation of this project along with more code samples and screenshots in our reference guide:
1
# .gitlab-ci.yml
2
default:
3
image: node:16
4
​
5
stages:
6
- test
7
​
8
test:
9
stage: test
10
script:
11
- npm ci --cache .npm --prefer-offline
12
- npm install --cache .npm --prefer-offline --no-save @testmo/testmo-cli
13
​
14
# Optionally add a couple of fields such as the git hash and link to the build
15
- npx testmo automation:resources:add-field --name git --type string
16
--value $CI_COMMIT_SHORT_SHA --resources resources.json
17
- npx testmo automation:resources:add-link --name build
18
--url $CI_PIPELINE_URL --resources resources.json
19
​
20
- npx testmo automation:run:submit
21
--instance $TESTMO_URL
22
--project-id 1
23
--name "Mocha test run"
24
--source "unit-tests"
25
--resources resources.json
26
--results results/*.xml
27
-- npm run mocha-junit # Note space after --
28
​
29
cache:
30
- key:
31
files:
32
- package-lock.json
33
paths:
34
- .npm/
35
​
36
artifacts:
37
when: always
38
reports:
39
junit:
40
- results/mocha-test-results.xml
Copied!
Advanced GitLab CI/CD workflows
We also support advanced GitLab CI/CD workflows. For example, you can submit test results for many test suites from a GitLab pipeline run and Testmo also fully supports parallel test runs.
- Running multiple test suites in a build If you have multiple test suites for your project (such as frontend, backend, API, mobile tests etc.) you can simply submit these as separate test runs from the same GitLab CI/CD pipeline. You would just use multiple calls to
automation:run:submitto submit each test suite separately. Important: if you have different test suites, you need to use different source names for the test runs in Testmo. This way Testmo can differentiate where the test results are coming from. Learn more about test automation sources. - Using parallel testing jobs with GitLab CI/CD Testmo also fully supports tracking test runs with multiple parallel test jobs. So if you have configured your GitLab CI/CD workflow to run multiple parallel jobs to speed up your test execution, you can also submit each job as a separate thread of the same run to Testmo. You can learn more about implementing this in the following example.
GitLab CI/CD parallel testing workflow
Testmo comes with full support for GitLab CI/CD workflows with parallel testing jobs, so you can submit the results as separate threads of the same run to Testmo. You can then see the results (and console output, exit codes & testing times) of each thread separately, as well as the overall results and statistics for the entire test run in Testmo. The following illustration shows how to implement the workflow with GitLab CI/CD.
Example parallel testing workflow
In this workflow we use separate jobs to set up our test run (
test-setup), then submit a thread for each testing job of the run (test), and finally mark the test run as completed (test-complete).We have a detailed guide with example code, screenshots and explanations on implementing this for GitLab CI/CD. You can also find a complete example project repository on GitLab. You can find the full guide here:
1
# .gitlab-ci.yml
2
default:
3
image: node:16
4
cache:
5
- key:
6
files:
7
- package-lock.json
8
paths:
9
- .npm/
10
​
11
stages:
12
- build
13
- test-setup
14
- test
15
- test-complete
16
- deploy
17
​
18
build:
19
stage: build
20
script:
21
- echo "Building .."
22
​
23
test-setup:
24
stage: test-setup
25
script:
26
- npm ci --cache .npm --prefer-offline
27
​
28
# Optionally add a couple of fields such as the git hash and link to the build
29
- npx testmo automation:resources:add-field --name git --type string
30
--value $CI_COMMIT_SHORT_SHA --resources resources.json
31
- npx testmo automation:resources:add-link --name build
32
--url $CI_PIPELINE_URL --resources resources.json
33
​
34
- npx testmo automation:run:create
35
--instance $TESTMO_URL
36
--project-id 1
37
--name "Parallel mocha test run"
38
--resources resources.json
39
--source "unit-tests" > testmo-run-id.txt
40
- echo "TESTMO_RUN_ID=$(cat testmo-run-id.txt)" > testmo.env
41
artifacts:
42
reports:
43
dotenv: testmo.env
44
​
45
test:
46
parallel: 4
47
stage: test
48
script:
49
- npm ci --cache .npm --prefer-offline
50
- npx testmo automation:run:submit-thread
51
--instance $TESTMO_URL
52
--run-id $TESTMO_RUN_ID
53
--results results/*.xml
54
-- npm run mocha-junit-parallel # Note space after --
55
artifacts:
56
when: always
57
reports:
58
junit:
59
- results/mocha-test-results.xml
60
dependencies:
61
- test-setup
62
​
63
test-complete:
64
stage: test-complete
65
script:
66
- npm ci --cache .npm --prefer-offline
67
- npx testmo automation:run:complete
68
--instance $TESTMO_URL
69
--run-id $TESTMO_RUN_ID
70
dependencies:
71
- test-setup
72
when: always
73
​
74
deploy:
75
stage: deploy
76
script:
77
- echo "Deploying .."
Copied!
Important security considerations
When working with the Testmo API and CLI to submit test automation results to Testmo, you need to specify an API key for the authentication. You can learn more about generating API keys in Testmo here.
When integrating Testmo with your GitLab CI/CD pipelines, it is important not to store the API key in your code repository. If you store the API key in your code repository, everyone with access to the repository would be able to access the key. You should never add and store your Testmo API key as part of your code repository.
Instead, use GitLab's project variable feature to securely store keys and passwords. You can then make these secrets available as environment variables in your pipeline. You can see this in the example configuration above, as well as in our detailed GitLab CI/CD guide. You can also learn more about GitLab CI/CD project variables in the official documentation.
Last modified 19d ago