GitLab Jobs
Launching Automation Jobs in GitLab CI/CD (Beta)
Last updated
Launching Automation Jobs in GitLab CI/CD (Beta)
Last updated
In Beta & Early Access
Testmo now allows you to launch test automation jobs in your CI/CD pipelines directly from Testmo. Read more below, or sign up for beta access here:
Testmo enables you to launch automated test runs using GitLab CI/CD pipelines directly from the Testmo Automation Jobs view in any project. This only requires an existing GitLab integration configuration of one or more GitLab targets. Any test automation tool, framework and platform can be used.
You can launch simple GitLab CI/CD pipelines (such as running just a single test automation suite) as well as complex workflows with multiple test suites (e.g. frontend, backend, API etc.) and even parallel testing jobs with multiple threads.
Regardless of your specific configuration and workflow, the basic concept of launching automation jobs from Testmo is always the same.
From the Automation > Jobs view in Testmo, click on the Add Automation Job button.
Select the desired target (the CI/CD pipeline) for the job, and confirm the job.
Testmo will trigger the job and let you know the outcome.
To launch automated tests in GitLab CI/CD, you first need to set up a secured integration with your GitLab account. If you have already configured the GitLab Issues integration for your Testmo instance, you may be able skip to the next step (as the GitLab Issues integration and GitLab CI/CD automation launching integration can use the same connection). Otherwise, follow the steps below to connect to your GitLab account.
Note: You will need Site Admin permissions in Testmo to configure the Testmo GitLab CI/CD integration.
Using a self-hosted GitLab server installation?
Testmo also works with self-hosted GitLab server installations. For Testmo to be able to connect to GitLab using GitLab's API, your GitLab server must be reachable and accessible over the Internet. If your GitLab server is only available on your private network, Testmo will not be able to connect to it. In this case, you can still use our alternative link-based GitLab integration.
Navigate to the Admin > Integrations view in your Testmo instance
From the list of integrations, select GitLab
Click + GitLab account
In the Add connection dialog, enter the details of your GitLab account:
Name: Enter the name you want to use for the integration. The name will be displayed in Testmo when choosing a connection for the issue tracker integration or automation targets. This could be your organization (company) name, team name, project name or similar.
GitLab address: Either select GitLab.com for their cloud service, or select a custom self-hosted GitLab server installation. For a self-hosted GitLab server, also enter the full web address of your GitLab server installation. Important: The server and web address must be accessible over the Internet by Testmo to connect to it (see above).
Set up token authentication: In GitLab, create an access token so Testmo can connect to GitLab and authorize with its API. Note that any projects and issues accessible by this user can be looked up and accessed from Testmo by any Testmo user, as this access token is used for any app to app communication. You might want to use a separate integration user in GitLab if you want to limit project access. You can either use a personal access token for a user or a project access token (only supported by some GitLab subscription plans). To create a new access token in GitLab, either from the user preferences or project settings select Access Tokens in the left sidebar. Then create a new token with the following settings:
Token name: Testmo
Expiration date: Leave empty
Select scopes: Check api
After saving/creating the token, copy the new access token to Testmo's dialog.
Set up OAuth authentication: To enable Testmo users to authenticate with GitLab OAuth to push issues, configure a new Application in GitLab. This can be done on a user, group or instance-wide basis in GitLab. You can learn more in GitLab's documentation. You only need to configure this once, not for all your users. For example, to configure the application on a user basis, select Applications from the sidebar of the user preferences page in GitLab. Then add a new application with the following settings:
Name: Testmo
Redirect URI: https://interop.testmo.net/gitlab-app/oauth/callback
Confidential and Expiration: Leave default
Select scopes: Check api
After saving/creating the token, copy both the Application ID and Secret to Testmo's dialog.
Test connection: You can click this button to make sure everything works.
Save connection: click Add connection to save the connection for use in automation targets and jobs
The new connection has been created.
Once you have connected to your GitLab account, you can set up an automation target in Testmo. Automation targets give you a way to specify pipelines, branches, or tags that you would like to be able to trigger ahead of time. While setting up automation targets, you can also include unique parameters, inputs, or variables to trigger pipelines for particular environments, datasets, etc.
To set up an automation target in Testmo, you will need to have an existing GitLab CI/CD pipeline defined in your connected GitLab account (e.g. the pipeline you would like to trigger from Testmo).
In the Admin > Automation view, click on the Add target button
In the Add Target dialogue:
Name: Give your target a suitably descriptive name (you’ll need to be able to easily identify it when adding jobs, later)
Connection: Select the CI/CD connection, GitLab in this scenario.
Project Path or ID: Add the GitLab group and name of your project (or the project ID) from GitLab. The group name and project in which you’ve implemented the pipeline example from above, for example: “yourgroupname/example-automation-launching”.
Testmo expects your GitLab group & project ID will be entered here.
When you navigate to a project in GitLab, it’ll be the following string from the URL: groupid/project (after https://gitlab.com/), e.g. “mike1234/myapp”
Reference: Add the repository branch or tag name to be used in the pipeline.
This will be specific to your project, and can be found in various GitLab project views including the main project page and the code repository
Variables: under the Variables tab, add any variables you would like to pass to your GitLab CI/CD pipeline.
We expect automation launching may be used alongside parameters such as the browser, environment or platform to be tested, although you may define many other kinds of variables in your pipeline's .yml file!
You may also wish to set up more advanced logic/switching scenarios. Please refer to GitLab documentation for how variables may be used in the context of a pipeline.
The Testmo CLI also provides advanced switching mechanisms, for which more information is supplied here: Reference | Testmo Docs
Under the Projects tab, you can select which Testmo projects you would like this target to be available for
By default, automation targets will be available for all Testmo projects.
Save target: click Add target to save the target once you’ve added all the settings & variables.
Your automation target is now ready to trigger with an automation job in the Automation > Jobs view of any enabled project in your Testmo instance.
Once you have set up an automation target, you can use Automation jobs to trigger your GitLab CI/CD pipelines right from Testmo.
Navigate to the Automation > Jobs view of one of your projects, then click Add job.
For more on automation jobs, see Automation jobs.
Testmo comes with full support for GitLab CI/CD pipelines via the Testmo CLI. You can read more information about how to set up a test automation pipeline using the Testmo CLI here: GitLab CI/CD | Testmo Integrations
Automation Launching is designed to build on the existing pipeline workflow by providing testers with a capability to launch their automation pipelines directly from Testmo’s Automation > Jobs area.
A basic workflow might look like this:
Add an automation job to launch your target GitLab CI/CD pipeline
GitLab CI/CD executes your automated tests controlled by the workflow
GitLab CI/CD submits test results back to Testmo using the Testmo CLI
Testmo makes it easy for you to control where your jobs can be launched and by whom.
When setting up GitLab targets, you can select which projects they may be used in via the Target > Projects tab. Simply enable the target for all projects, or select which one(s) you wish to use it in:
If you want to limit which user types are able to launch automation jobs from the Testmo UI, you can control this via the Users & Roles > Roles & Permissions settings in the Admin area. Simply create or edit the user role and apply or deselect the permissions from it:
To read more about frequently asked questions around automation launching and GitHub Actions, see our Automation Jobs FAQs page.
The steps above assume prerequisite steps of configuring your GitLab integration and setting one or more targets in the Testmo administration area have been completed. Refer to the documentation below for more information:
