Testmo works with any test automation tool, including the popular Playwright cross-platform end-to-end testing framework. This quickstart guide provides an overview of how to report Playwright test automation results to Testmo to get started with Playwright test management.
Run Playwright & generate XML file
Playwright comes with full support for generating JUnit-style XML files, which has become a standard format to exchange test results between tools. Simply set the junit reporter in your Playwright config and configure the filename and recommended settings:
playwright.config.ts
import type { PlaywrightTestConfig } from '@playwright/test';
const config: PlaywrightTestConfig = {
reporter: [['junit', {
outputFile: 'results/test-results.xml',
embedAnnotationsAsProperties: true,
// Not used by Testmo
// embedAttachmentsAsProperty: undefined
}]],
};
export default config;
$ npx playwright test
This will run your Playwright tests and automatically write all test results to an XML file in the results directory. You can learn more about configuring the JUnit reporter in the Playwright documentation.
To submit your test results to Testmo, you simply use our cross-platform testmo CLI tool. The CLI tool is distributed as an NPM package and is easy to install on any system. Simply install our official @testmo/testmo-cli NPM package:
We can now send the Playwright test results to Testmo. To do this, make sure to generate an API key in Testmo from your profile page. The API key is used to authenticate with Testmo to send the results. We can then use the testmo CLI tool so submit our results (note that we first set the TESTMO_TOKEN variable, which the tool expects):
On Windows, the command line uses a slightly different syntax to set a variable:
> set TESTMO_TOKEN=********
> testmo automation:run:submit --instance https://<your-name>.testmo.net --project-id 1 --name "Playwright test run" --source "frontend" --results results/*.xml
See command output
Collecting log files ..
Found 1 result file with a total of 855 tests
Created new automation run (ID: 254)
Created new thread (ID: 608)
Sending tests to Testmo ..
Uploading: [|||||||||||||||||||||||||] 100% | ETA: 0s | 855/855 tests
Successfully sent tests and completed run
Marked the run as completed
That's it! 🎉 This will automatically analyze the XML result file, create a new test run in Testmo, submit all tests & results and mark the run as completed. There's no need to manually create any tests, map tests or IDs or build any custom API code. Everything is handled automatically for you.
In the above example, we first launched Playwright to generate the XML file and then used our CLI tool to submit the results in a second step. As an improvement to the above example, we can ask our CLI tool to launch Playwright (testmo then starts and waits for Playwright to finish). This has the following additional benefits:
a) Capture full console output and send it to Testmo
b) Accurately measure test times
c) Record the Playwright exit code
Launch Playwright with CLI tool
We can ask our CLI tool to launch Playwright, so we can capture the console output and more:
To include additional fields for tests, use Playwright's custom annotations. Make sure to enable the embedAnnotationsAsProperties option in the Playwright JUnit reporter config as in the above config example. Testmo will automatically read these properties from the XML file and submit these as fields to Testmo. Testmo supports many automation field types including strings, links, rich HTML, steps and attachment links.
Add custom fields with Playwright
To log additional fields for a test, you can use custom annotations:
// example.spec.ts
test('user profile', async ({ page }) => {
test.info().annotations.push({ type: 'author', description: 'Adrian' });
test.info().annotations.push({ type: 'language', description: 'english' });
test.info().annotations.push({ type: 'url:github',
description: 'https://github.com/project/code.js' });
// Testmo also supports automation steps, including statuses & sub fields
test.info().annotations.push({ type: 'step[passed]', description: 'The first step' });
test.info().annotations.push({ type: 'step[passed]', description: 'The second step' });
test.info().annotations.push({ type: 'step[failed]', description: 'The third step' });
// You can also include attachment links for test artifacts
test.info().annotations.push({ type: 'attachment',
description: 'https://ci-server/browser.log' });
test.info().annotations.push({ type: 'attachment',
description: 'https://ci-server/screenshot.png' });
// [..]
});
You can learn more about Playwright's custom annotations in their documentation. And you can learn more about fields and types supported by Testmo.
Now that you are familiar with submitting your Playwright test results to Testmo, you might also find the following additional examples, topics and references useful for more advanced workflows.
