Testmo also supports multiple ways to include and submit fields, properties and references for automated tests. Testmo reads JUnit-style XML files to submit results with any automation tool (see many examples). We support various extensions for this format to include additional properties on a test case level.
Our CLI tool (which you use to submit test results to Testmo) then reads and submits these fields together with your test results. This topic explains how to include fields for your tests and the various field types and conventions Testmo supports.
If you are using a test automation framework (or reporter module) that can output properties for test cases in the resulting XML files (properties are an extension feature not yet supported by all tools), then this is the easiest way to include additional fields. The resulting XML file should include properties like this:
<testcasename="testCase3"classname="Tests.Registration"time="3.441"> <properties> <propertyname="priority"value="high" /> <propertyname="language"value="english" /> <propertyname="author"value="Adrian" /> <propertyname="attachment"value="https://ci/files/22-dashboard.png" /> <propertyname="attachment"value="https://ci/files/23-users.png" /><!-- Optional support for properties with text values --> <propertyname="description"><![CDATA[ This text describes the purpose of this test case and provides an overview of what the test does and how it works. ]]></property> </properties><!-- Optional output or error/failure/skipped --></testcase>
Testmo supports various special field names (for steps and attachments) as well as various additional field types (such as links or HTML). See the additional sections below to learn more.
Some frameworks and their APIs do not support outputting multiple properties with the same name; in this case you would just append a unique index suffix like this:
Alternatively, if your test automation tool doesn't support properties on a test case level yet, our Testmo CLI tool can detect and read properties included in the system-out or system-err sections of the XML file. This means that if your automation tool supports capturing the console output of your tests (and many do), you can simply write properties to standard out using the following syntax:
<testcasename="testCase3"classname="Tests.Registration"time="3.441"> <system-out><![CDATA[Output line #1Output line #2[[PROPERTY|author=Adrian]][[PROPERTY|language=english]][[PROPERTY|browser-log]]Log line #1Log line #2Log line #3[[/PROPERTY]] ]]></system-out></testcase>
Testmo can detect and read single line and multi line properties from the output fields. Additionally, Testmo understands the output attachment syntax used by various tools.
[[PROPERTY|author=Adrian]]
[[PROPERTY|language=english]]
[[PROPERTY|browser-log]]
Log line #1
Log line #2
Log line #3
[[/PROPERTY]]
[[ATTACHMENT|https://ci/files/22-dashboard.png]]
[[ATTACHMENT|https://ci/files/23-users.png]]
Each property is listed on its own line without any additional text. Each property consists of a name / value pair. Optionally, multi-line properties just specify the name without a value in the opening tag and end with a closing tag on its own line.
Testmo supports various field types for automated test properties, which can be useful so Testmo can nicely display various values. For example, you can tell Testmo that a certain field includes an URL, so Testmo can display it as a link. Likewise, you can ask Testmo to render values as console output, or even render HTML for more complex data (including source code highlighting, tables etc.).
To tell Testmo which type a certain field uses, you use so called type hints as part of the field names. Simply prepend the type and a colon before the field name. Testmo currently supports the following types:
string - The default for single line fields
text - The default for multi-line fields
url - Renders a clickable link
console - Renders as monospaced console/output text
For HTML, Testmo supports a subset of HTML tags and attributes. Please refer to our full HTML reference, including examples on how to render tables, source code, lists etc.
Here are some examples on how to include type hints with your fields:
You can add type hints in front of the property names. The type is removed from the resulting names in Testmo.
<properties> <propertyname="string:priority"value="high" /> <propertyname="url:github"value="https://github.com/project/code.js" /> <propertyname="text:description"value="Long text field.." /> <propertyname="console:log"><![CDATA[ Log line #1 Log line #2 Log line #3 ]]></property> <propertyname="html:richtext"><![CDATA[ <h1>Title</h1> <p>Text</p> ]]></property></properties>
Type hints work exactly the same with properties included in outputs. Simply add the type in front of the field name.
[[PROPERTY|string:priority=high]]
[[PROPERTY|url:github=https://github.com/project/code.js]]
[[PROPERTY|text:description=Long text field..]]
[[PROPERTY|console:log]]
Log line #1
Log line #2
Log line #3
[[/PROPERTY]]
[[PROPERTY|html:richtext]]
<h1>Title</h1>
<p>Text</p>
[[/PROPERTY]]
Testmo also supports & understands a couple of special field names, specifically to support steps and attachment/artifact links for test automation.
Test steps
Name: step*
You can optionally include test steps as properties to render all the steps an automated test performs. Testmo optionally also supports a status per step as well as sub fields.
Testmo expects the fields to be named step or, if your tool doesn't support multiple properties with the same name, you can add a unique index like step1, step2 etc. See below for examples on the syntax to include steps and optionally statuses and sub fields.
Simply name the fields step (or use unique indexes such as step1, step2 ..) and optionally include a status in square brackets. You can use all status automation aliases as configured in Testmo under Admin > Statuses.
<properties><!-- Either use multiple properties named "step" --> <propertyname="step"value="This is the first step." /> <propertyname="step"value="And this is the second step." /><!-- Or append a unique index if your test framework requires this --> <propertyname="step1"value="Testmo will automatically index the steps." /> <propertyname="step2"value="And this is another test step." /><!-- Some tools support multi-line properties. Steps always support HTML --> <propertyname="step"><![CDATA[ This is a more complex test step with <strong>multiple lines</strong> of text using a property with a text value. ]]></property><!-- You can optionally also include a step result status as part of the property name in square brackets. All status automation aliases as configured in Testmo under Admin > Statuses work. --> <propertyname="step[passed]"value="This is a test step that passed." /> <propertyname="step[failure]"value="And this step had a failure." /> <propertyname="step[skipped]"value="This step wasn't executed." /> <propertyname="step[error]"value="And this step had an error." /><!-- Testmo also supports step sub fields using the following special tags. --> <propertyname="step[passed]"><![CDATA[ <test-step-subfield name="Step"> This line describes this step. You can also use <strong>HTML</strong>. </test-step-subfield> <test-step-subfield name="Expected"> You can optionally include more sub fields, e.g. for the expected results. </test-step-subfield> ]]></property></properties>
You can also include steps via test outputs. The step names and values work exactly the same. You can also use all status automation aliases as configured in Testmo under Admin > Statuses.
# Either use multiple properties named "step"
[[PROPERTY|step=This is the first step.]]
[[PROPERTY|step=And this is the second step.]]
# Multi-line properties also work. Steps always support HTML
[[PROPERTY|step]]
This is a more complex test step with <strong>multiple lines</strong>
of text using a property with a text value.
[[/PROPERTY]]
# You can optionally also include a step result status as part
# of the property name in square brackets. All status automation
# aliases as configured in Testmo under Admin > Statuses work.
[[PROPERTY|step[passed]=This is a test step that passed.]]
[[PROPERTY|step[failure]=And this step had a failure.]]
[[PROPERTY|step[skipped]=This step wasn't executed.]]
[[PROPERTY|step[error]=And this step had an error.]]
# Testmo also supports step sub fields using the following special tags.
[[PROPERTY|step[passed]]]
<test-step-subfield name="Step">
This line describes this step. You can also use <strong>HTML</strong>.
</test-step-subfield>
<test-step-subfield name="Expected">
You can optionally include more sub fields, e.g. for the expected results.
</test-step-subfield>
[[/PROPERTY]]
Attachments
Name: attachment*
You can also include links to external attachments and Testmo will automatically render these as artifact links.
Testmo expects the fields to be named attachment or, if your tool doesn't support multiple properties with the same name, you can add a unique index like attachment1, attachment2 etc. Please note: Attachments need to be URLs / links to externally hosted files, e.g. in your CI pipeline service. In the future Testmo might also support automatically uploading local attachments.
Simply name the fields attachment (or use unique indexes such as attachment1, attachment2 ..). Attachments need to be URLs / links to externally hosted files.
<properties><!-- Either use multiple properties named "attachment" --> <propertyname="attachment"value="https://ci/files/22-dashboard.png" /> <propertyname="attachment"value="https://ci/files/23-users.png" /><!-- Or append a unique index if your test framework requires this --> <propertyname="attachment1"value="https://ci/files/24-sessions.png" /> <propertyname="attachment2"value="https://ci/files/25-failure.png" /></properties>
You can also include attachments via test outputs. Testmo also supports the alternative syntax with ATTACHMENT entries as supported by many tools.
# Either use multiple properties named "attachment"
[[PROPERTY|attachment=https://ci/files/22-dashboard.png]]
[[PROPERTY|attachment=https://ci/files/23-users.png]]
# Or append a unique index if your test framework requires this
[[PROPERTY|attachment1=https://ci/files/24-sessions.png]]
[[PROPERTY|attachment2=https://ci/files/25-failure.png]]
# Or use the common attachment syntax used by many tools
[[ATTACHMENT|https://ci/files/25-admin.png]]
[[ATTACHMENT|https://ci/files/26-login.png]]
We try to make it as easy as possible to include additional test fields. Please see our guides on using test automation with Testmo and many test automation frameworks and tools: