Contributing
Tips and guidelines to optimizing and developing test automation tools to work well with Testmo.
Last updated
Tips and guidelines to optimizing and developing test automation tools to work well with Testmo.
Last updated
Are you working on an automation tool, are an open source developer/maintainer or a Testmo enthusiast looking into improving a tool to work better with Testmo? Great! In this guide we are sharing some tips on how to improve & optimize tools for Testmo. Also feel free to contact us at any time if you have questions or need additional tips.
The ideal test automation integration workflow with Testmo is as follows:
Use a standard test automation tool/framework to write tests
Configure the tool to generate JUnit XML files with the test results
Optionally extend the tests with additional fields, steps & attachments
Then use our CLI tool to submit the results from the XML result file(s) to Testmo
This workflow has many advantages. It works well with any test automation tool & platform, as well as in any environment (any operating system, CI services, containers, developer machines etc.), as we've designed our CLI tool to be easily deployed & used everywhere.
This workflow also doesn't require any custom programming, API code, complicated configurations or similar. Testers and developers also automatically benefit from our CLI tool's many optimizations & extra features, such as automatic fast bulk uploads, field mapping & parallel test job support. Therefore, here are some do's and don'ts when it comes to building tools optimized for Testmo:
It is easy to optimize JUnit XML files to better support Testmo (and other test management tools). Here are a few tips for optimizations & useful features.
The best way to optimize a test automation tool/framework for Testmo and other test management or reporting tools is to extend the existing JUnit XML file format to support test case properties. This is a popular extension to the JUnit XML format that is used by many tools. If a tool supports test case properties for XML files, then all Testmo features can be used:
When adding support for test case properties, ideally choose an API that allows multiple properties with the same name. This can be useful to easily include multiple attachments, steps etc. Otherwise testers need to add a unique index after the name, such as attachment1
, attachment2
etc. Here is a pseudo-code API to illustrate this:
If you are contributing to an existing open source automation tool/framework, ideally contribute to and extend the existing JUnit XML logger/reporter to implement these features and work with their maintainers. This way other testers and teams can also benefit from these changes.
Testmo and other tools automatically use the class names and test names to nicely group test results into folders. It is encouraged to use the class name to indicate the hierarchy of the tests, and only output the name of the test (without the class/folder name) for the tests' name attributes. Here's a full example:
If a test automation tool/framework doesn't have good JUnit-style XML file support, or the existing maintainers aren't interested in supporting test case properties, then you can fork/build a simple new third-party JUnit XML logger. Generating JUnit XML files is pretty simple and we also have full documentation for this format:
Usually you can start by forking an existing reporter/logger library or module for the testing tool/framework you are targeting. You can then either write a new JUnit XML logger, or extend the existing JUnit XML logger to extend it with additional features (such as test case properties as shown above).
Once you have optimized and added better Testmo support to a testing tool/framework, we encourage you to publish and promote the integration, and let us know so we can help spread the word. Here are some ideas to promote and make it easier to use the integration:
Publish the fork, library or code on GitHub
Publish your package to your platform's registry (such as NPM, Maven, NuGet etc.)
Share the integration on Twitter and LinkedIn (so we can re-share it)
Update the tool's documentation to mention the Testmo support/optimization
Write a blog posting explaining how to use the tool with Testmo
Do: Optimize the automation tool to write JUnit-style XML files
Do: Extend the JUnit XML support with test case properties (to enable all Testmo features)
Do: Alternatively develop a new Testmo-optimized logger/reporter for JUnit XML files
Do: Encourage testers to generate JUnit XML files & to use our CLI tool
Don't: Extend a tool by calling Testmo's API directly (this is less flexible for users)