Requirements Coverage & Traceability
The Issue Coverage & Traceability Report enables you to report on how requirements are covered by test cases, plans, and tests.
Last updated
Was this helpful?
The Issue Coverage & Traceability Report enables you to report on how requirements are covered by test cases, plans, and tests.
Last updated
Was this helpful?
Reporting Center is currently being made available to all customers as a Preview feature.
After the preview, the feature will be available for customers on Business & Enterprise plans.
Requirements, use cases, or other scenarios that you may need to test against often exist in the form of issues, epics, stories, bugs or other ticket types logged in issue trackers like Jira, GitHub Issues, or GitLab. You can link those requirements to test cases, test runs, milestones, and sessions in Testmo by adding the issue ID from your external issue tracker to the Issues field on the related objects in Testmo. This allows you to trace your requirements through your test cycle and ensure you are covering the full scope of what needs to be tested.
This page explains how to use the Requirements Coverage & Traceability Report to show how your issues are covered by planned testing activities, and what the latest results of those tests are. .
The Options section controls the scope of the report to be generated, starting with project selection and gradually drilling down to further selections enabling fine-grained control over objects to be included in the report.
Project
Use this dropdown to select which single project the report will cover. Reports can be generated for one project at a time, and cannot retrieve data from multiple projects in the same report.
Issues
Use this dropdown to determine the level of issue coverage for the report. You have 3 options to choose from:
All linked issues: If you want to include all issues which are linked to cases, results, runs, sessions & milestones in the scope of your report, choose this option.
Select trackers: If you use more than one issue tracker in your project, and wish to limit the scope of your report by one or more trackers, use this option (by clicking the window button). Note that your selection here is overridden if you select option 1 “All linked issues”, or option 2 by entering specific issue IDs.
Enter issue ID’s: If you wish to constrain your report to a specified list of issue ID’s, you may enter them (one per line) in the Enter issues dialog box (by clicking the window button).
Cases
The cases dropdown enables you to select which cases should be included in the scope of your report. You again have 3 options to select from:
All cases: If you want to include all cases from your project in the scope of your report, choose this option.
Filter cases: If you want to define and apply a filter to your cases, select this option and define the filter in the dialog box (by clicking the filter button).
Select cases: If you want to select specific cases for your report, select this option and apply your selections in the dialog (by clicking the window button).
Runs
The runs dropdown enables you to select which runs should be included in the scope of your report. You have 4 options to select from:
All runs: Choose this option if you want to include all the runs from your project in your report. Note that selecting this option will include runs marked as closed as well as open/active runs.
All active runs: Choose this option if you only want to include open/active runs and exclude closed runs from your report.
All closed runs: Choose this option if you only want to include closed runs and exclude open runs from your report.
Filter runs: Choose this option if you want to define a custom filter for which runs are included in the scope of your report (by clicking the filter button).
Sessions
The sessions dropdown enables you to select which sessions should be included in the scope of your report. You have 4 options to select from:
All sessions: Choose this option if you want to include all the sessions from your project in your report. Note that selecting this option will include sessions marked as closed as well as open/active sessions.
All active sessions: Choose this option if you only want to include open/active sessions and exclude closed sessions from your report.
All closed sessions: Choose this option if you only want to include closed sessions and exclude open sessions from your report.
Filter sessions: Choose this option if you want to define a custom filter for which sessions are included in the scope of your report (by clicking the filter button).
Milestone
This field may be used to configure a single milestone filter which is applied across both runs and sessions, rather than doing so for each individually. Note that this filter will override any filters which have already been applied on the run & session filter configuration. Only a single milestone may be selected at a time using this controller, versus multiple milestones using the run &/or session filters. Note that only runs & sessions from the selected milestone will be included in the report, not those from sub milestones. (If you want to specify sub milestones to include, leave this setting blank and use run/session filters instead).
Created
This field may be used to configure a single creation-date filter which is applied across both runs and sessions, rather than doing so for each individually. Note that this filter will override any filters which have already been applied on the run & session filter configuration.
The report content controls determine what major sections will be included in the generated report. Each section can be toggled off or on by unchecking or checking the relevant checkbox. By default, all sections are included in the finished report.
Key Metrics
This initial section of the report contains high level metrics for the configured scope, in summary form. Depending on the options selected, the key metrics will display total numbers in the report for:
Issues
Test cases
Runs
Sessions
Coverage chart
This section displays a stacked bar chart view of issue coverage across the selected report options (issues, cases, runs, sessions), making it easy for consumers of the report to see how well issues are covered.
Coverage statistics:
This section displays a tabular version of the issue coverage metrics, with counts and percentages. If you have more than one issue tracker integrated with your project, you’ll see a breakdown of issues by tracker as well.
Issues
The issues table contains a detailed breakdown of all issues in the scope of the report along with details of how they’re covered by cases, runs and sessions (depending on selected report options).
Show status column
Gauge whether your issues are ready to ship, or not, by displaying the latest status of the tests covering them.
You can follow example steps below to create a requirements coverage & traceability report of your Jira issues.
Prerequisite steps
Create the necessary issues in Jira. Typically, these will take the form of Epics, Stories, Tasks or Bugs. For this example we’ll assume you’re using Jira Stories as placeholders for requirements (AKA some feature or improvement in your product).
Add the Jira issue references to the test cases, sessions or runs, which cover your Stories.
Execute your tests.
Creating the report with all issues included
Once the prerequisite steps have been completed, you can create a requirements coverage report for your Jira issues, as follows. On the Requirements coverage & traceability report page:
Select your project.
If you have more than one issue tracker integrated with your project, click the Issues dropdown, click the Select trackers option, click the dialogue icon and select only the Jira tracker.
Click the Generate report button.
The report will show you coverage for all Jira issues which are linked to your cases, runs & sessions:
High level metrics:
Numbers of issues, test cases, runs, sessions covering your issues
Chart breakdown
Total issues and issues with coverage
Issue trackers in scope (Jira, in this case)
Detailed metrics:
A breakdown of case, run & session coverage by issue - with the issue ID on the left, the entity type, case, run or session name, and latest test status (if you’re using the default “Show status column” setting
Creating the report with explicitly specified issues
If you want to take the report a step further, and show which Jira issues aren’t covered by your cases, runs or sessions, we need to apply a slightly different approach:
Click the Issues dropdown and select the Enter issue IDs option.
Click the dialogue button in the Issues dropdown.
Enter the issue IDs you wish to report on, in a list, then click Save.
Once done, click the Generate report button.
With the issues explicitly specified, Testmo can report on whether or not all of them are covered by your cases, runs or sessions. Thereby helping you to identify potential gaps in your testing:
The number of issues without coverage are highlighted in the graph, and in the coverage statistics.
Issues without coverage are also reported on in a separate table in an additional Issues without coverage table.
You can follow example steps below to create a requirements coverage report for your automated tests.
Prerequisite steps
In order to create a requirements coverage report for your automated tests, you’ll need to follow similar steps as already outlined above, with an additional automation linking step.
Assuming that you’re still using Jira as the source of your requirements:
Create the necessary issues in Jira.
Add the Jira issue references to the test cases, sessions or runs, which cover your Stories.
Execute your automated tests.
Creating the report with all issues included
Once the prerequisite steps have been completed, you can create a requirements coverage report for your Jira issues and your automation coverage, as follows.
On the Requirements coverage & traceability report page:
Select your project.
If you have more than one issue tracker integrated with your project, click the Issues dropdown, click the Select trackers option, click the dialogue icon and select only the Jira tracker.
Click the Cases dropdown, select Filter cases and click the Filter icon to open the filter dialogue. Select the Automation dropdown, and set it to Yes. Applying this setting means that only cases with automation coverage will be visible in the report.
Uncheck the Runs & Sessions Options.
Click the Generate report button.
The report will show you coverage for all Jira issues which are linked to your cases, and which also have automated test coverage.
If you want to see which test cases have requirements but no automation test coverage, you can simply change the Cases > Automation filter to No, instead of Yes.
The report view settings can be used to enable or disable components of the report so they can more easily be customised and shared to suit different use cases.
Show Links
By default, reports include links to the relevant pages in Testmo. For example, you can quickly navigate to tests by clicking their names. Or you can navigate to a test run by clicking its ID. Alternatively, if you want to share reports with persons who don’t have access to Testmo or shouldn’t even know where the tests are, you can completely disable and remove all links, so you can share reports without any links back to your Testmo account, by unchecking this option.
Show Persons
A major benefit of using Testmo for your projects is that it’s easy to track and understand which person on your team made changes to test cases or added results. Testmo also helps improve collaboration and task management by allowing you to assign tests to team members. If you don’t want to share that information in your report however, you can easily hide all personal names and avatars by unchecking this option.
Show Header & Show Footer
Don’t want to include a header or Testmo branded footer in your report? No problem, you can also disable those by unchecking this option.
Once your report includes all the data and content you wish to include based on the settings above, you can share it with your team and stakeholders by clicking on the Print & PDF button.
Testmo utilises the built-in capabilities of your web browser to generate the Print or PDF version of your report, which you can send to your printer or save to a PDF using your browser controls.
The report configuration panel consists of 3 main areas, covered in each section below. Many of the configuration options correspond with controls in our existing PDF reports, which you can read more about in the guide:
Ensure you’ve setup the necessary integration with Jira first, by following the steps here:
Ensure you’ve setup the necessary integration with Jira first, by following the steps here:
Link your test cases to the automated tests which cover them. Follow the instructions here:
