Milestone Summary & Metrics

The Milestone Summary & Metrics report enables you to report on the testing activities and results associated with a specific milestone.

Linking test runs, sessions and automation runs to milestones in Testmo is a great way to aggregate all the results of your testing activities into a single view. With Testmo’s Milestone Summary & Metrics report, you can generate a summary and detailed breakdown of all the manual, exploratory, and automated testing activities carried out under the umbrella of a single milestone, whether that’s a release, sprint or version of your product.

This page explains how to use the Milestone Summary & Metrics report to show what testing was carried out during the milestone, and what the results of those testing activities were.

The report configuration panel consists of 3 main areas, covered in each section below. Many of the configuration options correspond with controls in our existing PDF reports, which you can read more about in the guide: Print PDF Reports

Generating the Report

The Options section controls the scope of the report to be generated, starting with project selection and gradually drilling down to further selections enabling fine-grained control over objects to be included in the report.

Project

Use this dropdown to select which single project the report will cover. Reports can be generated for one project at a time, and cannot retrieve data from multiple projects in the same report.

Milestone

Use this dropdown to select which milestone the report will cover. Only a single milestone may be selected. If the selected milestone has children, the activities & results from the child milestones will be displayed in the summary charts, but will not be included in the detailed run, session and automation run areas of the report.

Content

How much information the report will contain can be controlled using the Content settings. Check or uncheck the following controls depending on whether you want to display or omit the information accordingly:

• Milestone summary: Controls whether high level summary information and attributes are shown for the selected milestone, including run, session & automation run counts, the type of milestone, the schedule (start & due dates) and current status (derived from the due date & whether you’ve marked the milestone complete or not).

• Show sub milestones: If you’ve enabled the Milestone summary content, you can also select whether to display summary information associated with sub-milestones by enabling or disabling this setting.

• Run & session status chart: Controls whether or not the test run & session status chart is displayed. Note that the chart will include runs and session stats from all sub-milestones if the selected milestone has any.

• Automation status chart: Controls whether the latest automation statuses are displayed. Automation sources from all milestones in the scope of the report will be included in this chart, but only the latest status from each source will be displayed.

• Run & session progress chart: Shows the combined progress and forecast for all test runs & sessions in the scope of the selected milestone (and children). Refer to our Forecasting | Testmo Docs page for more information about how forecasting works and how to read the chart.

• Test runs: Controls whether to display test run information in the report, how to display it, and how much information should be displayed. If the setting is enabled, you have the following additional options:

  • Runs: The runs dropdown enables you to select which runs should be included in the scope of your report. Runs must be linked to the selected milestone (or children) to be included. You have 4 options to select from:

    • All runs: Choose this option if you want to include all the runs from your milestone (or sub-milestones) in your report. Note that selecting this option will include runs marked as closed as well as open/active runs.

    • All active runs: Choose this option if you only want to include open/active runs and exclude closed runs from your report.

    • All closed runs: Choose this option if you only want to include closed runs and exclude open runs from your report.

    • Filter runs: Choose this option if you want to define a custom filter for which runs are included in the scope of your report (by clicking the filter button).

  • Display as: Controls whether runs are displayed as individual cards or as lists.

    • Cards: If you want more granular information about each run, choose this option in order to display all status metrics and attributes associated with the run. Note that displaying cards can result in a much longer report, if you have lots of runs in the scope of your milestone.

    • Lists: If you’re ok with a higher level view of your run information, only displaying the name, state and status bar for each run, choose this view.

  • Maximum runs: Limits the number of runs to the selected maximum. The limit is set to 200 by default.

• Sessions: Controls whether to display session information in the report, how to display it, and how much information should be displayed. If the setting is enabled, you have the following additional options:

  • Sessions: The sessions dropdown enables you to select which sessions should be included in the scope of your report. Sessions must be linked to the selected milestone (or children) to be included. You have 4 options to select from:

    • All sessions: Choose this option if you want to include all the sessions from your milestone (or sub milestones) in your report. Note that selecting this option will include sessions marked as closed as well as open/active sessions.

    • All active sessions: Choose this option if you only want to include open/active sessions and exclude closed sessions from your report.

    • All closed sessions: Choose this option if you only want to include closed sessions and exclude open sessions from your report.

    • Filter sessions: Choose this option if you want to define a custom filter for which sessions are included in the scope of your report (by clicking the filter button).

  • Display as: Controls whether sessions are displayed as individual cards or as lists.

    • Cards: If you want more granular information about each session, choose this option in order to display all status metrics and attributes associated with the session.

    • Lists: If you’re ok with a higher level view of your session information, only displaying the name, state and status bar for each run, choose this view.

  • Maximum sessions: Limits the number of runs to the selected maximum. The limit is set to 200 by default.

• Automation runs: Controls whether to display automation run information in the report, how to display it, and how much information should be displayed. If the setting is enabled, you have the following additional options:

  • Runs: The runs dropdown enables you to select which automation runs should be included in the scope of your report. Automation runs must be linked to the selected milestone (or children) to be included. You have 2 options to select from:

    • All runs: Choose this option if you want to include all the automation runs from your milestone (or sub-milestones) in your report.

    • Filter runs: Choose this option if you want to define a custom filter for which automation runs are included in the scope of your report (by clicking the filter button).

  • Display as: Controls whether runs are displayed as individual cards or as lists.

    • Cards: If you want more granular information about each automation run, choose this option in order to display all status metrics and attributes associated with the run.

    • Lists: If you’re ok with a higher level view of your automation run information, only displaying the name, source, status & timestamp for each automation run, choose this view.

  • Maximum runs: Limits the number of automation runs to the selected maximum. The limit is set to 200 by default.

Using the Report

Example: creating a comprehensive release testing report using milestones

You can follow the example steps below to create a comprehensive milestone summary report for a release to share with cross-functional stakeholders.

Prerequisite steps

1. Ensure you’ve created a milestone in Testmo. Testmo treats all milestones the same, but for this scenario, let’s select Release type.

2. Create and execute runs & sessions as you normally would, linking them to the release milestone via the edit > run/session dialog & milestone dropdown:

1. If you’re running automated tests, make sure tests associated with the release are linked to the milestone as well. You can pass the milestone ID or name in at the point of execution, using the Testmo CLI. Alternatively you can utilise tags and have Testmo automatically select the milestone based on the tags associated with it. Alternatively, you can manually update the automation runs to point at the appropriate milestone after the results have been uploaded to Testmo.

Creating the report

Once you’ve satisfied the prerequisites, you can create the milestone report as follows. On the Milestone summary & metrics report page:

1. Select your project.

2. Select the milestone you wish to report on.

3. Modify the content options according to your preferences.

4. Click the Generate button.

The report will show you all the selected testing information for the chosen milestone.

Example: reporting on Sprint testing using milestones

If you’re using more of a sprint based development methodology, you can follow a similar process as above, but use a Sprint milestone type instead.

Sprint milestones can be sub-milestones of a parent milestone of type release, for example. Be careful when organising your milestones though, since the milestone summary & metrics report can only show runs, sessions and automation runs for the selected milestone.

Prerequisite steps

1. Set up your milestones in Testmo so they resemble a structure similar to the one below:

├── Milestone: Release
│    ├── Sub-Milestone: Sprint 1
│    ├── Sub-Milestone: Sprint 2
│    └── Sub-Milestone: Sprint 3

1. Link runs, sessions & automation runs to the appropriate sub-milestone.

Creating the report

When using sub-milestones, you can approach reporting in summary or detailed form:

• Summary: Select the parent release milestone, uncheck the Test runs, Sessions & Automation runs checkboxes, and click generate for a summary level view of testing activity across all your sprints (sub-milestones).

• Detailed: Select the sub-milestone (sprint) of interest, configure the report settings and generate a detailed report of all run, sessions & automation run activity for the sprint.

Report View Settings

The report view settings can be used to enable or disable components of the report so they can more easily be customised and shared to suit different use cases.

Show Links

By default, reports include links to the relevant pages in Testmo. For example, you can quickly navigate to tests by clicking their names. Or you can navigate to a test run by clicking its ID. Alternatively, if you want to share reports with persons who don’t have access to Testmo or shouldn’t even know where the tests are, you can completely disable and remove all links, so you can share reports without any links back to your Testmo account, by unchecking this option.

Show Persons

A major benefit of using Testmo for your projects is that it’s easy to track and understand which person on your team made changes to test cases or added results. Testmo also helps improve collaboration and task management by allowing you to assign tests to team members. If you don’t want to share that information in your report however, you can easily hide all personal names and avatars by unchecking this option.

Show Header & Show Footer

Don’t want to include a header or Testmo branded footer in your report? No problem, you can also disable those by unchecking this option.

Exporting & Sharing the Report

Once your report includes all the data and content you wish to include based on the settings above, you can share it with your team and stakeholders by clicking on the Print & PDF button.

Testmo utilises the built-in capabilities of your web browser to generate the Print or PDF version of your report, which you can send to your printer or save to a PDF using your browser controls.