Automation Runs

The automation/runs namespace provides API methods for automation runs and threads. You can use these methods to retrieve automation runs and threads for a project, create new runs and threads, submit test results (including custom fields, artifacts & links) and finally mark runs and threads as completed.

GET /projects/{project_id}/automation/runs

Returns all automation runs for a project.

This method uses pagination so you might need to request additional pages to retrieve all automation runs.

project_id id required ID of the project.

Request

page integer Number of page to return (default: first page)

per_page integer Maximum number of automation runs to return (supported: 15, 25, 50, 100; default: 100)

sort string Sort field for the list of automation runs (supported: automation_runs:created_at; default: automation_runs:created_at)

order string Sort order (ascending or descending) (supported: asc, desc; default: desc)

config_id string Comma-separated list of configurations to filter by.

created_after string Limit result to automation runs created after (in ISO8601 format and UTC time zone).

created_before string Limit result to automation runs created before (in ISO8601 format and UTC time zone).

created_by string Comma-separated list of users to filter by.

milestone_id string Comma-separated list of milestones to filter by.

source_id string Comma-separated list of automation sources to filter by.

status string Comma-separated list of statuses to filter by. Use: 2 for success, 3 for failure, 4 for running.

tags string Comma-separated list of tags to filter by.

expands string Comma-separated list of expands to return.

This method supports the following expands so you can automatically include additional information for referenced objects:

  • automation_sources

  • configs

  • milestones

  • statuses

  • users

Response

GET /api/v1/projects/1/automation/runs
200 OK
{
    "page": 1,
    "prev_page": null,
    "next_page": 2,
    "last_page": 2,
    "per_page": 100,
    "total": 150,
    "result": [
        {
            "id": 1,
            "project_id": 1,
            "name": "Run 1",
            "status": 2,
            ..
            "is_completed": false,
            "untested_count": 0,
            "status1_count": 90,
            "status2_count": 10,
            "status3_count": 0,
            "status4_count": 0,
            "status5_count": 0,
            "status6_count": 0,
            "status7_count": 0,
            "status8_count": 0,
            "status9_count": 0,
            "status10_count": 0,
            "status11_count": 0,
            "status12_count": 0,
            "status13_count": 0,
            "status14_count": 0,
            "status15_count": 0,
            "status16_count": 0,
            "status17_count": 0,
            "status18_count": 0,
            "status19_count": 0,
            "status20_count": 0,
            "status21_count": 0,
            "status22_count": 0,
            "status23_count": 0,
            "status24_count": 0,
            "success_count": 90,
            "failure_count": 10,
            "completed_count": 100,
            "total_count": 100,
            "thread_count": 8,
            "thread_active_count": 2,
            "thread_completed_count": 6,
            "created_at": "..",
            "created_by": 2,
            "updated_at": null,
            "updated_by": null,
            "completed_at": null,
            "completed_by": null
        },
        {
            "id": 2,
            "project_id": 1,
            "name": "Run 2",
            "status": 3,
            ..
            "is_completed": true,
            "untested_count": 1,
            "status1_count": 20,
            "status2_count": 5,
            "status3_count": 0,
            "status4_count": 0,
            "status5_count": 0,
            "status6_count": 0,
            "status7_count": 0,
            "status8_count": 0,
            "status9_count": 0,
            "status10_count": 0,
            "status11_count": 0,
            "status12_count": 0,
            "status13_count": 0,
            "status14_count": 0,
            "status15_count": 0,
            "status16_count": 0,
            "status17_count": 0,
            "status18_count": 0,
            "status19_count": 0,
            "status20_count": 0,
            "status21_count": 0,
            "status22_count": 0,
            "status23_count": 0,
            "status24_count": 0,
            "success_count": 20,
            "failure_count": 5,
            "completed_count": 25,
            "total_count": 26,
            "thread_count": 8,
            "thread_active_count": 2,
            "thread_completed_count": 6,
            "created_at": "..",
            "created_by": 2,
            "updated_at": null,
            "updated_by": null,
            "completed_at": "..",
            "completed_by": 1
        },
        ..
    ],
    "expands": {
        ..
    }
}

Examples

// Get latest 100 automation runs for project with ID 5
GET /api/v1/projects/5/automation/runs

// Get second result page (pagination)
GET /api/v1/projects/5/automation/runs?page=2

// Get latest 100 automation runs
GET /api/v1/projects/5/automation/runs

// Get latest 100 failed automation runs
GET /api/v1/projects/5/automation/runs?state=3

// Get latest automation runs created after a certain date & time
GET /api/v1/projects/5/automation/runs?created_after=2023-02-15T00:00:00.000Z

// Get automation runs and include expands
GET /api/v1/projects/5/automation/runs?expands=automation_sources,configs,users

Status codes

200 400 401 403 422 (details)

GET /automation/runs/{automation_run_id}

Returns a single automation run.

automation_run_id id required ID of the automation run to return.

Request

expands string Comma-separated list of expands to return.

This method supports the following expands so you can automatically include additional information for referenced objects:

  • automation_sources

  • configs

  • milestones

  • statuses

  • users

Response

GET /api/v1/automation/runs/1
200 OK
{
    "result": {
        "id": 1,
        "project_id": 1,
        "name": "Run 1",
        "status": 2,
        ..
        "is_completed": false,
        "untested_count": 0,
        "status1_count": 90,
        "status2_count": 10,
        "status3_count": 0,
        "status4_count": 0,
        "status5_count": 0,
        "status6_count": 0,
        "status7_count": 0,
        "status8_count": 0,
        "status9_count": 0,
        "status10_count": 0,
        "status11_count": 0,
        "status12_count": 0,
        "status13_count": 0,
        "status14_count": 0,
        "status15_count": 0,
        "status16_count": 0,
        "status17_count": 0,
        "status18_count": 0,
        "status19_count": 0,
        "status20_count": 0,
        "status21_count": 0,
        "status22_count": 0,
        "status23_count": 0,
        "status24_count": 0,
        "success_count": 90,
        "failure_count": 10,
        "completed_count": 100,
        "total_count": 100,
        "thread_count": 8,
        "thread_active_count": 2,
        "thread_completed_count": 6,
        "created_at": "..",
        "created_by": 2,
        "updated_at": null,
        "updated_by": null,
        "completed_at": null,
        "completed_by": null
    },
    "expands": {
        ..
    }
}

Examples

// Get the automation run with ID 5
GET /api/v1/automation/runs/5

// Get a automation run and include expands
GET /api/v1/automation/runs/1?expands=automation_sources,configs,users

Status codes

200 400 401 403 404 422 (details)

POST /projects/{project_id}/automation/runs

Creates a new automation run in a target project in preparation for adding threads and test results.

project_id id required ID of the target project.

Request

name string required Name of the new automation run.

source string required Name of the source for the new automation run. If this source does not already exist in the target project, Testmo automatically creates a new one. It is recommended to keep source names short. Good examples are backend, frontend or mobile-iphone.

config string Name of the configuration for the new automation run.

config_id id ID of the configuration for the new automation run. If both config and config_id are specified, config_id is given precedence.

milestone string Name of the milestone for the new automation run.

milestone_id id ID of the milestone for the new automation run. If both milestone and milestone_id are specified, milestone_id is given precedence.

tags array List of tags for the new automation run. If a milestone in the same project has one or more matching automation tags, the new automation run is automatically linked to that milestone (unless milestone or milestone_id is also specified).

artifacts array List of externally stored test artifacts to link to the new automation run (such as log files, screenshots or test data). (see schema for details)

fields array List of fields to attach to the new automation run (such as environment variables, errors or terminal output). (see schema for details)

links array List of links to attach to the new automation run (such as a link back to the build in the CI tool that triggered the tests). (see schema for details)

Example

POST /api/v1/projects/1/automation/runs
{
    "name": "Run 1",
    "source": "frontend",
    "tags": [
        "tag-a",
        "tag-b"
    ],
    "artifacts": [
        {
            "name": "Artifact 1.png",
            "url": "https://example.com/artifact1.png",
            "mime_type": "image/png",
            "size": 1048576
        }
    ],
    "fields": [
        {
            "name": "Field 1",
            "type": 4,
            "value": ".."
        }
    ],
    "links": [
        {
            "name": "Link 1",
            "url": "https://example.com"
        }
    ]
}

Returns the ID of the created automation run on success:

201 Created
{
    "id": 1
}

Status codes

201 400 401 403 404 422 (details)

POST /automation/runs/{automation_run_id}/append

Appends test artifacts, fields or links to an existing automation run.

path automation_run_id id required ID of the automation run.

Request

artifacts array List of externally stored test artifacts to link to the automation run (such as log files, screenshots or test data). (see schema for details)

fields array List of fields to attach to the automation run (such as environment variables, errors or terminal output). (see schema for details)

links array List of links to attach to the automation run (such as a link back to the build in the CI tool that triggered the tests). (see schema for details)

Example

POST /api/v1/automation/runs/1/append
{
    "artifacts": [
        {
            "name": "Artifact 1.png",
            "url": "https://example.com/artifact1.png",
            "mime_type": "image/png",
            "size": 1048576
        }
    ],
    "fields": [
        {
            "name": "Field 1",
            "type": 4,
            "value": ".."
        }
    ],
    "links": [
        {
            "name": "Link 1",
            "url": "https://example.com"
        }
    ]
}
204 No Content

Status codes

204 400 401 403 404 422 (details)

POST /automation/runs/{automation_run_id}/complete

Marks an automation run and its threads as completed and closes it for new threads and test results.

automation_run_id id required ID of the automation run to complete.

Request

measure_elapsed boolean Defines if the execution time of the automation run should automatically be set to the elapsed time between the creation and completion date of the run. This is useful in scenarios where the overall execution time of the run is independent from the execution times of its threads.

Example

POST /api/v1/automation/runs/1/complete
{
    "measure_elapsed": true
}
204 No Content

Status codes

204 400 401 403 404 422 (details)

POST /automation/runs/{automation_run_id}/threads

Creates a new thread in an automation run in preparation for adding test results.

path automation_run_id id required ID of the target automation run.

Request

elapsed_observed integer Observed overall elapsed (execution time) of the thread in microseconds.

elapsed_computed integer Computed overall elapsed (execution time) of the thread in microseconds. The observed elapsed is usually derived from measuring the execution time of tests whereas the computed elapsed is calculated from result log files.

artifacts array List of externally stored test artifacts to link to the new thread (such as log files, screenshots or test data). (see schema for details)

fields array List of fields to attach to the new thread (such as environment variables, errors or terminal output). (see schema for details)

Example

POST /api/v1/automation/runs/1/threads
{
    "elapsed_observed": 600000000,
    "artifacts": [
        {
            "name": "Artifact 1.png",
            "url": "https://example.com/artifact1.png",
            "mime_type": "image/png",
            "size": 1048576
        }
    ],
    "fields": [
        {
            "name": "Field 1",
            "type": 4,
            "value": ".."
        }
    ]
}

Returns the ID of the created thread on success:

201 Created
{
    "id": 1
}

Status codes

201 400 401 403 404 422 (details)

POST /automation/runs/threads/{automation_run_thread_id}/append

Appends test artifacts, fields or test results to an existing thread in an automation run.

path automation_run_thread_id id required ID of the automation run thread.

Request

elapsed_observed integer Partial observed elapsed (execution time) in microseconds to add to the overall observed time of the thread.

elapsed_computed integer Partial computed elapsed (execution time) in microseconds to add to the overall computed time of the thread. The observed elapsed is usually derived from measuring the execution time of tests whereas the computed elapsed is calculated from result log files.

artifacts array List of externally stored test artifacts to link to the thread (such as log files, screenshots or test data). (see schema for details)

fields array List of fields to attach to the thread (such as environment variables, errors or terminal output). (see schema for details)

tests array List of tests to add to the thread. (see schema for details)

Example

POST /api/v1/automation/runs/threads/1/append
{
    "elapsed_observed": 600000000,
    "artifacts": [
        {
            "name": "Artifact 1.png",
            "url": "https://example.com/artifact1.png",
            "mime_type": "image/png",
            "size": 1048576
        }
    ],
    "fields": [
        {
            "name": "Field 1",
            "type": 4,
            "value": ".."
        }
    ],
    "tests": [
        {
            "key": "5ccaaa9ca9f351d0c36b45c59728f1a23d16601d",
            "name": "Test 1",
            "folder": "Folder A",
            "status": "passed",
            "elapsed": 60000000,
            "file": "test.js",
            "line": 100,
            "assertions": 20,
            "artifacts": [
                {
                    "name": "Artifact 1.png",
                    "url": "https://example.com/artifact1.png",
                    "mime_type": "image/png",
                    "size": 1048576
                }
            ]
        },
        {
            "key": "7295b69b0df06d518d481c54d71973f9f911520a",
            "name": "Test 2",
            "folder": "Folder A",
            "status": "passed"
        },
        {
            "key": "db0003c47b7565af20cacd3370f8152fd02b5a00",
            "name": "Test 3",
            "folder": "Folder B",
            "status": "failed",
            "fields": [
                {
                    "name": "Error",
                    "type": 4,
                    "value": "..",
                    "meta": {
                        "type": "NullPointerException"
                    },
                    "is_highlight": true
                }
            ]
        },
        ..
    ]
}
204 No Content

Status codes

204 400 401 403 404 422 (details)

POST /automation/runs/threads/{automation_run_thread_id}/complete

Marks an automation run thread as completed and closes it for new test results.

path automation_run_thread_id id required ID of the automation run thread to complete.

Request

elapsed_observed integer Observed overall elapsed (execution time) of the thread in microseconds.

elapsed_computed integer Computed overall elapsed (execution time) of the thread in microseconds. The observed elapsed is usually derived from measuring the execution time of tests whereas the computed elapsed is calculated from result log files.

Example

POST /api/v1/automation/runs/threads/1/complete
{
    "elapsed_observed": 600000000
}
204 No Content

Status codes

204 400 401 403 404 422 (details)

Last updated