A powerful plugin that integrates your tests with Testomat.io platform for test management, reporting and analytics

• ✅ Sync tests with testomat.io
• 📊 Real-time test execution reporting

• https://testomatio.github.io/check-tests/ - for sync
• https://testomatio.github.io/reporter/ - for reporting

• Installation
• Quick Start
• Advanced Usage
  • Basic Configuration
  • Import Listener
    • Listener Configuration
    • Clean Test IDs
    • Disable Detach Test
    • Remove Empty Suites
    • Keep Test IDs Between Projects
    • Keep structure
  • Report Listener
    • Listener Configuration

Prerequisites:

• Python 3.10+
• Robot Framework 4.0+
• Active testomat.io account

Install via pip:

pip install robot-framework-reporter

1. Login to Testomat.io
2. Create project or go to existing project
3. Click on "Import Tests from Source Code"
4. Copy your project token(starts with "tstmt_")

Synchronize tests to Testomat.io using ImportListener:

TESTOMATIO=your_token robot --listener reporter.listener.ImportListener path/to/tests

Execute tests and send results to Testomat.io using ReportListener:

TESTOMATIO=your_token robot --listener reporter.listener.ReportListener path/to/tests

After importing tests to Testomat.io, each test is automatically assigned a unique Test ID.
Testomat.io Test ID is a string value that starts with @T and contains 8 characters after.

Test ID is appended to the test name:

*** Test Cases ***
Test Addition @T96c700e6
    [Documentation]    Check addition of two numbers
    [Tags]    math    positive
    ${result}=    Evaluate    10 + 5
    Should Be Equal As Numbers    ${result}    15

Before import (original test):

*** Test Cases ***
Test Addition
    [Documentation]    Check addition of two numbers
    [Tags]    math    positive
    ${result}=    Evaluate    10 + 5
    Should Be Equal As Numbers    ${result}    15

After import (with Test ID):

*** Test Cases ***
Test Addition @T96c700e6
    [Documentation]    Check addition of two numbers
    [Tags]    math    positive
    ${result}=    Evaluate    10 + 5
    Should Be Equal As Numbers    ${result}    15

💡 Note: Test ID is added automatically to the test name when using ImportListener. You don't need to add them manually.

Testomat.io integration with Robot Framework is implemented through the Listener Interface. Currently, two Listeners are available:

• ImportListener. Used for synchronizing tests with Testomat.io
• ReportListener. Used for reporting test results to Testomat.io

Listeners can be configured through parameters or environment variables. Each Listener has its own configuration options, which are described in the corresponding sections.

💡 Note: Parameters and environment variables configure different aspects of the Listener's behavior. Each configuration option is available through only one method - either as a parameter or as an environment variable, not both.

Both Listeners use the following environment variables:

Used for importing tests to Testomat.io.

If you want to import the synced project as new project, you have to clean the test ids. To clean up test ids use remove_ids parameter:

TESTOMATIO=your_key robot --listener reporter.listener.ImportListener:remove_ids=1 path/to/tests

This method may be unsafe, as it cleans all @T* tags from tests names. So if you have a tag like @Test1234 in test name this may also be removed. If you use this option make sure if all the test titles a proper before committing the tests in GIT.

If a test from a previous import was not found on next import it is marked as "detached". This is done to ensure that deleted tests are not staying in Testomatio while deleted in codebase.

To disable this behavior and don't mark anything on detached on import use no_detach parameter:

TESTOMATIO=your_key robot --listener reporter.listener.ImportListener:no_detach=1 path/to/tests

If tests were marked with IDs and imported to already created suites in Testomat.io newly imported suites may become empty. Use no_empty parameter to clean them up after import.

TESTOMATIO=your_key robot --listener reporter.listener.ImportListener:no_empty=1 path/to/tests

This prevents usage structure parameter.

To import tests with Test IDs set in source code into another project use create parameter. In this case, a new project will be populated with the same Test IDs.

TESTOMATIO=your_key robot --listener reporter.listener.ImportListener:create=1 path/to/tests

When tests in source code have IDs assigned and those tests are imported, Testomat.io uses current structure in a project to put the tests in. If folders in source code doesn't match folders in Testomat.io project, existing structure in source code will be ignored. To force using the structure from the source code, use structure parameter on import:

TESTOMATIO=your_key robot --listener reporter.listener.ImportListener:structure=1 path/to/tests

Used for reporting test results to Testomat.io. By default, sends test results in batches after each test suite completes.

Currently, has no parameters