Collect results

One test history for every framework

Send the report files your automated tests already write, send results from your continuous integration (CI) jobs, or send them live as tests run. TestLookup works out the format, keeps the build, branch and commit details you send, and links each test to its own history.

Get started Read the guide to sending results

Runs table for 13 test runs, showing each run's number and suite, failure signature, status, AI verdict, test counts, pass rate and timing. — Sample data Every run with its status, AI verdict and pass rate

11 report formatsJUnit, pytest, TestNG, Allure, Playwright, Cypress and more. TestLookup reads the file to tell which one it is.
11 live reportersWatch results arrive live from pytest, Jest, Playwright, TestNG, JUnit 5 and 6, Go and others.
CI details keptTestLookup finds the CI run link and pull request. The build, branch and commit you send stay with every run.

Send results the way you work

Every way of sending results ends the same way: a test run in your project, with one row per test. TestLookup then queues the run for analysis.

Report files

11 formats, detected for you

Upload reports from JUnit, TestNG, Allure, Playwright, pytest, Robot Framework, Cucumber, NUnit, TRX, xUnit.net or Cypress through Mochawesome, or a ZIP file of several. TestLookup reads what's inside the file, not its name, to tell the format. If it picks the wrong one, send the file again and name the format.

Works with the standard reports from pytest, .NET's dotnet test and Robot Framework
Cucumber JSON reports from cucumber-jvm, cucumber-js, behave or SpecFlow
A test that stopped with an error (broken) stays separate from one that failed a check (failed)

Upload a report from CIbash · TestLookup CLI

pytest --junitxml=results.xml || true
testlookup upload file results.xml --project "$PROJECT_ID" --build "$BUILD_ID" \
  --branch "$BRANCH" --commit "$COMMIT_SHA" --wait

Details: install the command-line tool from the cli folder of a copy of the TestLookup code, with python -m pip install -e ./cli. It reads your server address and key from TESTLOOKUP_URL and TESTLOOKUP_API_KEY. Give the project's ID with --project. The tool doesn't read the branch or commit from CI, so pass them as shown. With --wait, the step fails if TestLookup refuses the report. To send a whole folder, use testlookup upload dir. To name the format, add --format, or use the format field when you upload through the API. Common report options: pytest --junitxml, dotnet test --logger trx, and Robot Framework's output.xml.

Upload, API or CLI

Upload it, post it or script it

To try a report once, drop files on Runs › Upload report, and the browser zips several files together for you. In CI, send the file through the API or the command-line tool. You can also send your own results in batches of JSON, a common data format, from your own test script.

Skip AI analysis on an upload when you only need the results
Resending a batch for the same build and CI job updates the same run
Name a release when you send results, and the run joins it

Send a batch of resultsbash · REST API

curl -X POST "http://localhost:8000/api/v1/ingest" \
  -H "X-API-Key: $TESTLOOKUP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"project_id": "'"$PROJECT_ID"'", "build_number": "ci-1234", "branch": "main",
       "results": [{"test_name": "test_checkout_total", "status": "FAILED", "duration_ms": 812}]}'

Each result needs a test name and a status. The API reference lists every optional field, under POST /api/v1/ingest.

Watch results arrive live

Live

11 reporters send each result as it finishes

Reporters are small add-ons for your test framework that send each result as soon as a test finishes. They cover pytest, Robot Framework, Jest, Mocha, Playwright, Vitest, Cypress, Cucumber JS, TestNG, JUnit 5 and 6, and Go's testing package. The Live Execution page, under Runs › Live, shows active runs, tests in progress and a pass rate checked against your release rules.

Download each reporter from your own TestLookup, with its setup code already filled in
Cypress sends results once per spec file instead of once per test
If a test runner crashes partway, TestLookup closes the run and keeps what arrived
When the tests end, TestLookup finishes the run just like an upload

Send pytest results livebash · Python reporter

pytest --testlookup-url http://localhost:8000 \
       --testlookup-api-key "$TESTLOOKUP_API_KEY" \
       --testlookup-project "$PROJECT_ID" \
       --testlookup-build build-42

Reporters need an API key created by someone with the QA engineer role or higher.

Keep the context of every run

The details around a test result often explain it, so TestLookup saves them with the results.

CI details

Keep the CI details of each run

The command-line tool and the code libraries (SDKs) find the CI system, repository, pull or merge request, who started the job, and the run link. This works in GitHub Actions, GitLab CI, Jenkins multibranch, Azure DevOps and CircleCI. Outside CI, they send none of these details.

Settings you pass yourself always win over what's detected
The command-line tool and the Python library need the branch and commit passed in. The Go library reads them from CI.
The tool and the Python library read the list of commits from your local Git copy, with no token or network call
Up to 100 commits per run, with the names of changed files but not the changes
The Python library and the tool also record runtime versions, dependency lock file fingerprints and container image IDs

Timing

See where the time went

Each run keeps its total time, the sum of its test times and how much ran in parallel. The Timing tab draws one row per parallel worker and lists the 20 slowest tests. TestLookup flags a test that's much slower than its usual time, or a suite that keeps getting slower over recent runs.

Tests: flagged at 3 standard deviations above their usual time, measured over 10 or more runs
Suites: flagged when the median of the last 5 runs is at least 1.5 times the median of the 20 before
If a report format records no times, the page says not reported instead of zero
Export every test's timing as a CSV file from the Timing tab
Details: testlookup runs timing --fail-on-regression ends with code 7, so a CI step can fail on a slowdown

Run timing timeline with one row per worker showing each test as a bar from start to finish, passed in green and failed hatched in red. — Sample data Per-worker test timeline

Web requests

Capture the web requests behind a failure

When you turn on capture, TestLookup records the HTTP calls each test made: method, address, status, headers and bodies. The pytest plugin records calls made with requests and httpx, and the Java library records RestAssured calls. The reporter hides sign-in details, tokens and email addresses before sending, TestLookup hides them again before saving, and the calls appear on the test's own page.

Off by default. Details: --testlookup-capture-http in Python, or the RestAssured filter in Java.
Keeps up to 20 calls per test and 1,000 per run
Stores up to 16 KB of each body, and counts anything over a limit
Needs Python reporter 1.3.0, or the Java library with RestAssured 5.x

Limits worth knowing

These limits affect how you set up your pipeline.

50 MiB per upload
TestLookup refuses a report file or ZIP over 50 MiB, and any empty file. ZIP files also have a limit on how large they can grow when unpacked.
Accepted doesn't mean finished
TestLookup replies accepted (HTTP 202) as soon as your report is waiting to be processed. It replies the same way even if the file has no results. Before you rely on the run, check its upload task, or let the command-line tool wait for it.
Saving files works for two formats
Allure and TestNG results can also arrive as files saved to a new folder in TestLookup's file storage, including the built-in one. Save a marker file named upload_complete.json last, and TestLookup imports the folder within about a minute, with nothing else to set up. Send every other format with an upload.

What happens to a test report

TestLookup accepts a report right away, then reads and saves it in the background. Analysis comes afterwards and can take a while.

Diagram of a test report’s journey: TestLookup checks the sender and its capacity, accepts the report right away, a background worker reads the results and saves them as a test run, totals and history are updated, notifications go out if set up, and analysis later suggests likely causes and a release recommendation. — **What happens to a test report**
What happens to a test report
The full text description remains below the inline figure.
Scroll within the image to inspect it at full size.
Every step is written out under Text equivalent below.

Text equivalent

1. Check
TestLookup checks that the sender may use the project and that it has room for the work. If not, it refuses the report or asks the sender to try again later.
2. Accept
It accepts the report straight away and gives back a task number. The report has not been read yet.
3. Read and save
A background worker reads the results and saves them as a test run. Then it updates the run’s totals and history, and sends notifications if they are set up.
4. Analyze
Analysis runs afterwards. It uses rules, a trained model or an optional AI model, and suggests likely causes and a release recommendation.

Boundaries and limitations

Accepted doesn’t mean saved. A problem while reading the report is recorded on the upload, so check the upload or the run before you rely on it.
Analysis can fail or be retried without changing the saved run.
Live reporters send results another way, which this diagram leaves out.

Send your first report

Start TestLookup on your computer, upload a JUnit file, and watch it become a test run.

Import your first report Get started