User workflows

These workflows take you from sending a test result to a release decision. At every step, these are separate outcomes: TestLookup accepting a result, storing it, analysing it, a person reviewing it, and a release decision.

Send your first run

1. Send a report to a project, from Runs › Upload report in the web app or with the API. The quick start shows both, and the first report tutorial walks through one.
2. Add the build label and any CI details you have, such as the branch, the commit and the CI run URL. With these details, TestLookup can compare the run with others.
3. Keep the run_id and task_id from the response.
4. Wait until the run appears under Runs before you compare it with history.

The guide to sending results covers every report format and every way to send results.

Triage

Triage means looking at a failure and deciding what to do with it. Open a run from Runs, or use the Failures page, which groups failing tests by their error message and sorts them by kind of failure.

For each failure, look at:

• its evidence and its history;
• the classification and how sure the analysis is about it;
• any evidence the analysis says is missing.

When you correct a classification, TestLookup updates that failure’s record. That does not mean a new model has already been trained or put into use.

Inbox › Assigned to me lists the failures that TestLookup assigned to you automatically, with their status and actions to reassign them.

Review

AI reports wait for review under Inbox › Approvals › AI reports. A QA lead or administrator, signed in to the web app, can accept a report or reject it with a reason code. An API key cannot make this decision.

There is no “request changes” decision. To get a corrected report, reject it and run the analysis pipeline again. The new pipeline run’s report gets its own review request. A review is a separate record from the pipeline finishing, and from the report being sent anywhere.

Release decision

Before you rely on a release decision, check these things:

• which runs are linked to the release;
• whether their results are complete;
• whether the report review is done.

A whole release gets GO, NO-GO or Not evaluated. A single run’s decision can also be Conditional GO, and it uses the project’s policy, so check which policy applies. If you disagree with a run’s decision, record an override with your reason instead of agreeing to it informally. The release-readiness overview and the reports guide explain the decisions.

Authoring

Manage the test cases, plans, strategies and knowledge sources your team writes in test management. Each one has its own stages and history. They stay separate from the test results in your runs, both in counts and in status changes.

Handing a problem to an administrator

Administrators look after sign-in, storage, background workers, integrations, how long data is kept, backups and policy settings. When you hand a problem to an administrator, include the project, the run ID, the task ID and the step that failed.

Roles and their limits

TestLookup has five roles, and each one includes everything the one before it can do: Viewer, Tester, QA engineer, QA lead and Administrator. Even so, three things can refuse an action that your role allows. They are your project membership, the actions an API key is allowed to take, and the few actions that need a person signed in to the web app.

Automate with CI

Your CI system is the service that builds and tests your code on every change, such as GitHub Actions or Jenkins.

CI integration

Give the CI job a service account or an API key that can reach the project, and keep the key in your CI system’s secret store. Send a report file or a batch when you only need the final results. Stream live events when you want to watch results as the tests run.

When the job retries, send the same build and CI details as the first attempt. If TestLookup answers 429 or 503, wait for the time in the Retry-After header before you try again. Wait for the run to finish before you use it to pass or fail the job, because an accepted upload is not a decision yet.

The testlookup ci-verdict command passes the CI job when every failure in the run is quarantined. A quarantined test is a flaky test that your team has set aside, so its failures do not block the build. A flaky test passes or fails at random, with no change to the code. This command does not check a release. To check a release, use testlookup release gate. See the integration guide.

To see the web app with sample data first, open the demo, which uses made-up data.