Walkthrough with sample data
Look into a failed test with made-up sample data. See the evidence behind the explanation and where it came from, then record a correction. Correcting a result, training a model and creating an issue are three separate things.
Failure analysisTry it
State 1 of 4: The sample run
Nothing plays on its own, and nothing is uploaded or sent to TestLookup or any other service.
The sample run has passed, failed, broken and skipped tests. One failed test is ready for you to look at.
The buttons appear once the page has finished loading. All the steps are already written out below.
Step by step
Example with made-up data. Nothing on this page contacts a server.
Start with the sample run and choose the failed checkout test, as you would on the run page's Tests tab. No real data or issue tracker is involved.
Choose Inspect failure. The page shows the evidence, with sensitive details removed, and where each item came from, as on TestLookup's test result page. The Product Bug classification is an example that comes from rules. No model is called.
Choose missing evidence. Without the stack trace and the comparison, the classification is Unknown, with insufficient evidence to determine the root cause. The page shows no made-up confidence score.
Choose Correct the classification. The correction stays on this page only. In TestLookup, Correct and then Record correction save it as a version in the AI history, used the next time the test is analyzed. Feedback is something TestLookup can learn from later. Nothing is trained or put into use, and no other tool is contacted.
Example with made-up data. Nothing on this page contacts a server.
The sample run has passed, failed, broken and skipped tests. Choose the made-up failed test “checkout rejects expired card”.
All the sample text is made up, with nothing sensitive in it. The walkthrough never contacts TestLookup, an AI model provider, an integration or an issue tracker.
The workflow guide on GitHub treats three things as separate: spotting a failure, giving feedback, and creating an issue in another tool.
The selected test shows an example FAILED result, with no issue in another tool. Choosing it only changes what this page shows.
It doesn’t assign work or change a run. It also doesn’t quarantine the test, which would set it aside so its failures stop blocking builds.
Choose Inspect failure. The walkthrough shows made-up output with sensitive details removed, a comparison with an earlier run, and an example confidence score. Each item is labeled with where it came from.
Choose missing evidence removes the stack trace and the comparison. A stack trace is the list of code steps that led to the error. The walkthrough doesn’t invent anything to replace what’s missing.
In TestLookup, the evidence for a single test is on the test result page, not on the run’s Evidence tab. To open it, go to the run’s Tests tab and choose Open full page. The test result page has:
The run’s Evidence tab holds material about the whole run instead. That’s the decision report, under Decision intelligence, and the run’s AI report with its Agent stages. The walkthrough puts the single-test items in one step to keep it short.
The explanation is labeled rules. Its example classification is Product Bug. That’s one of TestLookup’s categories: Flaky, Product Bug, Infrastructure, Test Data, Automation Defect and Unknown.
No AI model is called.
TestLookup shows this engine as “rules engine”. It labels an explanation written without an AI language model “deterministic”, which means the same input always gives the same answer.
The AI guide on GitHub covers the rules, how TestLookup picks a model, spending limits, and what happens when a step can’t run. The walkthrough doesn’t suggest that a model is available. Its example score isn’t a measured confidence.
When the evidence it needs is missing, the walkthrough gives the category Unknown. Its summary reads “insufficient evidence to determine root cause”, and it shows no confidence score.
Analysis doesn’t depend on integrations with other tools. So this path doesn’t claim that anyone created an issue or sent a notification.
Choose Correct the classification. The correction stays on this page only, and it updates the explanation you see.
Feedback is something TestLookup can learn from later. This step doesn’t train a model, check one or put a new one into use. It also doesn’t create an issue or send a message.
In TestLookup, you correct a result on the test result page:
TestLookup saves the correction next to the original, with your name, as a new version in that test result’s AI history. It uses the correction the next time it analyzes the same test.
Recording a correction doesn’t train a model, check one or put a new one into use. It becomes a label that a separate training run may use later. The walkthrough saves nothing and changes no later analysis.
Back and Reset walkthrough only change what this page shows. Nothing is saved.
The step-by-step text at the end of this page covers the sample run, the evidence, the missing-evidence path and the correction. It’s complete even with JavaScript turned off.
