Where test results come from
Your CI jobs and test runners produce the results. They send them to TestLookup in several ways: an upload in the web app, the command-line tool, a request to the API, or a small live reporter added to your test framework that streams results while the tests run. Teams that can only save files can also drop result files into TestLookup’s file storage.
Whichever way you choose, the results end up in the same place, so the web app, the command-line tool and AI assistants all see the same runs.
The TestLookup server
TestLookup has three main parts that you install and upgrade together:
- The web app is what your team uses in the browser.
- The API answers every request: from the web app, the command-line tool, CI jobs and AI assistants. It replies quickly and hands slow work to the background workers.
- Background workers do the slow jobs, such as reading test reports, running analysis and sending notifications.
The architecture overview on GitHub describes each part in more detail.
Where your data is kept
TestLookup keeps your data in services you run yourself:
- PostgreSQL, a database, holds the main records: projects, runs, tests and decisions.
- MongoDB holds evidence and summaries.
- Valkey (compatible with Redis) runs the job queue between the API and the workers.
- S3-compatible file storage keeps uploaded reports, files and exports.
Each step is saved on its own; no single transaction covers all of these stores. That is why a run can appear before its analysis is ready. For a complete backup, you need PostgreSQL, MongoDB and the file storage, plus the encryption key, as the deployment guide explains. The data model lists what each store holds.
Optional AI
TestLookup works without any AI model. Rules explain failures from the start, and a model trained on your own labelled failures joins in once you have enough of them. You can add an AI model for deeper analysis and for Ask AI. By default, AI runs on your own machine through a local model server. Hosted AI services stay blocked until an administrator allows them.
What happens to a test report
TestLookup accepts a report right away and does the work in the background. Accepted doesn’t mean saved yet, so check the upload or the run before you rely on it.
Steps in this diagram, as text
- Send a test report → Allowed and not overloaded?
- Allowed and not overloaded? → (no) → Refused, or try again later
- Allowed and not overloaded? → (yes) → Accepted right away
- Accepted right away → Report stored, job queued
- Report stored, job queued → Worker reads the results
- Worker reads the results → Results saved as a test run
- Worker reads the results → (if it fails) → Error recorded
- Results saved as a test run → (if it fails) → Error recorded
- Results saved as a test run → Run totals and history updated
- Run totals and history updated → Notifications, if set up
- Run totals and history updated → (later; can fail on its own) → Analysis: rules, ML, optional AI
- Analysis: rules, ML, optional AI → Release recommendation
- Analysis: rules, ML, optional AI → Likely causes and a summary
Ways to run it
Every option runs the same application. Pick the one that fits your team:
- One machine, one command: an install script downloads ready-made images and starts everything with Docker Compose.
- One machine, from the code: clone the project and run one command. It builds TestLookup and adds demo data.
- Several teams: Kubernetes or OpenShift, with ready-made setups for each platform and databases you provide or run in the cluster.
- No internet access: an offline bundle, built on a machine with internet access and copied in. MongoDB and AI models are brought in separately.
Steps in this diagram, as text
No internet – offline bundle: Bundle built with internet access, Copied in, images loaded, Compose, Kubernetes or OpenShift, MongoDB, added separately
Several teams – Kubernetes or OpenShift: Ready-made setups for each platform, Several copies, rolling upgrades, Databases: managed or in the cluster
One machine, from the code: Clone and run make quickstart, Builds the app, adds demo data
One machine, one command: Run the install script, Downloads prebuilt images
- Run the install script → Downloads prebuilt images
- Downloads prebuilt images → Databases run alongside – Docker Compose
- Clone and run make quickstart → Builds the app, adds demo data
- Builds the app, adds demo data → Databases run alongside – Docker Compose
- Ready-made setups for each platform → Several copies, rolling upgrades
- Several copies, rolling upgrades → Databases: managed or in the cluster
- Bundle built with internet access → Copied in, images loaded
- Copied in, images loaded → Compose, Kubernetes or OpenShift
- MongoDB, added separately → Compose, Kubernetes or OpenShift
After any install, check that the parts run the same version, the database is up to date, the workers listen to all their queues, and one test report makes it all the way to a finished run.
How a release decision is made
TestLookup judges a single test run and a whole release in different ways. Reviewing an AI-written report is a separate step that never decides a release on its own.
Steps in this diagram, as text
Report review, a separate step: Person reviews the AI report, Holds back sharing until reviewed, if switched on
For a whole release: Latest result of each test, Open defects and their severity, GO, NO-GO or Not evaluated
For one test run: Results and pass rate, Project policy or defaults, Failure groups and flaky tests, Strictest result wins, GO, Conditional GO or NO-GO, Person may override, with reason
- Failure groups and flaky tests → Strictest result wins
- Project policy or defaults → Strictest result wins
- Results and pass rate → Strictest result wins
- Strictest result wins → GO, Conditional GO or NO-GO
- GO, Conditional GO or NO-GO → Person may override, with reason
- Open defects and their severity → GO, NO-GO or Not evaluated
- Latest result of each test → GO, NO-GO or Not evaluated
- Person reviews the AI report → Holds back sharing until reviewed, if switched on
Trade-offs to know about
- One application, several stores. You get one thing to install and upgrade, but each store has to be backed up, and if one store is down, the work that needs it waits.
- Fast replies, slower results. The API accepts work at once and the workers catch up. Under heavy load, results take longer to appear.
- Retries are safe, but not every side effect happens exactly once. When a sender retries an upload, TestLookup matches it to the same run. Still, it can’t promise that every follow-up step runs exactly once.
- AI is optional and local first. Without a model you get rules and the trained model; with a model you get more detail, at the cost of more memory and time.
For the full list, see limitations and trade-offs.