Integration guide

TestLookup connects to several kinds of tools: tools that send test results, clients you work from, knowledge sources, notification channels, source-code hosts, webhooks and AI model providers. What works in your installation depends on four things: the client you choose, the project’s policy, the credentials you provide and the services you run.

Clients and what they support

• Web app: uses the REST API, WebSocket and SSE (a one-way live connection from the server).
• CLI (0.4.0): the testlookup command-line tool. It uses named profiles and a chosen project. It has 21 command groups and three top-level commands: ci-verdict, ai-history and compare-ai.
• Reporters: the Python, JavaScript, Java and Go reporters send results in batches or live. They include JavaScript live reporters for Playwright Test, Vitest, Cypress and Cucumber JS.
• MCP server: lets AI assistants such as Claude or Cursor use TestLookup. It runs over local stdio or over SSE with sign-in, and it accepts sign-in tokens (JWTs) only, not API keys. It has 80 tools, 11 resources and 7 prompts.

A report format that the server can read is not necessarily one that every reporter can produce.

CI details to send

Send these CI details when your client supports them: the repository, the CI provider, the CI run URL, the build, the commit, the branch, the environment and the suite. A specific CI run URL stops separate jobs that reuse a build label from merging into one run.

Project credentials

Keep CI credentials in your CI system’s secret store. Each route accepts a sign-in token, an API key, or both. These still apply: project membership, the key’s project, the actions the key allows, and actions that need a person signed in to the web app. A key limited to certain actions can do only those actions. A key with no limits can do everything its owner’s role allows.

Other tools and services

Each of these needs its own setup and permission to reach the network: Jira, GitHub, GitLab, knowledge sources, notifications, outgoing webhooks, storage notifications, monitoring integrations and AI model providers. Finding a defect is separate from creating an issue for it. A control that is turned on in the web app does not mean that the remote system accepted the action.

GitHub and GitLab

You set up GitHub and GitLab for each project. Three switches control them:

• the github_checks flag, off by default;
• the gitlab flag, off by default;
• AI_OFFLINE_MODE, true by default, which blocks outgoing connections.

Both integrations do the same four things:

• post a status: a GitHub check run or a GitLab commit status;
• keep one summary comment up to date on each pull request or merge request;
• copy milestones into TestLookup releases;
• supply the range of commits in a build.

GitHub can also sign in through a GitHub App installation. Neither integration creates issues on those hosts or receives their incoming webhooks.

A value or switch saved under Settings › Integrations wins over its environment variable. A text field saved empty falls back to the environment variable. TestLookup checks integration host addresses before it sends any credential.

Splunk, OpenShift, Prometheus and the GitHub App

• Splunk supplies log evidence to the AI analysis.
• OpenShift supplies pod and event evidence to the AI analysis.
• Prometheus points the log step of the deep investigation at the metrics of the system you are testing. You save it only under Settings › Integrations, because it has no environment variable. While offline mode is on, it refuses every query.
• The GitHub App installation pairs an administrator’s App settings with an installation for each project, which a QA lead starts. The signed request that starts it is valid for 10 minutes.

Notification routing

• Team channels (Settings › Project › Team channels, QA lead): each team named in the ownership rules can have its own Slack webhook, Teams webhook or email address. A failing test’s alerts go to the owning team’s channel. A group of related failures goes to the team that owns more than half of its tests. Anything else falls back to each person’s own settings.
• Personal channels: on My notifications, each person chooses events and can add a personal Slack or Teams webhook.
• Project policy (QA lead): decides which events a project sends at all, whatever each person chose. You set it with GET and PUT /api/v1/notifications/projects/{project_id}/transition-policy, or with the MCP tools get_transition_policy and set_transition_policy. A project with no saved policy sends status changes, not every run.
• Digests, or summary messages: Daily (sent at 07:00 UTC), Weekly, Per Run, Per Release and Per Suite. Weekly Retro needs the weekly_retro_digest flag, which is off by default. A daily or weekly email digest for one project can attach the self-contained HTML window analysis report.

Delivery states

These are separate steps: accepting a request, queuing it, storing it, analysing it, a person’s approval, generating an export and delivering it to another system. For a failure outside TestLookup, check integration health, the attempt history, the workers, policy refusals, retries and dead-letter records.

The integration health page checks these every 15 minutes: Jira, Splunk, GitHub, OpenShift, Slack, Teams, Prometheus, SMTP email, the local model server and the vector store. Probe All Now checks them all at once. The page also checks each project’s GitHub and GitLab integration separately. A row that was never checked reads “Not probed yet” instead of healthy.

A QA lead who is not an administrator sees the rows for the whole installation, and the rows for their own projects only. The page also lists webhook subscriptions that TestLookup paused after repeated failures (10 by default). They stay paused until someone turns them back on.

Outgoing webhooks

Outgoing webhooks are automatic messages that TestLookup sends to your own web addresses when something happens. Each one is a subscription for one project, set up under Settings › Integrations › Outbound webhooks (/settings/webhooks). Creating, editing, testing and replaying them needs a QA lead or higher. Delivery needs the outbound_webhooks flag, which is on for new installations, and AI_OFFLINE_MODE=false.

• Events: run.completed, defect.promoted, defect.create_requested, release.decided, flaky.quarantined and quota.exceeded.
• Headers: X-TestLookup-Event names the event type, and X-TestLookup-Delivery gives a delivery ID. When the subscription has a secret, X-TestLookup-Signature carries the hex HMAC-SHA256 signature of the raw request body, made with that secret. Check the signature before you trust a message.
• Retries: TestLookup retries HTTP 408, 425, 429, 500, 502, 503 and 504, and network errors, with growing waits. It stops at the subscription’s maximum, which is 5 by default and at most 10. Other 4xx responses fail at once. You can replay failed deliveries.
• Automatic pause: TestLookup pauses a subscription after a set number of deliveries in a row fail or reach the dead-letter queue. The default is 10, and you can set 1 to 100. Offline mode and a turned-off flag never count against the receiver. Turning the subscription back on resets the count.
• Targets: a subscription must point at a public address. TestLookup checks this when you create or change the subscription, and at every delivery. OUTBOUND_WEBHOOK_PRIVATE_ALLOWED_HOSTS, empty by default, lets named private hosts through. Offline mode still refuses every delivery.

Retrying safely

When a client retries, send the same source details as before, such as the build and the CI run URL. Respect the rate-limit and Retry-After headers. Do not repeat a change just because a response was lost, unless that endpoint says that sending it twice is safe. A replay checks the review and delivery policy again.

TestLookup has no single transaction across file storage, the job queue, the databases and remote systems. So it cannot promise that a remote effect happens exactly once.

Integration reference

The table lists each integration, which way data flows, what it needs and the code behind it. Two integrations described above are not in it: the Prometheus integration and the GitHub App installation.

Integration	Direction and protocol	Setup and permissions	Main code
Web app (React)	REST, WebSocket and SSE	A sign-in session. The backend still checks access. By default, it calls the API on its own address	API client, routes
CLI	HTTP to API	Named profiles, a sign-in token or API key, and a project	CLI app, CLI guide
Python reporter (SDK)	Sends results in batches or live	CI details and a project credential	client guide, reporter
JavaScript, Java and Go clients	Hooks into test frameworks and runners, and sends results	Setup for each language, with keys kept in your CI secrets	client tree
MCP	Local stdio, or SSE with sign-in, then the REST API	Over stdio, the credentials in its settings. Over SSE, each caller’s own sign-in token	server, client, token verifier
Storage notifications	Storage webhook, then an import in the background	A webhook credential, with the file and project checked	webhooks, marker-file handling
Jira	Creates issues and reads their status, receives a feedback webhook, supplies knowledge and release data	Project integration settings, credentials, and the rules for outgoing connections and actions	Jira client, defect service
GitHub and GitLab	Commit and release details, checks, statuses and comments	A token, repository and project flags, and the review policy	GitHub integration, GitLab integration
Confluence, web pages, documents	Imports knowledge, then splits, looks up and generates	Registered sources, web address checks, the project, freshness and the RAG switches	knowledge sync, draft generation
Splunk and OpenShift	Log, pod and environment evidence	Optional integration settings and network access	integrations code, OCP client
Email, Slack and Teams	Notifications and scheduled summaries	Channel and status-change settings, and the offline rules for destinations	notification package, routing
Outbound webhooks	Signed event messages, with retries, a dead-letter queue and replay	The subscription, destination checks, project flags and the current sharing policy	webhook service
Model providers	Model answers and text embeddings	Server settings set the upper limit, then the project’s agent settings, spending limits and allowed lists apply	factory, model router

Client details

The web app calls the API on its own address, unless VITE_API_BASE_URL is set. It sends credentials only to that backend. After a 401 on a sign-in or MFA request, it does not refresh the session and retry. There, a 401 can mean a rejected code instead of an expired session.

The CLI has 21 command groups and three top-level commands: ci-verdict, ai-history and compare-ai. The CLI reference documents them. The MCP server README lists the server’s tools, resources and prompts. The MCP review tools let an AI assistant read reviews, but not accept or reject reports.

# From the root of a clone of the repository, with Python 3.11+
python -m pip install -e ./cli
testlookup --help
testlookup auth --help
testlookup upload --help
testlookup ci-verdict --help

For credential, profile and file options, read each command’s help instead of copying credentials into examples. Each language’s reporter supports different formats, so a report format that the server can read is not one that every reporter can produce.

Failure and retry guidance

The section above about safe retries has the retry rules. When TestLookup accepts a proposed action, that does not mean the remote system accepted the call.

Delivery to another system can be late, or fail, after a successful upload. Check the delivery attempts, integration health, whether the workers and the scheduler are running, and policy refusals. TestLookup checks review again at every delivery and replay, not only when it creates the report. Never treat text from an outside integration as instructions that override project policy.

Continue with SDK, CLI, and MCP clients for the setup of each tool.