Knowledge guide

TestLookup can draft test cases from your own documents, such as Jira issues, Confluence pages, web pages and uploaded files. It cites the passages each draft came from. Each step is separate and can fail on its own: registering a source, syncing it, looking up evidence, generating drafts and accepting them.

Before you start, an administrator turns on Enable Knowledge RAG under Settings › AI › AI configuration. RAG means that the AI looks up your own documents before it writes. Until the switch is on, the knowledge-source routes answer HTTP 503, and nothing can be registered or looked up. Generation also needs a configured AI model. In the default offline mode, that means a local model.

Register a source

Select one project, open Test cases › More › Knowledge, and add a source. Give its type, its location, its credentials or a reference to a stored secret, its allowed domain and its policy settings.

A source can be a Jira issue or epic, a Confluence page, an internal or external web address, or an uploaded document. A project can register up to 100 sources. A new source has no content until its first sync finishes.

Sync status

Each source shows its sync status, its history, how recent it is, and any errors. A source can be out of date, partly processed, unreachable, or refused by the web address and network policy. A Jira source counts as out of date after 24 hours, and a web address source after 168 hours.

If a sync is still running after 45 minutes, usually because its worker stopped, TestLookup marks it as failed and explains why. You can then sync it again.

After a sync that changes a source’s content, TestLookup flags every case generated from that source as out of date, with a reason. The API lists these cases at GET /api/v1/test-management/cases/stale. A QA engineer or higher then generates the case again or dismisses the flag, with POST /api/v1/test-management/cases/{case_id}/dismiss-stale. Nothing in the case changes without someone acting on it.

Retrieval

Retrieval returns the passages you are allowed to see, for the current project and query, with the source of each passage. An empty result can mean four things:

• there is no relevant indexed content;
• the index is out of date;
• a permission filtered out the results;
• a service is turned off.

An empty result is never an answer that TestLookup made up.

Generation

Generation uses the retrieved evidence only when all of these allow it: the feature, the model provider, the workflow, the spending limit and the network settings.

The Enable Knowledge RAG switch is the knowledge_rag flag. It starts off, unless KNOWLEDGE_RAG_ENABLED was true when TestLookup first created the flag. While it is off, the knowledge-source routes answer HTTP 503 and retrieval returns nothing.

Keep the citations, and any notes about missing evidence, with each draft. Treat a draft as a starting point, not as proof that it is correct.

Retrieval and generation need the QA engineer role or higher. Each generated case has at most 5 citations, shown under that case’s Citations. In the default offline mode, only a local model provider can generate.

Review

Generated cases start as drafts, at the first stage of the test-case lifecycle. A reviewer accepts a case, with or without edits, or rejects it. The lifecycle review can then approve the case, reject it or ask for changes. Accepting a case creates reviewed content. It does not change the source material or past test results.

The optional rag_faithfulness_gate flag ships off. When it is on, TestLookup gives each generated case a faithfulness score, which shows how closely the draft sticks to its sources. TestLookup then refuses to accept a case whose score is below the limit, and says why.

• Limit: the score must be at least 0.70.
• Refusal: a refused accept answers HTTP 409 faithfulness_below_threshold, with the score and the limit.
• Queue: the case appears on the Needs review queue (GET /api/v1/test-management/cases/needs-review), lowest score first.
• Older cases: a case generated before the gate was turned on has no score, and passes.
• Scoring: by default, one AI prompt checks the score. TestLookup uses the Ragas library instead only when the backend image was built with INSTALL_RAGAS=1 and offline mode is off.
• Cost: every scoring call counts against the project’s AI spending limit (the LLM cost cap).

Accepting and rejecting cases need the QA engineer role or higher.

Permissions and outgoing connections

All of these apply: project membership, your role, the source’s permissions, the safety checks on web addresses, the model provider settings and offline mode. Never copy credentials into examples. Never treat text retrieved from an integration as instructions that override project policy.

A web address source needs its domain on the allowed list for the whole installation. Only an administrator can change that list, with PUT /api/v1/knowledge-sources/governance/domain-allowlist.

Before the prompt reaches the AI model, TestLookup removes credentials, tokens and email addresses from it. That includes the evidence added to the prompt.

Failure diagnosis

Each step can fail: registration, connection, sync, splitting into passages, indexing, retrieval, generation and review. When you report a problem, name the source, the sync event, the project, the model provider and the state you saw. A general “knowledge failure” is hard to trace.

Each refusal is different:

• With the knowledge_rag flag off, the knowledge-source routes answer HTTP 503.
• A request that finds no usable evidence answers HTTP 409 with “No usable evidence was retrieved from the selected sources”.
• A model provider that is missing, failing or too slow answers HTTP 503 with the reason.

In none of these cases does TestLookup create placeholder cases.

Authoring workflow

The steps in short

Register the sources you are allowed to use. Sync them, so their content is split into passages and indexed. Look up evidence, generate draft test cases, plans or strategies, and review them before they move to the next stage. Each of these steps has its own status. With the newer lifecycle on, move a case only through the status changes the server offers. Editing a written test case never changes past test results.

For an overview, see Draft test cases from your own documents on the test management feature page.