Knowledgebase

A knowledgebase is a set of documents that the agents and judges linked to it can read while they work.

It exists so you can give several agents the same reference material, such as a policy, a style guide, or a product catalogue, without copying it into each one’s prompt. You write the documents once, link the knowledgebase to the agents and judges that need it, and update it in one place when the material changes. A judge scoring runs against a policy, for example, can look the policy up instead of relying on what fits in its criterion.

A knowledgebase belongs to the account, and its name is unique within it. Its documents are text: Markdown, plain text, JSON, YAML, or CSV. Each document has a path such as policies/refunds.md, and the folders in a path group documents together. A link joins one knowledgebase to one agent or judge; an agent can be linked to several knowledgebases, and one knowledgebase can serve many agents. The knowledgebase’s description matters more than it looks: agents see it as part of their tool descriptions, so it’s how they decide whether the knowledgebase is worth reading.

How agents read it

When Prefactor hosts a run, as it does for a hosted judge, and the agent or judge has active linked knowledgebases at the start of the run, the run gets three tools: kb_list to list documents, kb_read to read one, and kb_search to search their text. Which tools a run has is fixed when it starts. Each call reads the knowledgebase as it is at that moment, so a document saved during a run is visible to later calls in the same run, and a knowledgebase unlinked or archived during the run stops resolving. Every tool call is recorded as a span on the run, including which knowledgebase and document revisions it read. An agent that runs in your own infrastructure doesn’t get these tools; it can read knowledgebases through the Prefactor API instead.

Search is full-text search over each document’s path and content, with matches in the path ranked higher. Only the first 200,000 characters of each document are searched, but kb_read returns the whole document.

A hosted judge version saved before judges could read knowledgebases doesn’t offer these tools, even when the judge is linked. Saving the judge again creates a new version that does; it’s deployed to your lowest environment first, and higher environments keep the old version until you promote it.

Revisions

Every change to a knowledgebase’s documents, whether creating, editing, renaming, archiving, or deleting one, produces a new revision of the knowledgebase. Agents always read the current revision. Archiving a document removes it from later revisions but keeps its history, and you can unarchive it. Deleting a document removes it and all its revisions permanently, so it can’t be read through any earlier revision either.

Archiving a whole knowledgebase makes it read-only and stops linked agents and judges from reading it. Deleting a knowledgebase removes it, its documents, and all their revisions permanently.

In the web app

• Knowledgebases page — browse your knowledgebases and create new ones.
• Knowledgebase page — manage a knowledgebase’s documents and the agents and judges linked to it.

Related concepts

• Agent — agents and judges are linked to knowledgebases and read them during their runs.
• Account — knowledgebases belong to the account, and their names are unique within it.
• Span — each knowledgebase tool call is recorded as a span, with the revisions it read.
• Environment — a re-saved hosted judge version reaches higher environments only when you promote it.