Source-grounded AI answers from sources too large to prompt
A source-grounded answer is an AI response drawn from your own material instead of the model's memory, with the supporting evidence available for review. When the material is too large to fit in a prompt, answer quality drops. The Distil API handles this by letting you send a source once, get back a reusable context handle, and ask questions against it as often as you need.
Why do large sources break prompts?
Pasting a long document into a prompt looks like reading it, but a model's attention is uneven across a long input. Questions that need several far-apart sections at once tend to drift, even when each section would be answered correctly on its own. This pattern is called context collapse.
How does a Distil context handle compare with prompt caching?
| Prompt caching | Distil handle | |
|---|---|---|
| Lasts | Minutes | 365 days |
| Works with | One provider | Any model |
| Resend the source | Every session | Never |
| Media | Text | Text, image, audio, video |
How do I try it in two calls?
Send your source once to get a handle, then ask the handle as many questions as you like. Use an API key with the compress:write scope.
const api = "https://dev.primarydesignco.com/api/v1";
const headers = {
Authorization: `Bearer ${process.env.DISTIL_API_KEY}`,
"Content-Type": "application/json"
};
// 1. Send once
const { data } = await fetch(`${api}/compress`, {
method: "POST", headers,
body: JSON.stringify({ text: sourceText, task: "Answer rollout questions." })
}).then(r => r.json());
// 2. Ask many times
const answer = await fetch(`${api}/context/run`, {
method: "POST", headers,
body: JSON.stringify({ context_handle: data.context_handle, task: "What blocks the next rollout?" })
}).then(r => r.json());What kinds of sources does it work with?
Docs, code, video, tickets, policies, manuals, audio and images. Demonstrated scale is up to one billion source tokens.
How should I evaluate it before rolling out?
- Define what an acceptable answer is before you test.
- Compare like for like, using the same questions and the same source on your current approach and on a Distil handle.
- Roll out with a fallback so you can fall back to your current approach if quality slips.
Measure your own workflow; any illustrative numbers are not a benchmark. See the workflow audit and the developer docs.
Frequently asked questions
What is a source-grounded answer?
A response drawn from your own source material instead of the model's memory, with the supporting evidence available for review.
What is a Distil context handle?
A reusable hosted reference to a source you sent once. You ask questions against the handle instead of resending the source.
How is a handle different from prompt caching?
Prompt caching lasts minutes and works with one provider. A Distil handle lasts 365 days, works with any model, and does not require resending the source.
How long do handles last, and can I delete them?
Handles live for 365 days. You can delete a handle at any time, and deleting is free.
How is usage paid for?
With prepaid credits. See the plans page for current options.
Build on it
Open the studio, compare plans, or request enterprise access.