Integrate
What verification costs your engineering team
One package, one handler, one process that polls. Nothing inbound, nothing in your request path, and nothing of your production traffic leaving your network.
The SDK
Sixty lines, and none of them in your request path
The runner is a separate process from the one serving your customers. It polls for cases, runs them through the same handler your product uses, and posts the answers back. Your production traffic never goes through us, and never comes to us.
import { Presoja } from "@presoja/adapter-sdk";
export const presoja = new Presoja({
apiKey: process.env.PRESOJA_KEY,
system: "claims-qa",
supports: ["models"],
fingerprint: () => ({
system_prompt_sha256: sha256(SYSTEM_PROMPT),
// Honest nulls. A constant here is indistinguishable from a stable
// index and would let a confounded comparison through.
retrieval_index_version: null,
config_sha256: null,
decode_params_sha256: sha256(JSON.stringify({ temperature: 0 })),
}),
});
// Wrapping the client is what makes the fingerprint arrive without
// being plumbed, and what turns a model override into a parameter.
const model = presoja.instrument(createClient(), { role: "generation" });
// Optional: mark your own retrieval, so a trace can say which step
// stopped being called. The name is yours; nothing about it changes.
const retrievePolicy = presoja.traced("retrieve_policy", retrieve);
presoja.handler(async ({ input, tenant }) => {
if (tenant != null && !TENANTS.has(tenant)) {
// Fails the case, not the run.
throw new Error(`unknown tenant: ${tenant}`);
}
// Your own pipeline, unchanged: retrieval, prompt, model, parsing.
const retrieved = await retrievePolicy(String(input.question));
const answer = await yourOwnPipeline(model, retrieved, input.question);
return {
output: answer.text,
// What retrieval actually returned. The span and the score are what let a
// grader tell a hallucination from a passage nobody had a copy of.
retrieved_sources: retrieved.map((result) => ({
id: result.document.id,
uri: result.document.uri,
span: result.span,
score: result.score,
})),
tenant,
};
});import { presoja } from "./app";
// Everything is outbound - register, then poll for work. There is no
// port to open and nothing for a firewall to allow.
const controller = new AbortController();
presoja.start({ signal: controller.signal });- new Presoja({ ... })
- Authenticates with a key you issued in the UI, scoped to one system. The system and its deployments already exist - a runner cannot bring one into being by naming it, because a typo would become a second hash chain nobody notices until an evidence pack comes back empty.
- presoja.instrument(client)
- Wraps your model client. The model id, token usage and latency get captured where they are already known, so nobody has to thread those values through a response object they had no reason to build. It is also what turns a model override from a refactor into a parameter.
- presoja.traced(name, fn)
- Optional. Marks a function of yours - retrieval, usually - so a trace can say which step stopped being called. Structure only: names and timings cross the wire, never arguments or retrieved text.
- presoja.handler(fn)
- Your function, taking one case and running it through your pipeline as-is. Returns the answer, the sources your retrieval actually returned, and the tenant it ran as.
- presoja.start()
- Registers, then polls for work. Everything outbound. No port to open, no address to allowlist, nothing inbound to review.
How you connect
Three connections, and what each one proves
Detection
You won't run third-party code, and you won't build an endpoint.
We call
An API you already have. You change nothing.
It proves
That this case passed every run since March and failed on the 14th - from your output text alone.
It cannot say
Why. A model change, a prompt change, an index rebuild and plain non-determinism are indistinguishable, and the report prints that rather than glossing it.
Attribution
What we ask forYou'll add around sixty lines, and open no inbound path.
We call
Your handler, through the SDK - or a shim returning your models and a prompt hash.
It proves
Which component moved. A delta is pinned rather than guessed at.
It cannot say
Whether a model you have not switched to yet will hold. Only that this one broke.
Migration testing
You can run your pipeline against a model named at call time.
We call
Your pipeline, with overrides - or a shadow deployment.
It proves
Whether the next model holds, before you switch to it rather than after.
The security review
Secure by shape, not by badge
The questions a reviewer asks, answered before they are asked. Each answer is a property of the design rather than a commitment in a contract.
- No inbound path into your network
- Every connection is opened by your process, outbound, on 443. We do not hold a route into your infrastructure narrowly - we do not hold one at all, which is a stronger answer to exfiltration than any allowlist.
- Your production traffic never reaches us
- The only inputs that cross the boundary are the test cases we send you. We see the answers to those, and nothing else your system did that day.
- The key is scoped to one system
- It cannot create a system, cannot reach another one, and rotation overlaps rather than cutting over - so a rotation is never an outage.
- An unknown tenant fails the case, not silently
- Your handler is expected to refuse a tenant it does not recognise. Executing under a fallback configuration would return a plausible answer scored against a gold set built from a different customer's corpus, which is worse than an error.
- PII is removed before a case enters the set
- De-identification runs before anything is stored, and it is measured rather than promised: a labelled corpus and published recall numbers, so 'redacted' is a figure you can read rather than an adjective.
- Tenant isolation is enforced twice
- Once in the application and again inside the database, where row-level security fails closed. Guardrail tests break the build on any table that ships without it.
- The record is tamper-evident to its reader
- Runs are hash-chained and timestamped, and a reader verifies integrity in their own browser - trusting neither our word nor yours.
Certifications
SOC 2-aligned controls, and no badge yet
We operate the controls an audit tests - scoped access, logged changes, role-based administration - and the audit itself follows. Until it does, we say not yet rather than nearly. The whole of what we do not claim
If you can't run our code
The HTTP fallback
You expose an endpoint and we call it. Same request, same response, same contract - what differs is who opens the connection, and that difference is not free.
POST /verify
X-Verification-Run: run_01JR7...
# Same request and response bodies as above. What it costs you is the
# security review the SDK avoids: an inbound route, a credential we
# hold, and an egress IP for your reviewer to allowlist.It costs the security review the SDK avoids
An inbound route into production, a credential we hold and encrypt, and a static egress address per region for your reviewer to allowlist.
The run header is not optional politeness
Every request carries the run id so you can exclude our traffic from your own analytics. Without it we quietly corrupt the resolution-rate number you report to your board, and you find out late.
Under the SDK, that choice is yours
The run id reaches your handler as an argument, so whether it goes into your telemetry is your decision rather than ours.
Conformance
There is a conformance suite, and it is the actual deliverable
Run it against your integration and it tells you what your connection supports, which fields you return, and which of them you return honestly - an endpoint that echoes back whatever tenant it is sent fails, because an echo has told us nothing. What it produces is a capability record, and each absence in it carries its consequence in the report rather than being quietly dropped.
The reference implementation on this page is the one the suite runs against. If a sample here has drifted from it, the sample is wrong.