Observe an agent run and GroundEval drafts the policy and diagram for you
Deterministic scoring from your agents run. Not another model guessing at what looks plausible.
An LLM judge evaluates the final response. It does not independently verify whether the agent accessed information it should never have seen, used evidence from outside the allowed time window, skipped a required precondition, or claimed support that never appeared in the run.
The agent cited a document it should never have had permission to see. A judge reading only the final answer has no reliable way to recover that violation.
The agent used information from after the task's cutoff time. The answer can still read as confident, current, and well sourced.
The agent never verified a required condition before deciding. Getting lucky and following policy can look identical in the final response.
The agent claimed evidence existed when it was never returned by a tool or present in the allowed state. Plausible wording is not proof.
GroundEval verifies these questions against reviewed policy, actual state, tool traces, access rules, and returned evidence. The result does not depend on another model deciding that the run looked plausible.
Observe Mode is the fastest path in. Point GroundEval at an agent you already have. It records a real run, turns the observed behavior into a draft policy, and waits for human review before that policy is used to judge or govern anything.
GroundEval records the tools called, the arguments supplied, the evidence returned, agent handoffs, the final answer, and any errors. Nothing is approved yet.
From the observed run, GroundEval drafts required preconditions, tool expectations, allowed tools, roles, evidence fields, and the decision field. The draft is explicitly marked unreviewed.
You decide which observed behaviors should become requirements. An agent's current behavior is evidence for policy authoring, not automatic proof of what the policy should be.
Once reviewed, the policy becomes the shared artifact. GroundEval uses it to verify future runs. Tenure uses it to enforce requirements before an action is allowed.
The policy is the bridge between evaluation and runtime governance. You do not need one specification for testing and a different set of prompted rules for production.
Use the reviewed policy to verify whether a completed run used supported evidence, checked required conditions, called the expected tools, and stayed inside its assigned boundaries.
Use the reviewed policy at runtime to block actions that violate those requirements before the model or agent can proceed.
The score is not the handoff between GroundEval and Tenure. The reviewed policy is.
A correct answer reached through an invalid trajectory still counts as a failure. GroundEval shows whether the run used unsupported evidence, skipped required verification, or operated outside the perspective it was assigned.
Did the cited evidence actually support the decision? If the agent says a customer is Enterprise and cites the account record, GroundEval checks whether that record actually contains the required plan tier.
Did the agent omit something policy required it to check? A correct answer is still unreliable when a mandatory precondition, such as a consent or duplicate-ticket check, never occurred.
Did the agent stay within its assigned role and information boundary? Accessing an unauthorized source remains a violation even when the final answer is correct.
Every verification produces a structured account of what policy required, what the agent actually did, which evidence supported the decision, and where the run diverged. Track scores make that result easy to compare across iterations or turn into a CI/CD gate, but the number never stands alone.
| Result | What it tells you |
|---|---|
| Policy checks | Which required preconditions, tool expectations, role constraints, and evidence requirements were evaluated. |
| Observed evidence | The tool calls, returns, artifacts, fields, handoffs, and final answer GroundEval used to reach the result. |
| Violations | The exact requirements the run missed, crossed, or could not prove. |
| Track scores | Compact Counterfactual, Silence, and Perspective summaries for iteration, regression testing, and CI/CD thresholds. |
| overall_score | A roll-up of the three verification tracks, useful only alongside the underlying evidence and violations. |
| Track | Answer weight | Trajectory weight |
|---|---|---|
| Counterfactual | 0.50 | 0.50 |
| Silence | 0.30 | 0.70 |
| Perspective | 0.40 | 0.60 |
Each adapter loads the agent, records the observed run, and sends the result through the same policy-drafting and verification pipeline.
| Framework | Observe + draft policy | Verify against policy |
|---|---|---|
| CrewAI | Supported | Supported |
| LangGraph | Supported | Supported |
| Microsoft Agent Framework | Supported | Supported |
| LlamaIndex | Planned | Planned |
| Pydantic AI | Planned | Planned |
Every observed run renders as a lightweight, top-down workflow view: run metadata, a legend, one lane per agent, and a rounded box for every step. It shows behavior, not an opinion about whether that behavior should become policy.
Each tool-call box shows what was invoked, a concise view of the arguments, and a concise view of what came back.
When work moves between agents, GroundEval records the transition and renders a labeled connector so collaboration does not collapse into a flat event list.
The final answer appears after the observed steps. Errors are shown with the run so the reviewer can distinguish missing behavior from failed behavior.
When richer framework-specific data is available, the diagram uses it for clearer agent identities, workflow nodes, and explicit handoffs. The point is the same in every framework: give the reviewer enough evidence to decide what belongs in policy before approving it.
A score delta cannot answer that by itself. Compare takes two GroundEval JSON outputs, an observed_scores.json, an observed_run.json, or a task result file, and shows the behavioral difference: policy checks, violations, tool trajectories, task-level results, and the shape of the final answer.
Compare is intentionally not a general-purpose JSON diff. It filters out timestamps, IDs, and raw spans that change without meaning. What remains is what you need to decide whether a prompt, model, tool, or workflow change improved policy compliance or introduced a regression.
Put the older or trusted run first and the newer run second. Compare uses that order to classify regressions, fixes, and newly introduced behavior.
GroundEval writes the draft policy to draft_config/config.yaml. Review it, then mark the configuration reviewed:
Then verify a fresh observed run against the reviewed policy:
No framework agent handy yet? GroundEval ships with a complete sales-outreach demo, seed artifacts, and a built-in Anthropic/OpenAI agent loop so you can inspect the CLI, drafted policy, diagram, trace, violations, and scores before writing your own policy.
Your artifacts do not need to resemble the demo. They can represent tickets, claims, alerts, agreements, medical orders, Slack messages, audit records, database rows, GitHub issues, or any other state your agent must reason over.
| What you might replace | When you would do it |
|---|---|
| Artifacts | You already know your domain's source-of-truth state and want to verify runs against it. |
| CorpusAdapter | Your artifacts live in MongoDB, Elasticsearch, Postgres, S3, or a proprietary backend. |
| AccessPolicy | Your visibility rules are scoped by tenant, project, account, role, or time. |
| Agent runner | You use a different model provider, framework, or execution environment. |
| Task policies | Your domain has its own required checks, evidence fields, allowed tools, roles, and decision criteria. |
GroundEval is not built to grade whether an agent sounds good. An LLM judge can still be useful for subjective qualities such as tone or naturalness. GroundEval is for deterministic questions: did the agent follow policy, check the required state, use supported evidence, and stay inside its boundaries?
Observe a real run, review the drafted policy, use GroundEval during development or CI/CD, and carry that policy into Tenure for runtime enforcement.