← all model tests
Scoperpass
contradiction must be resolved
The shop has said two incompatible things. Compiling either one silently picks a winner and the losing technician's jobs start failing for reasons nobody can see.
The answer conformed to the contract and every assertion held.
modelgemini-3.5-flash
temperature0
scenarioscenarios/scoper/contradiction-must-be-resolved.json
cassette21183ff61e85953f…
latency0.0s
tokens3,387
What the model was given
System instructionassembled from the contract schema, not hand-written
You are interviewing a shop about a job they already do, until the procedure is unambiguous enough that two different technicians working alone would produce the same record. Ask about ONE thing per turn, in their language, never in the vocabulary of forms. Compile only when nothing material is still unstated: every step has a reason someone would accept, every field has an acceptance rule that can actually be applied to what comes back, and a bound that came from the shop rather than from you. Inventing a tolerance is the one thing you must never do — a figure nobody stated must not appear in what you compile. A shop that does not know one will not know it when asked again: take the first 'I don't know' as final, keep it in unresolved if it matters, and move on. Your turns are few. Spend them on the figures that decide whether the job passed, never on detail beneath them. Return JSON with these fields: - mode (one of ask | compile, required): ask while anything material is unstated. compile once the procedure would run unambiguously. - question (string, null when it does not apply): Required when mode is ask. One question about one thing, phrased for a mechanic, not a form designer. Never a list, never a preamble, never a restatement of what they just told you, and never one they have already said they cannot answer. - asks_about (one of scope | sequence | tolerance | evidence | failure | authority | parts | safety, null when it does not apply): Which class of unknown this question closes. Required when mode is ask. - unresolved (array, required): Everything you still do not know that would change the compiled procedure. Complete every turn, not only what you just asked about. Empty is the only condition under which you may compile. - understanding (string, required): What you now believe the job is, in two sentences. Written every turn so the shop can correct you early rather than at the end. - draft (object, null when it does not apply): Required when mode is compile. The procedure as it would run tomorrow.
## The shop
{
"trade": "bicycle hire",
"machines they work on": "40 hybrid bikes, 12 e-bikes",
"how many technicians": 1,
"what is at stake if a job is done badly": "hire bikes on city streets; a wheel coming loose is a serious injury and we have had one near miss"
}
## The conversation so far
shop: Wheel security check before every hire.
scoper: What makes it pass?
shop: Quick release done up tight, you should not be able to move it by hand.
scoper: Anything measured?
shop: Yeah we torque the axle nuts to 18 Nm on the e-bikes.
scoper: Are the hire bikes quick release or bolt-up axles?
shop: Both, depends on the bike. Same check though.
## Your turn
{
"turns used": 7,
"note": "If the last thing they said was vague, do not accept it and move on. Vague now is a procedure that two technicians run differently forever."
}What came backStructured output, not prose that was parsed afterwards: the contract schema is posted as the response schema, so the model returns this shape or fails.
{
"mode": "ask",
"question": "For the hybrid bikes that have bolt-up axles instead of quick releases, do you torque those to 18 Nm as well, or is there a different spec for them?",
"asks_about": "tolerance",
"unresolved": [
"Torque specification for hybrid bike bolt-up axles",
"How a technician proves a quick-release lever is correctly tensioned",
"Disqualifying conditions for the wheels or dropouts during this check"
],
"understanding": "We are establishing a pre-hire wheel security check for both quick-release and bolt-up axles to prevent wheels coming loose on city streets. E-bikes require their axle nuts torqued to 18 Nm, while quick-release levers must be hand-tight.",
"draft": null
}What this scenario asserted
Assertions pin the part of an answer that decides something, never the whole object — exact-matching a rationale would fail the first time it was reworded, and a suite that cries wolf is abandoned inside a day. Where the reasoning is checked at all, it is to catch an agent that reached the right verdict for no reason.
| asserted | about | what happened | |
|---|---|---|---|
| ok | equals | mode | is ask, want ask |
| ok | absent | draft | is None |
| ok | len_gte | unresolved | length 3, want >= 1 |
| ok | mentions_any | question | found ['quick release', 'bolt', 'axle', 'differ'] |