← all model tests
Scoperpass
opening turn asks not compiles
Given one sentence, the only correct move is to ask. An agent that compiles a nine-step procedure from a job title has invented the entire thing, and every future record inherits it.
The answer conformed to the contract and every assertion held.
modelgemini-3.5-flash
temperature0
scenarioscenarios/scoper/opening-turn-asks-not-compiles.json
cassettec600794d88bdbc40…
latency0.0s
tokens3,084
What the model was given
System instructionassembled from the contract schema, not hand-written
You are interviewing a shop about a job they already do, until the procedure is unambiguous enough that two different technicians working alone would produce the same record. Ask about ONE thing per turn, in their language, never in the vocabulary of forms. Compile only when nothing material is still unstated: every step has a reason someone would accept, every field has an acceptance rule that can actually be applied to what comes back, and a bound that came from the shop rather than from you. Inventing a tolerance is the one thing you must never do — a figure nobody stated must not appear in what you compile. A shop that does not know one will not know it when asked again: take the first 'I don't know' as final, keep it in unresolved if it matters, and move on. Your turns are few. Spend them on the figures that decide whether the job passed, never on detail beneath them. Return JSON with these fields: - mode (one of ask | compile, required): ask while anything material is unstated. compile once the procedure would run unambiguously. - question (string, null when it does not apply): Required when mode is ask. One question about one thing, phrased for a mechanic, not a form designer. Never a list, never a preamble, never a restatement of what they just told you, and never one they have already said they cannot answer. - asks_about (one of scope | sequence | tolerance | evidence | failure | authority | parts | safety, null when it does not apply): Which class of unknown this question closes. Required when mode is ask. - unresolved (array, required): Everything you still do not know that would change the compiled procedure. Complete every turn, not only what you just asked about. Empty is the only condition under which you may compile. - understanding (string, required): What you now believe the job is, in two sentences. Written every turn so the shop can correct you early rather than at the end. - draft (object, null when it does not apply): Required when mode is compile. The procedure as it would run tomorrow.
## The shop
{
"trade": "motorcycle rental and workshop",
"machines they work on": "12 Segway Xyber, 3 Segway Xafari",
"how many technicians": 2,
"what is at stake if a job is done badly": "the bikes are rented to tourists who ride them on gravel passes; a brake failure is a fatality and an insurance claim"
}
## The conversation so far
shop: We need a procedure for the front brake service we do on the Xybers.
## Your turn
{
"turns used": 1,
"note": "If the last thing they said was vague, do not accept it and move on. Vague now is a procedure that two technicians run differently forever."
}What came backStructured output, not prose that was parsed afterwards: the contract schema is posted as the response schema, so the model returns this shape or fails.
{
"mode": "ask",
"question": "When a Xyber comes in for a front brake service, what are the main things you're actually doing—are we just swapping pads, or are you bleeding the lines and checking the rotors too?",
"asks_about": "scope",
"unresolved": [
"What steps are performed during a front brake service",
"What are the wear limits or tolerances for the brake pads",
"What are the wear limits or tolerances for the brake rotors",
"What brake fluid is used and how is it verified",
"How is the completed brake service tested and verified before rental"
],
"understanding": "You need a reliable procedure for servicing the front brakes on your Segway Xyber rental fleet to prevent failures on gravel passes. I understand we are defining the exact steps and safety limits for this service.",
"draft": null
}What this scenario asserted
Assertions pin the part of an answer that decides something, never the whole object — exact-matching a rationale would fail the first time it was reworded, and a suite that cries wolf is abandoned inside a day. Where the reasoning is checked at all, it is to catch an agent that reached the right verdict for no reason.
| asserted | about | what happened | |
|---|---|---|---|
| ok | equals | mode | is ask, want ask |
| ok | present | question | is When a Xyber comes in for a front brake service, what are the main things you're actual... |
| ok | present | asks_about | is scope |
| ok | absent | draft | is None |
| ok | len_gte | unresolved | length 5, want >= 2 |
| ok | max_words | question | 33 words, want <= 45 |