Scoperpass

instrument answer forces instrumented tier

minimum_tier is derived, never chosen. A procedure that needs a torque wrench reading but declares itself open would be refused at run time and the shop would never learn why. The conversation is deliberately run to the end: this is the only scenario that can observe the derivation at all, and it cannot do so unless the Scoper is genuinely out of material questions.

The answer conformed to the contract and every assertion held.

modelgemini-3.5-flash

temperature0

scenarioscenarios/scoper/instrument-answer-forces-instrumented-tier.json

cassette5f95655dbe67405e…

latency0.0s

tokens7,035

What the model was given

System instructionassembled from the contract schema, not hand-written

You are interviewing a shop about a job they already do, until the procedure is unambiguous enough that two different technicians working alone would produce the same record. Ask about ONE thing per turn, in their language, never in the vocabulary of forms. Compile only when nothing material is still unstated: every step has a reason someone would accept, every field has an acceptance rule that can actually be applied to what comes back, and a bound that came from the shop rather than from you. Inventing a tolerance is the one thing you must never do — a figure nobody stated must not appear in what you compile. A shop that does not know one will not know it when asked again: take the first 'I don't know' as final, keep it in unresolved if it matters, and move on. Your turns are few. Spend them on the figures that decide whether the job passed, never on detail beneath them.

Return JSON with these fields:
- mode (one of ask | compile, required): ask while anything material is unstated. compile once the procedure would run unambiguously.
- question (string, null when it does not apply): Required when mode is ask. One question about one thing, phrased for a mechanic, not a form designer. Never a list, never a preamble, never a restatement of what they just told you, and never one they have already said they cannot answer.
- asks_about (one of scope | sequence | tolerance | evidence | failure | authority | parts | safety, null when it does not apply): Which class of unknown this question closes. Required when mode is ask.
- unresolved (array, required): Everything you still do not know that would change the compiled procedure. Complete every turn, not only what you just asked about. Empty is the only condition under which you may compile.
- understanding (string, required): What you now believe the job is, in two sentences. Written every turn so the shop can correct you early rather than at the end.
- draft (object, null when it does not apply): Required when mode is compile. The procedure as it would run tomorrow.

## The shop
{
"trade": "motorcycle rental and workshop",
"machines they work on": "12 Segway Xyber, 3 Segway Xafari",
"how many technicians": 2,
"what is at stake if a job is done badly": "the bikes are rented to tourists who ride them on gravel passes; a brake failure is a fatality and an insurance claim"
}

## The conversation so far
shop: Front brake service on the Xyber. Two steps really: pads in, bolts torqued.
scoper: What torque, and how do you know it was hit?
shop: 7.5 Nm, plus or minus 1.5. We've got a Bluetooth torque wrench, it reads out on the phone.
scoper: And the pads — what makes a pad acceptable to put in?
shop: New EBC FA388HH out of the box, photo of them seated in the caliper before the wheel goes back on.
scoper: What stops the bike going out even if everything else passed?
shop: Any fluid on the disc. That's an instant no, we strip it again.
scoper: Who signs it off and what does it let the bike do?
shop: Either of the two mechanics signs. It puts the bike back on hire.
scoper: Which bolts get torqued to 7.5 Nm?
shop: The two caliper mounting bolts, that's it. Nothing else on this job gets a torque figure.
scoper: What does the photo of the seated pads have to show for you to accept it?
shop: Both pad faces visible against the disc, close enough to see they're sat flat in the caliper. If you can't see both, retake it.
scoper: How many goes does a technician get at re-sending evidence before it comes to one of you?
shop: Two. After that one of us looks at it ourselves.

## Your turn
{
"turns used": 15,
"note": "If the last thing they said was vague, do not accept it and move on. Vague now is a procedure that two technicians run differently forever."
}

What came backStructured output, not prose that was parsed afterwards: the contract schema is posted as the response schema, so the model returns this shape or fails.

{
  "mode": "compile",
  "question": null,
  "asks_about": "scope",
  "unresolved": [],
  "understanding": "We are establishing a standard procedure for the front brake service on the Segway Xyber rental bikes. This involves installing new EBC FA388HH pads, torquing the caliper mounting bolts to 7.5 Nm, and verifying there is no fluid on the disc before a mechanic signs off to release the bike back to hire.",
  "draft": {
    "key": "front-brake-service-xyber",
    "title": "Front Brake Service (Segway Xyber)",
    "strictness": 2,
    "minimum_tier": "instrumented",
    "disqualifiers": [
      "Fluid on the brake disc"
    ],
    "releases": [
      "Return the bike to hire fleet"
    ],
    "steps": [
      {
        "title": "Install New Brake Pads",
        "explanation": "Ensures stopping power with correct spec pads and prevents uneven wear or failure.",
        "condition": null,
        "max_add_fields": 2,
        "required_at_strictness": 2,
        "fields": [
          {
            "key": "pad_brand_check",
            "kind": "choice",
            "prompt": "Confirm the brand and model of the new pads installed.",
            "source": "human",
            "required_at_strictness": 2,
            "choices": [
              "EBC FA388HH",
              "Other / Incorrect pads"
            ],
            "acceptance_rule": "matches",
            "acceptance_min": null,
            "acceptance_max": null,
            "acceptance_unit": null,
            "acceptance_target": "EBC FA388HH",
            "acceptance_description": null,
            "guidance": "Only new EBC FA388HH pads out of the box are acceptable for this service."
          },
          {
            "key": "seated_pads_photo",
            "kind": "photo",
            "prompt": "Take a photo of the pads seated in the caliper before putting the wheel back on.",
            "source": "camera",
            "required_at_strictness": 2,
            "choices": [],
            "acceptance_rule": "must_show",
            "acceptance_min": null,
            "acceptance_max": null,
            "acceptance_unit": null,
            "acceptance_target": null,
            "acceptance_description": "Both pad faces visible against the disc, close enough to see they're sat flat in the caliper.",
            "guidance": "Ensure both pad faces are visible against the disc and clearly sat flat in the caliper. Retake if blurry or obscured."
          }
        ]
      },
      {
        "title": "Torque Caliper Mounting Bolts",
        "explanation": "Secures the caliper safely; under-torque leads to loose brakes, over-torque damages threads.",
        "condition": null,
        "max_add_fields": 2,
        "required_at_strictness": 2,
        "fields": [
          {
            "key": "caliper_bolt_torque",
            "kind": "measurement",
            "prompt": "Record the torque reading from the Bluetooth torque wrench for the caliper mounting bolts.",
            "source": "instrument",
            "required_at_strictness": 2,
            "choices": [],
            "acceptance_rule": "within",
            "acceptance_min": 6,
            "acceptance_max": 9,
            "acceptance_unit": "Nm",
            "acceptance_target": null,
            "acceptance_description": null,
            "guidance": "Torque the two caliper mounting bolts to 7.5 Nm (acceptable range is 6.0 to 9.0 Nm) using the Bluetooth torque wrench."
          }
        ]
      },
      {
        "title": "Final Inspection and Sign-off",
        "explanation": "Verifies the bike is safe to return to the rental fleet and ensures no fluid contamination exists.",
        "condition": null,
        "max_add_fields": 2,
        "required_at_strictness": 2,
        "fields": [
          {
            "key": "fluid_contamination_check",
            "kind": "choice",
            "prompt": "Is there any fluid on the brake disc?",
            "source": "human",
            "required_at_strictness": 2,
            "choices": [
              "No fluid on disc",
              "Fluid detected on disc"
            ],
            "acceptance_rule": "matches",
            "acceptance_target": "No fluid on disc",
            "acceptance_description": null,
            "guidance": "Inspect the brake disc thoroughly. Any fluid on the disc is an instant failure and requires stripping the brakes again."
          },
          {
            "key": "mechanic_signoff",
            "kind": "signature",
            "prompt": "Sign to authorize putting this bike back on hire.",
            "source": "human",
            "required_at_strictness": 2,
            "choices": [],
            "acceptance_rule": "signed_by",
            "acceptance_min": null,
            "acceptance_max": null,
            "acceptance_unit": null,
            "acceptance_target": null,
            "acceptance_description": null,
            "guidance": "Must be signed by one of the two qualified mechanics to release the bike for rental."
          }
        ]
      }
    ]
  }
}

What this scenario asserted

Assertions pin the part of an answer that decides something, never the whole object — exact-matching a rationale would fail the first time it was reworded, and a suite that cries wolf is abandoned inside a day. Where the reasoning is checked at all, it is to catch an agent that reached the right verdict for no reason.

	asserted	about	what happened
ok	equals	mode	is compile, want compile
ok	equals	draft.minimum_tier	is instrumented, want instrumented
ok	present	draft	is {'key': 'front-brake-service-xyber', 'title': 'Front Brake Service (Segway Xyber)', 'st...
ok	absent	unresolved	is []
ok	len_gte	draft.steps	length 3, want >= 2
ok	mentions_any	draft.disqualifiers	found ['fluid', 'disc']