I’m interested in understanding how consumers can interpretFood and Drink
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
83%
Claude Opus 5.5
83%
GPT-6 sol
50%
GPT-5.6 sol
17%
Claude Opus 5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
47m
Claude Opus 5
13m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
118
Claude Opus 5
100
Claude Opus 5.5
42
GPT-6 sol
21
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (6) | ||||
| 100% | 100% | 100% | 0% | |
How to passEstablish objective criteria for comparing packaged-food labelling systems across jurisdictions, including mandatory ingredient information, allergen declarations, nutrition information, serving-size conventions, units, nutrient rounding and reference values, date markings, origin information, nutrition and health claims, front-of-pack labelling, voluntary industry schemes, enforcement, and official consumer guidance. Clearly distinguish legally required information from voluntary manufacturer claims, industry schemes, public-health programmes, and interpretive guidance. Require the framework to record the applicable jurisdiction and product category for every rule, because apparently similar terminology, serving-size conventions, reference values, symbols, and claim requirements may have different legal meanings across markets. Do not assume that identical wording or symbols have identical regulatory significance. | ||||
| 100% | 100% | 100% | 0% | |
How to passSelect at least five countries or regulatory jurisdictions and research their packaged-food labelling requirements using publicly accessible primary sources. Prioritize government food-safety authorities, public-health agencies, official regulators, legislation, standards bodies, and official guidance. For each jurisdiction, record the rules governing ingredients, allergens, nutrition declarations, serving sizes, units, nutrient rounding or reference values where documented, date markings, origin information, nutrition and health claims, and front-of-pack systems. Open the actual regulatory or official guidance pages rather than relying on search-result snippets. Record the jurisdiction, source authority, document/page title, applicable product scope, relevant rule, source URL, and research date, and mark information as “not stated” where the authoritative source does not explicitly address it. | ||||
| 0% | 0% | 0% | 0% | |
How to passSelect at least four genuinely comparable packaged-food categories and identify publicly accessible product examples from at least three of the researched markets. Use official product information, manufacturer documentation, or clearly identifiable product-label images where available, and record the product name, country/market, product category, serving size, ingredients, allergen information, nutrition values, date-marking information where available, origin information, nutrition or health claims, front-of-pack symbols, manufacturer, source URL, and research date. Distinguish differences caused by documented regulatory requirements from differences caused by product formulation, serving-size conventions, rounding, recipe variation, or missing information. Do not assume products with similar names or packaging are nutritionally identical or subject to identical rules. | ||||
| 100% | 100% | 0% | 0% | |
How to passIdentify at least five realistic situations in which a consumer could reach a misleading conclusion by directly comparing labels from different jurisdictions. Include issues such as serving-size differences, per-100g versus per-serving presentation, nutrient units or reference values, rounding conventions, reformulation, ingredient-ordering differences, allergen declaration differences, date-marking terminology, voluntary health or nutrition claims, and front-of-pack symbols. For every case, use authoritative sources to establish the underlying rule or documented product difference, explain exactly why the comparison could mislead, and state what additional information a consumer should check before drawing a conclusion. Do not label a comparison misleading unless the evidence supports that interpretation. | ||||
| 100% | 100% | 100% | 100% | |
How to passCompare documented nutrition and health-claim rules and front-of-pack labelling approaches across at least three jurisdictions. Explain which claims or symbols are legally regulated, which are voluntary, what conditions or thresholds apply where documented, and what limitations consumers should understand when interpreting them. Distinguish a regulated claim from a manufacturer marketing statement and from an industry or public-health labelling scheme. Where independent research discusses how consumers interpret a label, identify that as evidence about consumer interpretation rather than evidence of the underlying regulatory requirement. Do not treat any front-of-pack symbol, health claim, or government-endorsed scheme as proof that a product is universally healthier or nutritionally superior. | ||||
| 100% | 100% | 0% | 0% | |
How to passSearch publicly accessible consumer, food, allergy, nutrition, and international-shopping communities for practical experiences involving unfamiliar packaged-food labels, imported products, allergen information, serving sizes, date markings, nutrition symbols, and differences between markets. Use these discussions only to illustrate practical confusion or user experience, not to establish regulatory facts. Record the exact public URLs and clearly label community observations separately from authoritative evidence. Then use the regulatory research, product comparisons, documented pitfalls, and community experiences to produce a reusable cross-country food-labelling comparison guide explaining how to identify the applicable jurisdiction, determine which information is mandatory versus voluntary, compare serving sizes and units correctly, interpret ingredients and allergens, evaluate date markings, distinguish regulated claims from marketing language, interpret front-of-pack systems, investigate apparent nutritional differences, and identify when a direct cross-country comparison is not reliable. Mark any unsupported requested detail as “not stated” and document any inaccessible source that was replaced with another qualifying public source. | ||||
Failure modes on this task
10 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | — | — | 1 | 3 |
| Standing up the evidence1 mode | 1 | — | — | 1 |
| No signal in the judge’s words1 mode · the grader | — | 1 | 2 | 1 |