OdysseysMega
Switch benchOdysseysMega
Request dataset samplesby and Superintelligent Agents

I’m helping a family member understand the commonlyHealth

17 rubrics · 17 LLM-graded

TaxonomyHealthMedicineHealthHealth - Other

How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.

Core metrics

Check Pass %

Mean share of the task's checks passing, averaged over runs. N = 1 run per model.

  1. 100%

    Claude Opus 5.5

  2. 71%

    Claude Opus 5

  3. 35%

    GPT-6 sol

  4. 24%

    GPT-5.6 sol

Extended metrics

Median Time

Wall-clock minutes, median across runs. N = 1 run per model.

  1. 25m

    Claude Opus 5

  2. 8m

    GPT-5.6 sol

Tool Calls

Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.

  1. 109

    Claude Opus 5

  2. 71

    Claude Opus 5.5

  3. 34

    GPT-6 sol

  4. 15

    GPT-5.6 sol

Verifier × model pass rate

How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.

VerifierClaude Opus 5.5n=1GPT-6 soln=1GPT-5.6 soln=1Claude Opus 5n=1
checks (17)
100%100%0%0%

How to pass

Start with authoritative U.S. sources: https://www.fda.gov/drugs https://www.fda.gov/drugs/information-consumers-and-patients-drugs/find-information-about-drug https://medlineplus.gov/medicines.html https://dailymed.nlm.nih.gov/dailymed https://www.accessdata.fda.gov/scripts/cder/daf/ FDA's Drugs@FDA contains information on FDA-approved prescription and OTC human drugs, while MedlinePlus provides patient-oriented medication information from the National Library of Medicine. Identify the major OTC pain/fever medicine categories relevant to a general U.S. consumer, including at minimum: Acetaminophen Ibuprofen Naproxen Aspirin Do not assume that all products containing these ingredients are interchangeable. Create an initial medication inventory containing: Active Ingredient Common U.S. Brand Examples Drug Class OTC Status Primary Uses Use the active ingredient, not merely the brand name, as the primary identifier.

100%0%0%100%

How to pass

For each active ingredient, locate current U.S. labeling through FDA and/or DailyMed. For each medicine: Search the active ingredient. Open relevant current labeling. Identify the dosage forms covered. Determine whether multiple formulations exist. Record important differences between formulations. Record the label's intended uses. Record warnings and precautions. Record directions for use. Record age-related restrictions or instructions. Save the direct source URL. Do not rely on an old blog post or retailer description when the official label is available. FDA notes that DailyMed provides current marketed-drug labeling, while Drugs@FDA provides FDA-approved labeling and regulatory information; these sources can sometimes differ in update timing.

100%0%0%100%

How to pass

Using MedlinePlus and official labeling, determine the common uses of each medicine. For every active ingredient, document: Pain relief Fever reduction Inflammation-related uses where applicable Common conditions for which consumers may encounter the medicine Whether the use is explicitly included in U.S. OTC labeling Whether the information comes from labeling or general patient education Separate: FDA/label-supported uses from General medical information Do not present an off-label or commonly discussed use as though it were an FDA-approved indication.

100%100%100%100%

How to pass

Research reputable medical sources to explain the fundamental differences between the medicines. For each medicine, determine: Drug class General mechanism of action Whether it has meaningful anti-inflammatory activity Whether it affects platelet function Major organs/systems associated with important safety considerations General duration characteristics where authoritative sources provide them Use reputable sources such as: https://medlineplus.gov/ https://www.nih.gov/ https://www.nlm.nih.gov/ FDA resources Avoid relying on commercial supplement websites or anonymous medical blogs. Produce a concise mechanism and clinical-use comparison table.

100%0%0%100%

How to pass

For each medicine, open the official label and identify the most important warnings. Specifically investigate: Gastrointestinal risks Liver-related risks Kidney-related risks Cardiovascular risks Bleeding risks Allergic reactions Medication-overuse concerns where relevant Risks associated with combining products Age-related considerations Pregnancy-related warnings where applicable Other major boxed or prominent warnings Do not simply copy a list of side effects. Separate: Common side effects from Serious or medically important adverse effects from Situations where the medicine may be inappropriate or require professional guidance

100%0%0%0%

How to pass

Use official and authoritative medication databases to identify important interaction categories. For each medicine, investigate interactions involving: Other pain relievers Blood thinners/anticoagulants Antiplatelet drugs Corticosteroids Certain blood-pressure medicines Alcohol Other medicines containing the same active ingredient Other clinically important interaction categories identified by the official sources Do not attempt to create an exhaustive interaction database. Instead, identify high-value interaction warnings that a general consumer should know about. For every interaction: Identify the medicine combination. Explain why it matters at a high level. State whether the warning comes from FDA labeling, MedlinePlus, or another authoritative source. Avoid making a patient-specific recommendation.

100%0%0%100%

How to pass

This should be a separate research phase. Search FDA and MedlinePlus for information about combination products and duplicate active ingredients. Investigate scenarios such as: Taking a pain reliever together with a cold/flu product Taking two products containing the same active ingredient Using multiple NSAID-containing products Combining prescription and OTC products Brand-name products that contain a less obvious active ingredient Identify practical ways consumers can recognize duplicate ingredients by reading the Drug Facts label. Produce a short "Before Taking a Second Product" checklist based on authoritative information.

100%0%0%100%

How to pass

For each medicine, locate the current U.S. OTC label and extract the relevant consumer directions. Record, where applicable: Minimum age information Frequency instructions Maximum daily amount stated on the relevant label Duration limitations Whether the medicine should be taken with food or water according to labeling What to do if the consumer takes too much When the label directs consumers to ask a doctor or pharmacist Do not combine instructions from different formulations into one universal dosing recommendation. If different formulations have different directions, explicitly identify that difference. The final report must make clear that label instructions vary by product, formulation, strength, age, and circumstance.

100%0%0%100%

How to pass

Use authoritative U.S. sources to investigate how medication safety considerations may differ for: Children Older adults Pregnancy People with liver disease People with kidney disease People with a history of gastrointestinal bleeding/ulcers People taking anticoagulants or other medications that increase bleeding risk Do not create treatment plans for these groups. Instead, identify when official sources say consumers should consult a healthcare professional before use or exercise particular caution. Clearly distinguish: General safety information from Individual medical advice that requires a clinician.

100%0%0%100%

How to pass

For each active ingredient, investigate whether U.S. consumers may encounter different formulations such as: Tablets Capsules Liquid formulations Chewable products Extended-release products Combination products Determine whether the formulation changes: How the medicine should be taken Who can use it How frequently it can be taken Whether it can be substituted for another formulation Do not assume that two products with the same active ingredient have identical instructions.

100%100%100%100%

How to pass

Use FDA sources to verify how the medicines are regulated in the United States. Research: FDA role in OTC medicines OTC monograph framework where relevant FDA-approved drug databases Drug labeling Safety communications Drug recalls where relevant Use: FDA Drug Databases FDA Drug Safety Information FDA states that Drugs@FDA contains most human drug products approved since 1939, while its drug resources also include safety information and databases such as the NDC Directory and Drug Shortages Database. If you encounter a recent safety communication or regulatory change, verify it directly on FDA rather than relying on news coverage.

100%100%100%100%

How to pass

Check the FDA website for significant current or recent safety information relevant to the medicines being researched. Search for: Recalls Safety communications Label changes Important warnings Manufacturing-related concerns Drug shortages if relevant Do not assume that a recall affecting one manufacturer or product automatically applies to every product containing the same active ingredient. If a recall is found: Identify the specific product. Identify the manufacturer. Identify the affected lot/formulation if stated. Determine whether it affects the entire active-ingredient category or only a specific product. Link to the official FDA notice.

100%100%100%100%

How to pass

Research the practical U.S. consumer side using reputable retailers and price-comparison sources. Check several major U.S. retailers/pharmacies rather than relying on one store. Examples: https://www.cvs.com/ https://www.walgreens.com/ https://www.walmart.com/ https://www.target.com/ Compare representative products based on: Active ingredient Strength Number of units Formulation Approximate displayed price Store/availability information Do not treat displayed prices as permanent. Clearly label prices as observed prices during research. Do not allow price to override medical safety considerations.

100%100%0%100%

How to pass

Perform a dedicated verification pass. For every important safety or usage statement, determine whether it is supported by: FDA labeling MedlinePlus/NLM Another reputable medical or governmental source If two sources appear to disagree: Check whether they refer to different formulations. Check whether one source is older. Check whether the difference concerns OTC labeling versus general medical information. Prefer the current authoritative source for the specific question. Document meaningful discrepancies rather than silently resolving them. Do not merge conflicting information into an unsupported "average."

100%0%0%0%

How to pass

Create a framework that helps a general consumer understand what factors matter when discussing an OTC pain/fever medicine with a pharmacist or healthcare professional. For example, organize considerations around: Type of symptom Presence of inflammation Age Existing medical conditions Current medications Pregnancy considerations History of stomach/bleeding problems Liver considerations Kidney considerations Whether another product is already being taken How long symptoms have persisted Do not say: "Take Medicine X because it is best." Instead explain: "The appropriate choice depends on factors such as X, Y, and Z; the official label and a pharmacist/clinician should be consulted when these factors apply."

100%0%0%0%

How to pass

Create a final evidence matrix with one row per medicine and at least these columns: Medicine Drug Class Common Uses Anti-inflammatory Activity Major Safety Concerns Key Interactions Special Populations Label Source Patient Information Source Every major medical claim should be traceable to an authoritative source. Prioritize: FDA → NIH/NLM → other U.S. government/academic medical sources → reputable medical organizations → secondary sources.

100%0%0%0%

How to pass

Produce a comprehensive but consumer-friendly final report containing: 1. Executive Summary Explain: Which major OTC pain/fever medicines were researched Their broad differences The most important safety themes Why consumers should check active ingredients and labels 2. Medicine-by-Medicine Profiles For each medicine: Active ingredient Drug class Common uses How it generally works Important warnings Major interactions Special-population considerations Formulation differences Official U.S. labeling MedlinePlus information 3. Comparison Table Provide a side-by-side comparison of all researched medicines. 4. Safety Checklist Create a practical checklist covering: Check the active ingredient Check other medicines being taken Check age restrictions Check relevant medical conditions Read the current Drug Facts label Avoid duplicate ingredients Know when professional advice is appropriate 5. U.S. Regulatory and Safety Section Summarize relevant FDA information, recalls, warnings, and regulatory considerations discovered during the research. 6. Consumer Cost/Availability Snapshot Present the observed U.S. prices and availability for representative products, clearly labeled as time-sensitive observations. 7. Source Verification Table For every major medicine, provide: FDA source DailyMed source where applicable MedlinePlus source Other authoritative sources used Date/access context where useful 8. Important Limitations Explicitly state that: This is an information/research report, not a diagnosis or individualized treatment recommendation. OTC labels vary by product and formulation. Medication information can change. A pharmacist or healthcare professional should be consulted for individualized questions, medication interactions, pregnancy, children, older adults, chronic disease, severe symptoms, or uncertain situations. Do not fabricate medical information, drug approvals, dosage instructions, warnings, recalls, prices, or sources. If a fact cannot be verified from an authoritative source, mark it as unverified rather than filling the gap from assumption.

Failure modes on this task

29 tagged verifier failures, by the stage they were assigned to. Higher is worse.

Failure stageClaude Opus 5.5GPT-6 solGPT-5.6 solClaude Opus 5
Delivering the result1 mode—764
Reaching the page1 mode—2——
Standing up the evidence1 mode—1——
No signal in the judge’s words1 mode · the grader—171