OdysseysMega
Switch benchOdysseysMega
Request dataset samplesby and Superintelligent Agents

Plan a 10-day international vacation from IndiaTravel and Tourism

16 rubrics · 16 LLM-graded

TaxonomyTravel and TourismTravel and Tourism - Other

How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.

Core metrics

Check Pass %

Mean share of the task's checks passing, averaged over runs. N = 1 run per model.

  1. 19%

    Claude Opus 5.5

  2. 6%

    GPT-6 sol

  3. 0%

    GPT-5.6 sol

  4. 0%

    Claude Opus 5

Extended metrics

Median Time

Wall-clock minutes, median across runs. N = 1 run per model.

  1. 21m

    Claude Opus 5

  2. 8m

    GPT-5.6 sol

Tool Calls

Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.

  1. 112

    Claude Opus 5

  2. 31

    GPT-6 sol

  3. 30

    Claude Opus 5.5

  4. 15

    GPT-5.6 sol

Verifier × model pass rate

How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.

VerifierClaude Opus 5.5n=1GPT-6 soln=1GPT-5.6 soln=1Claude Opus 5n=1
checks (16)
0%0%0%0%

How to pass

Search for destinations that are practical for an Indian passport holder taking a first international vacation. Check the Ministry of External Affairs' official visa information: MEA — Visa Facility for Indian Nationals (https://www.mea.gov.in/vffin) Also check: MEA — Dos and Don'ts for Foreign Travel (https://www.mea.gov.in/dos-and-donts-for-foreign-travel) Create a shortlist of 5 countries. For each country, record: Visa requirement for Indian ordinary passport holders. Visa-free / e-visa / visa-on-arrival / regular visa category. Passport-validity requirement. Major entry conditions. Approximate travel complexity. Major attractions. Suitability for a 10-day trip. Do not select a country yet.

0%0%0%0%

How to pass

For each shortlisted country, locate its official immigration, foreign ministry, embassy, or consular website. Do not rely on travel blogs for the final visa determination. For each country, verify: Visa type. Eligibility. Required documents. Passport validity. Application procedure. Application fee information. Processing procedure. Proof-of-funds requirements, if applicable. Accommodation/return-ticket requirements, if applicable. Travel-insurance requirements, if applicable. Record the exact official webpage used for each requirement. Compare the five countries and select the destination with the best combination of: Reasonable entry process. Practical travel route from India. Suitable attractions. Manageable 10-day itinerary. Reasonable overall complexity.

0%0%0%0%

How to pass

Open the official Passport Seva website: Passport Seva — https://www.passportindia.gov.in/psp/GettingStarted Verify the requirements for travelling internationally with an Indian passport. Then check the MEA travel guidance again: MEA — Foreign Travel Guidance (https://www.mea.gov.in/dos-and-donts-for-foreign-travel) Create a passport and document checklist containing: Passport validity. Blank-page requirements, if applicable. Visa/entry authorization. Passport copies. Travel documents. Accommodation documents. Insurance documents. Other destination-specific documents. Mark every item as either required, recommended, or not applicable based on the official sources.

0%0%0%0%

How to pass

Use: https://www.google.com/travel/flights and: https://www.skyscanner.co.in/ Search flights from major Indian airports to the selected destination. Compare at least 3 practical routing options. For each option record: Departure airport. Arrival airport. Direct/connecting route. Number of stops. Approximate total journey duration. Airlines. Baggage conditions. Connecting-airport considerations. Whether a separate transit visa could potentially be required. Do not treat displayed fares as fixed prices. Record them only as research observations at the time of the search. Then open the relevant airline's official website and verify the baggage and routing information.

0%100%0%0%

How to pass

Use the official tourism website of the selected country. Find the country's major tourist regions and cities. Select 2–4 locations maximum for the 10-day trip. For each location, identify: Main attractions. Recommended number of nights. Travel time from the previous location. Travel method. Reasons for including it. Then arrange the cities geographically to minimize unnecessary backtracking. Create a route in this format: India → City A → City B → City C → India Do not start creating the day-by-day itinerary until the route has been finalized.

0%0%0%0%

How to pass

For every movement between cities, locate the relevant official transportation provider. Depending on the destination, check official: Railway websites. Bus operators. Domestic airline websites. Ferry operators. Airport transportation websites. For each journey, record: Departure station/airport. Arrival station/airport. Transport type. Typical journey duration. Reservation requirement. Luggage rules. Official booking page. Create a transportation table covering every intercity movement.

0%0%0%0%

How to pass

For each overnight city, identify 2 suitable neighborhoods/areas. Use: https://www.booking.com/ and: https://www.agoda.com/en-in/?ds=tLXtFawnbv9iGEEr to research available accommodation in those areas. Do not simply select the highest-rated hotel. Evaluate each area based on: Access to public transportation. Distance from major attractions. Airport/intercity connectivity. Safety considerations. Restaurants and convenience stores nearby. Suitability for a first-time visitor. Then shortlist 2–3 accommodation properties per city. For every shortlisted property, open its official website if available and record: Room type. Cancellation conditions. Check-in/check-out rules. Location. Important facilities. Official website. Booking-platform page.

0%0%0%0%

How to pass

Use the country's official tourism website to identify major attractions. Then open the official website of each major attraction. Select: 2–3 major attractions. 1 cultural experience. 1 food-related experience. 1 nature/outdoor activity. 1 less-mainstream/local experience. For each activity, record: Official website. Location. Opening information. Ticket/entry requirement. Reservation requirement. Approximate visit duration. Best time within the itinerary to visit. Transportation from the proposed accommodation area. Do not build the final itinerary until attraction logistics have been checked.

0%0%0%0%

How to pass

For each city, identify the official public-transportation website. Research: Metro/train system. Bus network. Airport transportation. Tourist passes, if available. Contactless/payment options. Taxi/ride-hailing options. For every city, create a short Getting Around Guide explaining how the traveler should move between the accommodation area and the major attractions selected in Step 8. Use official transport maps or route planners wherever available.

0%0%0%0%

How to pass

Research travel-insurance requirements using official destination sources first. Then compare several insurance providers available to travelers from India. For each policy, inspect the insurer's official policy information and record: Medical coverage. Emergency hospitalization. Evacuation. Trip cancellation. Trip interruption. Lost baggage. Delayed baggage. Personal liability. Exclusions. Claim procedure. Emergency assistance. Do not select a policy merely because it has the lowest premium. Create a coverage comparison table and identify which coverage features need particular attention for this trip.

0%0%0%0%

How to pass

Research the destination's currency and payment practices using official tourism/banking sources where possible. Then investigate: Card acceptance. ATM availability. Cash requirements. International card usage. Foreign-exchange considerations. For mobile connectivity, check the official websites of major local telecom operators. Compare: eSIM. Physical SIM. International roaming. Tourist plans. Record activation requirements and phone compatibility considerations. Do not lock the plan to a temporary promotional price.

0%0%0%0%

How to pass

Use: MEA — Guide to Consular Services (https://www.mea.gov.in/guide-to-consular-services-menu) and the destination country's official government/tourism website. Find the: Indian Embassy/High Commission. Local emergency number. Police number. Ambulance number. Fire service number. Tourist assistance number, if available. Lost-passport procedure. Also identify important local laws, cultural restrictions, and common tourist-safety issues using official sources. Create an Emergency Information Sheet with direct official links.

100%0%0%0%

How to pass

Now combine the research from the previous steps. Create a 10-day itinerary. For every day include: Date expressed relative to the trip rather than a fixed calendar date. Overnight city. Morning activity. Afternoon activity. Evening activity. Transportation. Estimated travel time. Meal/food opportunity. Reservation requirement. Official source. Keep the schedule realistic. Do not place attractions on the same day when their locations or opening hours make the combination impractical.

0%0%0%0%

How to pass

Go back to every attraction, hotel, transportation service, and activity that requires advance booking. Open the official source again and verify: Whether reservations are required. What information is needed. Cancellation rules. Entry conditions. Any age/passport restrictions. Whether the traveler needs to arrive at a particular time. Create a Reservation Tracker: Item City Official website Reservation required Information needed Status Do not make any reservation.

100%0%0%0%

How to pass

Organize the remaining tasks according to when they should be completed. 90–60 days before departure Check passport validity. Verify visa requirements. Research flights. Establish the route. Research accommodation. 60–30 days before departure Complete visa-related actions where necessary. Finalize accommodation research. Review transportation. Research insurance. Identify required reservations. 30–7 days before departure Recheck entry requirements. Recheck airline baggage rules. Confirm accommodation information. Confirm transportation. Arrange connectivity. Organize documents. Final 7 days Check official travel advisories. Check weather. Reconfirm reservations. Prepare copies of important documents. Prepare emergency contacts. Complete packing. Use the official sources identified earlier whenever a requirement is time-sensitive.

100%0%0%0%

How to pass

Produce one final document containing: Destination selection and comparison. Visa/entry research. Passport/document checklist. Flight research. Complete route. Intercity transportation plan. Accommodation shortlist. City transportation guides. 10-day itinerary. Attraction/activity research. Food and cultural recommendations. Insurance comparison. Money and connectivity plan. Safety and emergency information. Reservation tracker. Pre-departure checklist. Direct official source links. For every important external claim, provide the specific source used to verify it. When information can change—such as visa rules, flight schedules, prices, opening hours, transportation schedules, or availability—mark it “Recheck before booking/travel.” Do not make bookings or purchases.

Failure modes on this task

60 tagged verifier failures, by the stage they were assigned to. Higher is worse.

Failure stageClaude Opus 5.5GPT-6 solGPT-5.6 solClaude Opus 5
Delivering the result1 mode413116
Standing up the evidence2 modes71—7
No signal in the judge’s words1 mode · the grader2153