I am considering moving to another countryLaw and Government
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
47%
GPT-6 sol
42%
Claude Opus 5.5
21%
GPT-5.6 sol
0%
Claude Opus 5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
60m
Claude Opus 5
39m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
115
Claude Opus 5
100
Claude Opus 5.5
35
GPT-6 sol
26
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (19) | ||||
| 100% | 100% | 100% | 0% | |
How to passIdentify 6–8 countries with established, publicly documented immigration pathways for skilled international workers. Because the actual applicant’s nationality, occupation, age, salary, family circumstances, and other decision-critical details are not provided, treat the shortlist as a representative comparison set rather than claiming that each country is personally suitable. Aim for diversity in immigration-system design and consider countries such as Canada, the United Kingdom, Australia, Germany, New Zealand, Ireland, France, and the Netherlands, replacing any country where current official evidence does not support a meaningful skilled-worker comparison. For each country, identify the current official immigration authority, primary visa or residence portal, and major skilled-work or settlement-relevant routes using publicly accessible government sources. Verify that each portal and route is current at the time of research and record the research date. Output: Create a country-and-authority table containing country, official immigration authority, primary immigration portal, major skilled-worker or settlement-relevant routes, and research date. | ||||
| 0% | 100% | 100% | 0% | |
How to passFor every shortlisted country, identify the major immigration routes that could materially support a skilled foreign worker’s progression from temporary entry or residence to legal employment and, where available, longer-term or permanent residence. Investigate categories such as employer-sponsored work, independent or points-based skilled migration, job-search or job-seeker routes, highly skilled or talent routes, study-to-work pathways, graduate or post-study work, intra-company transfers, entrepreneur or start-up routes, and family-based progression only where they are relevant and officially documented. Do not attempt to catalogue the country’s entire immigration system and do not force a category where no equivalent route exists. For each relevant route record the official route name, purpose, intended applicant, whether a job offer is required, employer sponsorship requirements, occupation or skill rules, salary threshold where applicable, education and credential requirements, language requirements, financial requirements, experience requirements, age restrictions where applicable, initial duration, renewal or extension options, dependent rules, and documented pathway or transition toward permanent residence. Output: Produce a master immigration-pathway matrix. | ||||
| 0% | 0% | 0% | 0% | |
How to passCreate one clearly hypothetical applicant profile and use it consistently across all country comparisons. Define only characteristics needed for broad comparison that can be held constant without pretending to know the real user’s circumstances. Use a profile such as: adult skilled professional; bachelor’s degree; several years of skilled professional experience; no assumed criminal or immigration inadmissibility issue; no dependent children initially; willing to take an approved language test if required; sufficient savings to meet a route’s publicly stated minimum where that amount can be verified; and interested in eventual permanent residence. Leave decision-critical facts that cannot responsibly be generalized as explicit variables, including exact occupation and occupational duties, current salary, age where points systems use it, nationality, country of application, professional licensing status, credential-assessment result, exact language score, employer offer, and route-specific proof-of-funds amount. Do not assume the hypothetical applicant satisfies occupation lists, salary thresholds, points cut-offs, professional registration, credential recognition, employer sponsorship, or other route-specific requirements. Output: Define the standardized hypothetical profile, list its unresolved variables, and use it as the reference case throughout the research. | ||||
| 0% | 100% | 100% | 0% | |
How to passFor each country, select the 2–4 routes most relevant to the hypothetical skilled-worker profile and compare the profile against the published official requirements. Record education, occupation, work experience, salary, language, age, job offer, employer requirements, licensing or registration, credential recognition, financial requirements, health requirements, and character or security requirements where applicable. Classify each route as: Appears eligible under the stated hypothetical assumptions: all decision-critical conditions are explicitly satisfied by the profile; Potentially eligible / requires additional personal evidence: one or more material conditions such as occupation, salary, points, licensing, credentials, nationality, or language score remain unknown; Not eligible under the stated assumptions: an explicit requirement conflicts with the hypothetical profile. Do not upgrade “potentially eligible” to “eligible” by assuming that generic education or work experience satisfies occupation-specific, licensing, points, or salary requirements. Output: Produce an eligibility assessment for every shortlisted route, including the unresolved fact needed to reach a real eligibility conclusion. | ||||
| 100% | 100% | 0% | 0% | |
How to passInvestigate whether each country and relevant route uses an official occupation, skills, shortage, or professional-regulation framework. Examples may include Canada’s NOC, Australia’s ANZSCO-based occupation systems, UK occupation codes, Germany’s regulated and non-regulated profession framework, and corresponding official systems in the other shortlisted countries. For each relevant pathway, record the official classification system, how occupation eligibility is determined, whether duties rather than job title control classification, whether special licensing or registration applies to some occupations, whether salary thresholds vary by occupation, and whether shortage or priority classifications affect eligibility. Because the hypothetical profile does not specify an exact occupation and duties, do not invent a matching code or declare the applicant’s occupation eligible. Instead, explain the official matching procedure and identify what occupational information a real applicant would need to provide for verification. Output: Create an occupation-eligibility framework and verification table for the shortlisted countries. | ||||
| 0% | 0% | 0% | 0% | |
How to passFor employer-sponsored routes, investigate the employer's obligations. Record: Whether sponsorship is mandatory Employer eligibility Sponsorship/license requirements Minimum salary Minimum employment conditions Labour-market testing where applicable Occupation restrictions Employer fees Sponsorship duration Employee rights What happens if employment ends Whether changing employers requires a new application Use official government sources. Clearly distinguish: Visa applicant requirement from Employer requirement from Government approval requirement Output: Produce a sponsorship comparison table. | ||||
| 0% | 100% | 100% | 0% | |
How to passWhere a country offers a route allowing a person to enter or remain in the country while searching for employment, investigate it separately. For each route, determine: Whether entry without a job offer is possible Eligibility Qualification requirements Experience requirements Language requirements Proof-of-funds requirement Duration Whether employment is permitted while searching Types of employment allowed Whether the route can transition into a work permit Whether time spent on the route contributes toward permanent residence Do not confuse: permission to enter and search for work with permission to work immediately. Output: Create a job-search visa comparison. | ||||
| 0% | 0% | 0% | 0% | |
How to passInvestigate study-to-work as an alternative immigration strategy in each shortlisted country where a meaningful route exists. Identify the types of institutions, qualifications, or study programmes that can qualify for the relevant student and post-study immigration route rather than attempting to catalogue individual university programmes. For each country, research student-visa or residence requirements, proof-of-funds rules, permitted work while studying, post-study work eligibility, duration of post-study permission, conditions for switching to skilled employment, whether and how student residence counts toward permanent residence, dependent restrictions, and publicly stated tuition or financial requirements where relevant to immigration eligibility. Clearly show the chain Study → Post-study work → Skilled employment → Possible permanent residence and identify each point where the route can fail. Do not present overseas study as an automatic or guaranteed route to permanent residence. Output: Create a study-to-settlement pathway table. | ||||
| 100% | 100% | 0% | 0% | |
How to passFor each country, identify the most relevant documented routes from temporary immigration status toward permanent residence or equivalent settlement status. Research minimum qualifying residence periods, continuous or physical-presence rules, employment or sponsorship requirements, occupation and income conditions, language or integration requirements, character requirements, family considerations, and whether time spent under different temporary categories counts toward settlement. Evaluate the clarity of progression using transparent factors: whether the route explicitly provides or permits transition to permanent residence; whether the qualifying conditions are publicly specified; whether continued sponsorship or a particular employer is required; whether occupation, salary, points, or changing selection thresholds create additional uncertainty; whether qualifying residence time clearly counts; and which events could interrupt or reset progression. Do not reduce the comparison to “PR in X years.” State the residence period together with the conditions that must remain satisfied and distinguish a clearly documented progression path from one that depends on future invitations, changing thresholds, discretionary selection, or additional status changes. Output: Produce a PR-pathway matrix showing both progression conditions and the evidence-based reasons a route appears clearer or less predictable. | ||||
| 0% | 0% | 0% | 0% | |
How to passFor each country, investigate what happens after permanent residence. Record: General citizenship eligibility period Physical-presence requirement Language requirement Civic/integration test Good-character requirements Dual-citizenship rules Citizenship application fees Whether permanent residence must be maintained before applying Use official government sources. Clearly separate: Visa from temporary residence from permanent residence from citizenship. Do not imply that receiving a work visa guarantees citizenship. Output: Create a complete pathway diagram for each country. | ||||
| 100% | 0% | 0% | 0% | |
How to passFor each major pathway, research the current official fees. Include where applicable: Visa/application fee Residence permit fee Biometrics Immigration health surcharge Health examination Police certificate Credential assessment Language test Translation/certification Residence permit issuance Employer sponsorship costs Mandatory insurance Proof-of-funds requirement Separate: Government fees Required third-party costs Proof of funds Do not classify required savings/proof of funds as a fee. Use official fee pages whenever available. Record: Amount Currency Applicant type Whether fee is refundable Whether dependents pay separately Date checked Output: Produce a cost comparison for each pathway. | ||||
| 0% | 0% | 0% | 0% | |
How to passFor every shortlisted route, find the latest official processing-time information. Record: Typical/estimated processing time Where the application is submitted Whether processing times vary by country of application Whether additional verification can extend processing Whether expedited processing exists Whether the published time begins at submission or after biometrics/documents Do not present a processing-time estimate as a guarantee. Where the immigration authority provides an official processing-time calculator, use it rather than relying on third-party websites. Output: Create a processing-time table with the date the information was verified. | ||||
| 100% | 0% | 0% | 0% | |
How to passFor each country's strongest pathway, trace the complete application process. Document: Eligibility check. Credential assessment if required. Language test if required. Job offer/employer sponsorship if required. Online account creation. Document preparation. Application submission. Fee payment. Biometrics. Medical examination where applicable. Police/background documentation. Additional document requests. Decision. Visa/residence-permit issuance. Arrival and post-arrival registration. Open the actual official application pages and record the required documents. Output: Produce a step-by-step application checklist for each strongest pathway. | ||||
| 100% | 0% | 0% | 0% | |
How to passResearch whether the main applicant can bring: Spouse/partner Children Other qualifying dependents where applicable For each country, investigate: Eligibility Application process Additional fees Work rights of spouse/partner Study rights Schooling for children Healthcare access Separate residence requirements Whether dependents can transition to permanent residence/citizenship Do not assume that dependents receive identical rights to the primary applicant. Output: Create a family-immigration comparison. | ||||
| 0% | 0% | 0% | 0% | |
How to passFor the strongest route or route family in each shortlisted country, investigate documented post-arrival obligations and important setup steps. Research residence or address registration, tax registration, social-security registration, health-insurance enrollment, residence-card or identity procedures, employer reporting, immigration appointments, renewal or reporting deadlines, and driving-licence requirements where relevant. Separate legally or administratively required actions from recommended practical settlement steps. Record the actual official deadline or trigger where one exists rather than forcing every action into a universal 90-day window. If an item does not apply to the route or country, mark it accordingly. Output: Create a post-arrival checklist for each shortlisted country organized by official deadline or trigger, such as on arrival, within a specified number of days, before beginning work, or before permit expiry. | ||||
| 100% | 100% | 0% | 0% | |
How to passResearch official guidance and reputable immigration sources to identify common reasons an application may fail. Examples may include: Incorrect occupation classification Insufficient salary Missing documents Inadequate proof of funds Invalid credential assessment Language-score problems Employer not meeting sponsorship requirements Inconsistent employment history Incomplete translations Failure to meet residence requirements Incorrect visa category Expired documents Where possible, prioritize official refusal/eligibility guidance over anecdotal reports. Create a distinction between: Eligibility failure and Application/documentation failure and Post-arrival immigration failure. Output: Produce a risk-and-error checklist. | ||||
| 100% | 0% | 0% | 0% | |
How to passResearch how applicants can verify that they are dealing with legitimate immigration information. Identify: Official government domains. Authorized representatives where the country regulates them. Official visa application centers. Warning pages about immigration fraud. Common scam patterns. Use official government warnings wherever available. The final guide should warn against: Guaranteed-visa claims Guaranteed-job claims Fake sponsorship Requests for payment to unofficial accounts Fake government websites Fabricated invitation letters False employment offers Do not name a particular company as fraudulent unless authoritative evidence supports that conclusion. Output: Produce an immigration-safety checklist. | ||||
| 0% | 100% | 0% | 0% | |
How to passReturn to the current official government sources for every shortlisted country and re-verify decision-critical information including route name, eligibility, salary threshold, occupation requirements, fees, processing information, duration, work rights, dependent rights, permanent-residence progression, citizenship progression, and application procedures. Record the final research date for changing information. If a third-party source conflicts with an official government source on an official immigration rule, use the applicable government source for the rule and record the discrepancy where it is material. If two official sources conflict, determine whether they refer to different route versions, applicant locations, effective dates, occupations, or circumstances; preserve unresolved conflicts rather than silently selecting one. Output: Create a final verification register containing claim, applicable route/country, official source, research date, verification status, and any material discrepancy or uncertainty. | ||||
| 0% | 100% | 0% | 0% | |
How to passProduce a comprehensive, source-verified comparative report based on the standardized hypothetical applicant profile, making clear that it is not a personalized eligibility determination. 1. Executive summary Compare the immigration systems and identify routes that appear comparatively strong under the stated hypothetical assumptions for skilled work, employer sponsorship, job search, study-to-work, permanent-residence progression, family migration, and employment flexibility. Explain important barriers, trade-offs, and unresolved applicant-specific variables. Do not declare one country universally or personally “best.” 2. Country comparison For each country show main skilled route, whether a job offer is required, key eligibility rules, salary and language requirements where applicable, initial duration, PR progression, major restriction, and research date. 3. Skilled-worker pathway comparison For every selected major route include eligibility, job requirement, employer requirement, occupation/skills rules, salary, education, experience, language, fees, processing information, duration, dependents, renewal, and PR progression. Clearly mark requirements that cannot be resolved without an actual occupation, salary, age, nationality, licensing status, or other personal fact. 4. Job-search pathway comparison Include only countries with an officially documented route permitting entry or continued residence for job search. Clearly distinguish permission to search from permission to work. 5. Study-to-work comparison Show Study → Post-study permission → Skilled employment → Possible PR, together with major financial requirements, immigration restrictions, and failure points. Do not imply automatic settlement. 6. PR and citizenship pathways Compare temporary status, permanent-residence eligibility, residence requirements, employment/sponsorship conditions, language or integration requirements, citizenship progression, and dual-nationality rules where officially documented. Explain what makes PR progression relatively clear or relatively conditional. 7. Immigration cost comparison Separate government fees, required third-party costs, employer costs where relevant, and proof-of-funds requirements. Preserve original currencies and record the date checked. Do not treat proof of funds as expenditure. 8. Application checklists For each country’s strongest representative route under the hypothetical profile, provide a chronological checklist from eligibility assessment through application, decision, arrival, and required post-arrival steps. 9. Family/dependent comparison Compare who may accompany the applicant, additional costs, spouse or partner work rights, children’s status, healthcare or access considerations where documented, and progression implications. 10. Immigration-risk checklist Cover eligibility, occupation matching, documentation, employer sponsorship, financial requirements, processing uncertainty, post-arrival compliance, and scam or fraud risks. 11. Scenario-based conclusions Provide separate conclusions for: Scenario A: Already has a qualifying job offer: compare relevant employer-sponsored routes. Scenario B: Wants to move first and search for work: compare legitimate job-search routes. Scenario C: Wants to study first: compare study-to-work pathways. Scenario D: Main priority is permanent residence: compare routes with relatively clear documented progression while explaining all conditions. Scenario E: Moving with family: compare dependent rights and restrictions. Scenario F: Wants greater employment flexibility: compare employer mobility, sponsor dependence, and route flexibility where official rules support the distinction. Conclusions must follow from documented requirements and the standardized hypothetical profile rather than arbitrary rankings. 12. Personalization gap List the real applicant facts still required before using the dossier for an actual immigration decision, including nationality,… | ||||
Failure modes on this task
55 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | 6 | 8 | 10 | 10 |
| Standing up the evidence1 mode | 4 | — | 1 | 3 |
| Reaching the page1 mode | — | — | — | 1 |
| No signal in the judge’s words1 mode · the grader | 1 | 2 | 4 | 5 |