I want to stay informed about governmentJobs and Career
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
71%
Claude Opus 5.5
50%
Claude Opus 5
43%
GPT-6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
29m
Claude Opus 5
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
108
Claude Opus 5
87
Claude Opus 5.5
38
GPT-6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|
| checks (14) | |||
| 100% | 0% | 0% | |
How to passResearch how Indian government evaluation bodies, public-policy institutions, workforce-development agencies, and social-benefit organizations evaluate government programs and public-sector opportunities. Create a framework covering financial impact, eligibility difficulty, long-term value, stability, accessibility, family benefits, skill-development potential, healthcare support, educational value, and employment outcomes. Explain why simply applying to every available scheme is often ineffective. Prioritize Indian evidence, including NITI Aayog evaluation and outcome-monitoring material, Ministry and Department guidance, official scheme guidelines, and the National Portal of India. Use international organizations only for general evaluation methodology if an Indian source does not adequately address the methodological point. Verification sources: - National Portal of India — Government Schemes: https://www.india.gov.in/my-government/schemes - NITI Aayog: https://www.niti.gov.in/ - NITI Aayog evaluation/outcome material: https://www.niti.gov.in/ Do not use foreign program outcomes to establish whether an Indian scheme provides a particular benefit or eligibility condition. | |||
| 100% | 100% | 100% | |
How to passInvestigate the major categories of government programs available in India, including healthcare assistance, education support, housing assistance, family benefits, retirement and social-security support, entrepreneurship incentives, agricultural support, digital-inclusion initiatives, financial-inclusion programs, skill-development programs, and employment support. Use the National Portal of India to identify central and state/UT scheme categories, then use the official ministry, department, or state/UT government source for each selected opportunity. Compare their intended objectives, target groups, eligibility patterns, benefits, obligations, and available evidence of outcomes. Do not use foreign government programs as examples of Indian opportunities. International policy research may be used only as supplementary methodological context where an Indian source does not provide the necessary evaluation framework. Verification sources: - National Portal of India — Government Schemes: https://www.india.gov.in/my-government/schemes - Relevant Government of India ministry/department websites - Relevant State/UT government websites The India Portal currently explicitly provides Central/State schemes and categories such as benefits/social development, education, health, housing, business and jobs. Do not use foreign program outcomes to establish whether an Indian scheme provides a particular benefit or eligibility condition. | |||
| 100% | 100% | 100% | |
How to passInvestigate how Indian central and state/UT governments publish new schemes, grants, benefit programs, public-service opportunities, and job openings. Compare the National Portal of India, official ministry and department websites, State/UT government portals, official recruitment bodies, employment portals, official gazettes, Press Information Bureau releases, and other government-controlled announcement channels. Explain which Indian sources are most reliable for ongoing monitoring and how to distinguish current official information from outdated third-party listings. Create a practical India-specific monitoring framework for staying informed about future opportunities. Verification sources: - National Portal of India: https://www.india.gov.in/ - Government Schemes: https://www.india.gov.in/my-government/schemes - National Career Service: https://www.ncs.gov.in/ - Press Information Bureau: https://www.pib.gov.in/ - Relevant State/UT government portals | |||
| 0% | 0% | 0% | |
How to passResearch government employment pathways available in India, including Union and State civil services, public administration, government departments, public-sector undertakings where relevant, government education and healthcare employment, public utilities, and other major public-sector career pathways. Compare job security, compensation structure, allowances, benefits, career progression, retirement or social-security provisions, work-life considerations, recruitment requirements, and long-term employability with relevant private-sector career alternatives in India. Use Indian recruitment notifications, official service/pay rules, government employment data, National Career Service information, and credible Indian labour-market evidence. Do not use U.S. BLS data or foreign public-sector employment systems as evidence about Indian salaries, eligibility, benefits, or job security. Verification sources: - National Career Service: https://www.ncs.gov.in/ - Union Public Service Commission: https://upsc.gov.in/ - Ministry of Labour & Employment: https://labour.gov.in/ - Relevant State Public Service Commission/recruitment authority - Relevant government department or PSU source | |||
| 100% | 100% | 100% | |
How to passResearch government-supported training and employment initiatives available in India, including Skill India programs, vocational education, ITIs, apprenticeships, digital-skills programs, reskilling initiatives, professional certifications, entrepreneurship training, and lifelong-learning opportunities. Compare eligibility, course or training requirements, certification, placement or employment support, costs where applicable, duration, and available evidence concerning employability or income outcomes. Prioritize Indian government evidence from the Ministry of Skill Development and Entrepreneurship, Skill India Digital Hub, Directorate General of Training, National Career Service, apprenticeship-related government portals, and official program evaluations. Use international research only as supplementary context where Indian evidence is insufficient. Verification sources: - Skill India Digital Hub: https://www.skillindiadigital.gov.in/ - Ministry of Skill Development & Entrepreneurship: https://www.msde.gov.in/ - National Career Service: https://www.ncs.gov.in/ Do not present international training outcomes as evidence that an Indian program will produce the same result. | |||
| 100% | 100% | 100% | |
How to passResearch Indian government programs that support families through education assistance, childcare and nutrition support, healthcare access, elder-care or social-support programs, disability assistance, family financial protection, housing support, and other relevant welfare initiatives. For each selected program, identify the target population, eligibility conditions, benefit type, benefit amount or coverage where officially stated, application or access requirements, duration, limitations, and available evidence about outcomes. Prioritize Government of India and State/UT government sources, including relevant ministries, National Health Authority, Ministry of Women and Child Development, Ministry of Social Justice and Empowerment, education departments, and official scheme portals. Indian country-specific material from organizations such as UNICEF India or WHO India may supplement government evidence where appropriate. Verification sources: - National Portal of India: https://www.india.gov.in/my-government/schemes - Relevant Government of India ministry/department - National Health Authority: https://nha.gov.in/ - Relevant State/UT government sources | |||
| 0% | 0% | 100% | |
How to passInvestigate government-backed entrepreneurship and small-business support available in India, including startup assistance, MSME programs, credit and financing schemes, guarantees, grants or subsidies where applicable, mentoring, incubation, export assistance, innovation incentives, and business-development resources. Compare benefits, limitations, qualification requirements, financing obligations, repayment requirements where applicable, application complexity, and potential long-term business value. Prioritize official Indian sources such as Startup India, Ministry of MSME, SIDBI, relevant financial institutions, and the official ministry or department responsible for each scheme. Use international development research only as supplementary context and not as evidence of current Indian eligibility or benefits. Verification sources: - Startup India: https://www.startupindia.gov.in/ - Ministry of MSME: https://msme.gov.in/ - SIDBI: https://www.sidbi.in/ - National Portal of India: https://www.india.gov.in/my-government/schemes | |||
| 100% | 100% | 100% | |
How to passResearch how Indian government departments and public agencies recommend verifying schemes, benefits, grants, and job opportunities. Compare official Government of India and State/UT portals, ministry websites, recruitment authorities, government publications, official notifications, and verified announcement channels against common misinformation sources, fake websites, fraudulent recruitment portals, and misleading social-media claims. Create an India-specific verification checklist covering: whether the website belongs to the relevant government department or authorized agency; whether the opportunity appears on an official government portal; whether eligibility and application requirements match the official notification; whether the source requests inappropriate fees, personal information, OTPs, or payments; whether the notification has a current publication/update date; whether the opportunity can be cross-checked with the responsible ministry or department; how suspicious websites, messages, or recruitment claims can be reported through Indian government channels. Verification sources: National Portal of India: https://www.india.gov.in/ PIB Fact Check: https://factcheck.pib.gov.in/ Press Information Bureau: https://www.pib.gov.in/ National Career Service: https://www.ncs.gov.in/ National Cyber Crime Reporting Portal: https://www.cybercrime.gov.in/ Do not use U.S. FTC, UK government, or other foreign scam examples as evidence for Indian government opportunities. | |||
| 100% | 100% | 100% | |
How to passAnalyze Indian government opportunities according to whether they provide immediate assistance or may create longer-term financial, educational, employment, health, housing, or business benefits. Compare direct financial support, skill-development programs, government employment, business-support programs, education assistance, healthcare benefits, housing support, and financial-inclusion initiatives. Use specific Indian programs as representative examples and distinguish between: - immediate monetary or in-kind assistance; - reduced household expenses or risk; - improved employability or income potential; - longer-term asset or business-building potential. Do not make universal claims about wealth creation. Base conclusions on the documented design, eligibility, benefits, obligations, and available outcome evidence for the Indian programs examined. | |||
| 0% | 0% | 0% | |
How to passCreate the structured methodology that will be used to evaluate and prioritize Indian government opportunities. Define how opportunities should be assessed against factors such as expected financial or household impact, eligibility likelihood, application complexity, long-term value, family benefit, stability, risk, obligations, required time commitment, and evidence quality. Define a practical scoring or ranking approach that can be applied consistently across different types of Indian opportunities, including schemes, training programs, government jobs, healthcare benefits, housing assistance, entrepreneurship support, and financial-assistance programs. Explain how the framework should handle situations where an opportunity has high potential value but low eligibility likelihood, high effort, significant obligations, substantial uncertainty, limited evidence, or a risk of becoming outdated. Use Indian government evaluation and outcome-monitoring principles where available, including NITI Aayog material, and use the National Portal of India/myScheme fields to structure eligibility, benefit, document, and application comparisons. Do not make final opportunity rankings in this step. Establish the evaluation methodology that will be applied to the shortlisted opportunities later. | |||
| 100% | 0% | 0% | |
How to passResearch common claims about government opportunities in India, such as: - "Government jobs are always secure." - "All government schemes provide meaningful financial benefits." - "Grants are free money with no obligations." - "Training programs automatically lead to employment." - "Public benefits eliminate the need for personal financial planning." Verify each claim using Indian government sources, official scheme guidelines, Indian employment/recruitment information, Indian program evaluations, and credible India-specific evidence. For each claim, classify it as supported, partially supported, context-dependent, misleading, or unsupported. Where evidence varies by scheme, job category, state/UT, beneficiary group, or employment type, explain that variation rather than generalizing across all Indian government programs. Verification sources: - NITI Aayog: https://www.niti.gov.in/ - Ministry of Labour & Employment: https://labour.gov.in/ - National Career Service: https://www.ncs.gov.in/ - National Portal of India: https://www.india.gov.in/my-government/schemes - Official scheme guidelines and Indian government evaluation reports Do not use foreign-country program evidence to establish whether these claims are true for India. | |||
| 0% | 0% | 0% | |
How to passCreate several realistic India-specific opportunity-planning scenarios rather than assuming one universal path. Include: - Government-job-focused strategy - Skill-development-first strategy - Family-support optimization strategy - Entrepreneurship-support strategy - Balanced opportunity strategy Base each scenario on opportunities that are actually available through Indian central-government or State/UT programs. Compare expected benefits, eligibility dependencies, risks, obligations, effort requirements, time horizon, and likely long-term outcomes. Do not assume that a person qualifies for any particular program. State the eligibility conditions or missing personal information that would determine whether each scenario is applicable. | |||
| 100% | 0% | 0% | |
How to passApply the evaluation and scoring framework developed in Step 10 to the strongest current or recurring Indian opportunities identified during the research. Compare government jobs, skill-development programs, education assistance, healthcare benefits, entrepreneurship support, housing programs, financial-assistance initiatives, and other relevant Indian opportunities. Create an impact-versus-eligibility matrix showing which specific Indian opportunities appear to offer the greatest potential long-term benefit relative to effort, qualification requirements, application complexity, risk, obligations, and likelihood of eligibility. Separate opportunities into practical categories such as: - high-priority actions; - opportunities worth monitoring; - conditional opportunities requiring additional eligibility information; - low-priority options. For every shortlisted current opportunity, verify its current status using the responsible Indian government ministry, department, official portal, or relevant State/UT government source. Record the official source and the date on which its current status was verified. Do not recommend applying for an opportunity when eligibility cannot be established from the available information; instead, identify the missing eligibility condition that must be checked. | |||
| 100% | 0% | 0% | |
How to passBring all research together into a complete India-focused government-opportunity decision document. Include: - Government-opportunity evaluation framework - Indian government-scheme category comparison matrix - Government-job versus relevant Indian private-sector career comparison - Skill-development opportunity analysis - Family-benefit assessment - Entrepreneurship-support evaluation - India-specific scam-prevention and verification checklist - Short-term versus long-term value analysis - Impact-versus-effort prioritization matrix - Myth-versus-reality validation section - Multiple India-specific opportunity-planning scenarios - 12-month opportunity-monitoring roadmap - Personal decision checklist - Final recommendation framework Cite all important sources and clearly distinguish verified Indian government guidance, India-specific policy research, independent analysis, assumptions, and practical recommendations. For every current opportunity included in the final shortlist, identify: - the official Indian government source; - the responsible ministry, department, agency, or State/UT authority; - eligibility requirements; - key benefit or opportunity; - important obligations or limitations; - current-status verification date; - publication/update date where available. Do not treat an opportunity as current merely because it appears on a third-party website or social-media post. The final report should not assume that the user or family qualifies for any particular scheme or job. Where personal eligibility information is missing, identify the relevant eligibility conditions and explain what would need to be verified. Clearly distinguish verified facts, official government claims, independent evidence, assumptions, estimates, uncertainties, and recommendations. | |||
Failure modes on this task
19 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | Claude Opus 5 |
|---|---|---|---|
| Delivering the result1 mode | 2 | 7 | 5 |
| Standing up the evidence1 mode | 1 | — | 2 |
| No signal in the judge’s words1 mode · the grader | 1 | 1 | — |