Build a reusable Job Opportunity Verification & Offer-Qua…Jobs and Career
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
100%
Claude Opus 5
93%
GPT-6 sol
93%
GPT-5.6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
138m
Claude Opus 5
31m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
118
Claude Opus 5
57
GPT-5.6 sol
52
GPT-6 sol
31
Claude Opus 5.5
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (15) | ||||
| 100% | 100% | 100% | 100% | |
How to passFirst, establish exactly what the tracker is designed to evaluate and what it is not designed to determine. Define an “opportunity” broadly enough to cover permanent employment, contract roles, freelance engagements, internships, project-based work, and other legitimate work arrangements, while keeping the evaluation framework consistent. Separate opportunity quality from personal preference: a role can be legitimate but unsuitable, while another can look attractive but contain insufficient evidence. Define the core decision states, such as Pursue, Hold for Verification, or Reject, and explain what evidence should move an opportunity between them. Establish boundaries around legal, tax, immigration, or employment-law questions: record authoritative evidence and flag matters requiring professional or jurisdiction-specific review rather than pretending the tracker can resolve them universally. Design the tracker so it can be reused for opportunities encountered months later without depending on a particular hiring platform, employer, industry, salary level, or current labor-market condition. | ||||
| 100% | 100% | 100% | 100% | |
How to passResearch and define a source hierarchy specifically for employment verification. Prioritize official employer career pages, official company documentation, government employment resources, recognized professional bodies, regulatory or corporate registries where relevant, and reputable publications with identifiable editorial standards. Treat recruitment platforms, aggregators, social-media posts, employee-review sites, forums, and anonymous comments as secondary evidence rather than automatic proof. Explain which claims require primary-source confirmation, such as employer identity, job description, application channel, compensation terms, and stated employment arrangement. Build tracker fields for source URL, source type, publication/update date when available, exact claim supported, verification status, and date checked. Include a rule for conflicting sources: do not silently choose the more convenient claim. Record the contradiction and identify which source has stronger authority. Also include an accessibility rule so that a source that requires an inaccessible login, broken page, or unverifiable screenshot does not automatically count as confirmed evidence. | ||||
| 100% | 100% | 100% | 100% | |
How to passDesign the first major spreadsheet section as an Opportunity Intake Register. Each row should represent one distinct opportunity and contain a stable opportunity ID, employer name, role title, employment type, recruiting channel, application URL, date discovered, and current evaluation status. Add fields for employer identity verification, official careers-page match, recruiter identity where applicable, and whether the role can be independently located through an authoritative source. Include a field distinguishing an official employer posting from a third-party reproduction of a posting. The register should prevent duplicate evaluation when the same vacancy appears on several platforms. Add fields for last verification date and evidence freshness, but avoid arbitrary expiration rules unless supported by research. Make the structure flexible enough to compare opportunities from different industries without assuming that all employers publish identical information. The purpose is to create a reliable identity layer before deeper analysis begins, so later compensation, responsibility, and risk findings cannot accidentally be attached to the wrong employer or vacancy. | ||||
| 100% | 100% | 100% | 100% | |
How to passDevelop a structured verification process for determining whether the claimed employer and recruiting pathway are genuine. Research practical signals that can be checked against authoritative sources, including the employer's official website, official careers pages, identifiable company domains, recognised corporate or professional registers where appropriate, established employer contact channels, and independently verifiable corporate information. Compare the job listing with the employer's official vacancy information where available and check whether the role title, location, recruiter, application method, and contact details are consistent. Record unexplained discrepancies rather than assuming they are harmless. Treat requests for money, unusual payment methods, sensitive personal information before an appropriate hiring stage, unverifiable recruiters, suspicious application domains, or instructions that deliberately bypass normal employer channels as warning indicators requiring additional verification. Do not label an opportunity fraudulent solely because one unusual feature exists; record the evidence and uncertainty and apply the decision rules consistently. | ||||
| 100% | 100% | 100% | 100% | |
How to passTurn each job description into a structured set of claims rather than treating the posting as one block of information. Create fields for responsibilities, required qualifications, preferred qualifications, expected outputs, tools or technologies, seniority, work arrangement, schedule expectations, reporting relationship, travel requirements, and stated performance expectations. For each important claim, record whether it is explicitly stated, independently supported, unclear, internally inconsistent, or inferred. Research how vague employment language can affect interpretation and distinguish normal flexibility from material ambiguity. For example, “support multiple projects” is not equivalent to a clearly defined workload, while “up to 40 hours per week” is different from guaranteed full-time hours. Do not rewrite employer claims into stronger statements than the source supports. Build a claim-to-source structure so a stranger can trace an important tracker entry back to its evidence. This section should make later comparison possible without relying on memory or subjective impressions from reading several job descriptions. | ||||
| 100% | 100% | 100% | 100% | |
How to passCreate a compensation section that records the structure of compensation, not merely a headline number. Include salary or rate when explicitly published, pay frequency where available, hourly versus fixed compensation, guaranteed versus conditional components, bonuses, commissions, equity or other non-cash components, stated benefits, probation-related conditions, overtime language, and any material payment contingencies. Distinguish verified figures from ranges, estimates, or claims copied by third parties. Where compensation depends on productivity, accepted work, billable hours, commissions, milestones, or other conditions, record the mechanism rather than reducing it to a single number. Research authoritative guidance on interpreting compensation terminology and contractual language. Include a “Missing Information” field so the absence of a published figure is represented honestly instead of converted into a guessed value. The tracker should support apples-to-apples comparison while explicitly warning when two compensation structures are not directly comparable. Avoid using temporary promotions, unusually reported individual earnings, or current salary-ranking websites as the foundation of the system. | ||||
| 100% | 100% | 100% | 100% | |
How to passCreate a structured section for contractual and practical employment conditions. Capture employment classification, contract duration, renewal language, expected availability, working-hour structure, scheduling flexibility, notice provisions, exclusivity or conflict-of-interest requirements, intellectual-property language where publicly documented, equipment expectations, reimbursement rules, and any stated termination or withdrawal conditions. Research authoritative explanations of employment terminology and identify which fields cannot be generalized across jurisdictions. The tracker should distinguish explicit contractual evidence, job-posting language, recruiter statements, and assumptions made by the evaluator. Add a “Requires Document Review” status for terms that can only be verified after receiving an actual agreement. Do not instruct the evaluator to accept legal conclusions from generic websites. Instead, make the spreadsheet useful for identifying which questions need professional or employer clarification before commitment. Include a comparison mechanism showing where two opportunities differ materially in obligations even if their advertised compensation looks similar. | ||||
| 100% | 100% | 100% | 100% | |
How to passResearch how to evaluate the hiring process itself as a separate dimension of opportunity quality. Create fields covering application requirements, interview stages, assessments, expected preparation, unpaid work requests, portfolio requirements, background checks, identity verification, communication channels, and stated timelines when available. Distinguish legitimate role-related assessments from requests that appear disproportionate, irrelevant, or poorly bounded. The tracker should not declare a process fraudulent simply because it contains an assessment; instead, require evidence-based indicators and record uncertainty. Add a field for whether the process is described consistently across official sources and communications. Include a “Candidate Cost” section capturing substantial time, travel, unpaid assignment effort, or document preparation where these are explicitly required. The goal is not to discourage demanding recruitment processes but to help compare their proportionality and transparency. Create a simple risk classification supported by documented criteria and ensure that the classification can be revised if stronger evidence appears later. | ||||
| 100% | 100% | 0% | 100% | |
How to passBuild a dedicated Cross-Source Consistency Check into the spreadsheet. Compare the official job posting, employer careers page, recruiter communication, application portal, and other secondary evidence when available. Check whether role title, employment type, responsibilities, compensation structure, work arrangement, application route, and employer identity remain consistent. Create separate fields for “Claim A,” “Claim B,” “Conflict Type,” “Higher-Authority Source,” and “Resolution Status.” Research a sensible approach to resolving conflicting employment information without assuming that the newest-looking source is automatically correct. If a contradiction cannot be resolved, preserve it rather than forcing a definitive answer. Add a severity level based on whether the conflict affects a minor description detail or a material decision factor such as pay, contract type, work location, or required obligation. This audit should prevent the tracker from producing a falsely precise recommendation when the underlying evidence is contradictory. Make the spreadsheet capable of showing unresolved contradictions prominently enough that they cannot be overlooked during final comparison. | ||||
| 100% | 100% | 100% | 100% | |
How to passDevelop a comparison framework that combines evidence quality, role fit, compensation clarity, employment-term clarity, recruiting-pathway confidence, process burden, and unresolved risk. Research whether any established decision-making principles can justify the chosen dimensions, but do not pretend that a universal numerical score exists if the evidence does not support one. If you use weighted scoring, document every category, weight, scoring definition, and threshold in the spreadsheet. Include a “No Score / Insufficient Evidence” option so missing information does not automatically receive a neutral score that hides uncertainty. Separate objective evidence fields from subjective preference fields. For example, “official compensation published” is an evidence characteristic, while “I prefer this salary structure” is a personal preference. Allow the user to rank opportunities while still seeing why an opportunity received its position. Include a final recommendation column with a short evidence-based rationale and make the model easy to audit rather than turning it into an opaque formula. | ||||
| 100% | 100% | 100% | 100% | |
How to passCreate explicit decision gates for Pursue, Hold, Reject, and Recheck. Define what evidence is required for each state and identify circumstances that should trigger another verification pass. Examples may include a changed job description, altered compensation language, new recruiter contact, contract receipt, employer-domain change, conflicting information discovered later, or a previously unavailable authoritative document becoming accessible. Research reasonable principles for maintaining information quality over time without creating unnecessary repeated checking. Include “Last Verified,” “Next Review Trigger,” and “Evidence Changed?” fields. The tracker should not rely solely on calendar-based review if a material event is a better trigger. At the same time, provide a practical periodic review mechanism for opportunities that remain active for several weeks. This makes the system genuinely long-horizon: a user can revisit an opportunity later and immediately see what was verified, what remains unresolved, and what event would justify reopening the decision. | ||||
| 100% | 0% | 100% | 100% | |
How to passCreate 8 clearly labeled fictional stress-test scenarios and enter them into the tracker to prove that the system works under difficult conditions. Make the scenarios substantially different from one another. Include situations such as: a legitimate employer with conflicting third-party listings; a contract role with an attractive headline rate but unclear payment conditions; a recruiter using an authentic employer name through an unverifiable channel; two official pages containing different job responsibilities; a role with strong compensation evidence but unusually burdensome assessment requirements; an opportunity with excellent role fit but insufficient contract information; an employer whose official posting is temporarily unavailable while secondary evidence remains; and two opportunities that appear similar but have materially different employment obligations. For every fictional scenario, show the evidence status, contradictions, decision gate, comparison outcome, and reason for the final recommendation. Clearly label all scenarios as fictional examples, not real employers or vacancies. Use them as functional tests rather than storytelling exercises. | ||||
| 100% | 100% | 100% | 100% | |
How to passTry to break the system deliberately. Test whether it can be manipulated by an attractive salary figure, a prestigious employer name, a polished job advertisement, many secondary sources repeating the same unsupported claim, an apparently urgent deadline, or a recruiter making strong verbal promises without documentary evidence. Check whether missing information is accidentally treated as negative evidence, positive evidence, or neutral evidence without justification. Test whether duplicate listings distort comparison, whether subjective preferences overwhelm factual verification, and whether a high numerical score can conceal a serious unresolved contradiction. Audit formulas, categories, decision thresholds, source fields, and status labels for internal consistency. Confirm that the tracker never presents an inference as a verified fact. Record each weakness discovered and modify the structure accordingly. The final spreadsheet should remain useful even when evidence is incomplete, conflicting, duplicated, emotionally persuasive, or unusually attractive. The adversarial audit should result in concrete corrections rather than merely a statement that the tracker was reviewed. | ||||
| 100% | 100% | 100% | 100% | |
How to passGive the completed spreadsheet to a hypothetical stranger who has no knowledge of the research process and test whether they can use it correctly from the instructions alone. The stranger should be able to identify where to enter a new opportunity, verify the employer, record sources, distinguish primary from secondary evidence, document contradictions, evaluate compensation structure, compare opportunities, interpret the decision gates, and understand when re-verification is required. Check whether any column uses unexplained terminology, hidden assumptions, ambiguous scoring, or instructions that depend on knowledge from this task. Add a short How to Use This Tracker section at the beginning explaining the workflow from intake to final decision. Ensure that the eight fictional scenarios are clearly separated from live opportunities so they cannot be mistaken for real data. Revise any field that requires follow-up clarification. The stranger test is satisfied only when someone unfamiliar with the project can complete the workflow without asking follow-up questions. | ||||
| 100% | 100% | 100% | 100% | |
How to passAssemble the final spreadsheet into a clean, reusable system with clearly separated sections or tabs for Instructions, Opportunity Register, Evidence & Claims, Compensation, Employment Terms, Process Risk, Contradiction Audit, Comparison & Scoring, Re-Verification, and Stress Tests. Include consistent status labels, source fields, evidence notes, decision gates, and formulas where appropriate. Add sample fictional records without allowing them to interfere with future real entries. Check that every required field has a clear purpose and that no formula creates a misleading conclusion when information is missing. Verify that the spreadsheet supports comparison across different job types without assuming identical employment structures. Perform a final check for spelling, broken formulas, contradictory labels, duplicated fields, unsupported scoring assumptions, and output-format compliance. Confirm that the system is useful weeks or months later, not dependent on a single current vacancy. Final Quality Check: a stranger must be able to open the spreadsheet, understand the workflow, evaluate a new opportunity, trace important claims to evidence, compare alternatives, and reach a defensible Pursue / Hold / Reject decision without asking follow-up questions. | ||||
Failure modes on this task
2 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Standing up the evidence1 mode | — | 1 | 1 | — |