I want to test a small UK freelanceBusiness and Consumer Services
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
100%
Claude Opus 5
60%
GPT-6 sol
0%
GPT-5.6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
22m
Claude Opus 5
8m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
105
Claude Opus 5
67
Claude Opus 5.5
31
GPT-6 sol
7
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (5) | ||||
| 100% | 100% | 0% | 100% | |
How to passRecord the two revenue scenarios, the 20% operating-cost assumption, the eight-hour weekly limit, no employees and UK business clients. Use the rules, thresholds, fees and provider terms current when the task runs, with access dates. Treat GOV.UK, HMRC, Companies House and ICO as controlling sources, distinguish obligations from optional good practice, and flag questions needing a qualified accountant, solicitor or insurance adviser. | ||||
| 100% | 100% | 0% | 100% | |
How to passUse the current GOV.UK sole-trader and private-limited-company journeys to compare registration, public disclosure, liability, records, annual filings, payment of tax, VAT trigger points, extraction of money and closure duties for both revenue scenarios. Use the ICO fee self-assessment and controller-versus-processor guidance to map likely data-protection roles and registration questions for each of the three example engagements. Do not infer a personalised tax outcome from missing income or expense details. | ||||
| 100% | 0% | 0% | 100% | |
How to passFor each example engagement, create a pre-contract checklist covering scope, deliverables, acceptance, change control, intellectual property, confidentiality, security, data roles, subprocessors, retention and deletion, incidents, support, liability, termination and permitted AI use. Use ICO controller-processor contract guidance plus GOV.UK invoicing and late-payment rules to identify mandatory or decision-dependent clauses. Produce a checklist and escalation points rather than presenting a generated contract as legal advice. | ||||
| 100% | 100% | 0% | 100% | |
How to passCompare manual invoicing with bank transfer, Stripe and PayPal using current public pricing and service information. Explain the main fees, payment-processing considerations, invoicing workflow and practical pros and cons for a small UK freelancer. Also shortlist three UK IT-freelancer insurers using publicly available information, comparing relevant cover, exclusions, indicative public pricing where available, and policy features. Do not request quotes or provide personal insurance advice. | ||||
| 100% | 0% | 0% | 100% | |
How to passBring the research together into a practical launch plan covering the two revenue scenarios, sole trader versus limited company, the three engagement types, contract and data-handling checklist, payment options, insurance shortlist and recommended starting structure. Finish with a practical 90-day launch sequence showing the main actions, priorities and decision points. Include links to the official sources and clearly note any commercial-source substitutions. | ||||
Failure modes on this task
7 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | — | 1 | 2 | — |
| Reaching the page1 mode | — | — | 2 | — |
| No signal in the judge’s words1 mode · the grader | — | 1 | 1 | — |