I am spending a long stretch in ThailandLaw and Government
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
83%
Claude Opus 5
75%
GPT-5.6 sol
33%
GPT-6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
43m
GPT-5.6 sol
12m
Claude Opus 5
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
58
Claude Opus 5
43
GPT-5.6 sol
28
Claude Opus 5.5
25
GPT-6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (12) | ||||
| 100% | 100% | 100% | 100% | |
How to passOpen https://www.rd.go.th/english/6045.html and record the definition that separates a resident from a non-resident, including the day count and the period it is measured over. Record exactly what each of the two is liable to tax on, paying attention to the extra condition attached to foreign-source income for residents. | ||||
| 100% | 100% | 100% | 100% | |
How to passFor someone present 195 days in a calendar year who earns a salary from a foreign employer and moves part of it into Thailand, state what is taxable under the wording from step 1. Then state the answer for the same person present 150 days. Name the single word in the rule that decides the foreign-income part and explain why the two cases differ. | ||||
| 100% | 0% | 100% | 100% | |
How to passFrom the same page, record how many categories assessable income is divided into, and state which category covers employment and which covers liberal professions. Then record the deduction allowed against employment income including its cap, and record at least five allowances with their amounts, including the child allowance and its limit on the number of children. | ||||
| 100% | 0% | 0% | 0% | |
How to passRecord the progressive rate bands in full, then record the additional rule that applies when income in categories two to eight exceeds a stated threshold, including the percentage used and which of the two results the taxpayer pays. Work a case: taxable income of 800,000 baht arising from category eight income of 900,000 baht, showing both computations and which one governs. | ||||
| 100% | 0% | 0% | 100% | |
How to passRecord the note printed immediately under the rate table naming the tax years it was to be implemented for, and record the last-updated date shown at the foot of the page. State plainly that the allowances and bands may have changed since, and name what you would check before relying on them. Do not substitute figures from any other source. | ||||
| 100% | 0% | 100% | 100% | |
How to passFrom the same page, record the annual filing and payment deadline relative to the taxable year, and record which categories of income trigger an additional half-yearly return, together with that deadline. State what happens to tax already withheld or paid at the half year. | ||||
| 100% | 0% | 100% | 0% | |
How to passFrom the same page, record the withholding tax rate for each listed category of income, including the two different rates for a public entertainer depending on residence. Explain in one sentence how withholding interacts with the annual return. | ||||
| 100% | 100% | 100% | 100% | |
How to passOpen https://www.rd.go.th/english/23518.html and record what the certificate is, who issues it, how many days before departure the application must be filed, and the three situations that require one. Record the definition the page gives for public performer. | ||||
| 100% | 100% | 100% | 100% | |
How to passThe same page first states an obligation on a departing foreigner regardless of whether tax is payable, then describes both a category of stay that is exempt and a 1991 notification. Quote both parts, record the day threshold and the condition attached to the exempt category, and state which reading governs for an ordinary long-stay visitor with no Thai-source income. | ||||
| 100% | 0% | 100% | 100% | |
How to passFrom the same page, record the difference between the two forms, including how long each is valid, whether each covers one departure or several, and which cannot be renewed. Record the penalty for departing without a certificate when one is required, including the surcharge rate, the maximum fine and the maximum custodial term. | ||||
| 100% | 0% | 100% | 100% | |
How to passOpen https://www.rd.go.th/english/23517.html and record any taxpayer obligations, deadlines, forms, or compliance requirements described on the page that are relevant to filing, paying, or settling personal income-tax obligations. Exclude appeal procedures, enforcement powers, installment-payment rights, and penalty-discretion provisions that are not needed to determine a remote worker's filing and departure obligations. | ||||
| 100% | 0% | 0% | 100% | |
How to passProduce a tracker with one row per obligation, and columns for the trigger that switches it on, the deadline, the form or document named, the authority, and the page it came from. Mark every row whose figures carry the out-of-date warning from step 5. Close by naming the single obligation most likely to be missed by someone who only reads the income tax page, and why. | ||||
Failure modes on this task
13 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | — | 3 | 2 | — |
| Standing up the evidence1 mode | — | 1 | 1 | 2 |
| No signal in the judge’s words1 mode · the grader | — | 4 | — | — |