I’m considering launching or partnering with a streamingArts & Entertainment
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
67%
Claude Opus 5
50%
GPT-6 sol
0%
GPT-5.6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
12m
Claude Opus 5
4m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
50
Claude Opus 5.5
48
GPT-6 sol
33
Claude Opus 5
14
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (6) | ||||
| 100% | 0% | 0% | 100% | |
How to passStart by identifying the major services competing for viewers in South Korea. The goal is to understand the actual market structure before comparing individual platforms. Begin with official company and platform sources, then use industry research to identify significant services that may not appear prominently in international searches. Investigate categories including: Global subscription streaming platforms Korean streaming platforms Broadcaster-owned streaming services Internet Protocol Television (IPTV) and online TV services Advertising-supported streaming or FAST services, where available Live TV and catch-up services Start with platforms and organizations such as: https://www.netflix.com/kr-en https://www.wavve.com https://www.coupangplay.com https://www.disneyplus.com/ko-kr https://watcha.com https://www.kcc.go.kr https://www.kca.kr Search for additional relevant platforms through Korean media-industry reporting, company announcements, and official broadcaster websites. Create a market map containing approximately 10–15 significant services or distribution ecosystems. For each, record: Service Operator/Owner Service Type Main Content Free/Paid Model Korean Market Focus Deduplicate services that are simply different access points to the same underlying product. Do not automatically treat every global streaming service as a major local competitor. Look for evidence that the service is actually active and relevant in South Korea. Output Produce a South Korean streaming and online TV market map containing 10–15 significant platforms or service ecosystems. | ||||
| 100% | 100% | 0% | 0% | |
How to passUsing the market map, investigate who operates each platform and how the major players obtain or control content. For every significant platform: Open its official company or service information. Identify the operating company and parent organization. Determine whether the service is: Independent Broadcaster-owned Telecommunications-owned E-commerce or technology-company-owned International Investigate major broadcaster, studio, sports, telecommunications, or production partnerships. Identify evidence of exclusive content, original productions, or distribution agreements. Use: Official company announcements Investor relations pages Regulatory filings where applicable Official broadcaster and production-company announcements For major Korean broadcasters, inspect their corporate and digital distribution pages to understand relationships between television networks and streaming services. Create an ownership and content-control table: Platform Operating Company Parent/Key Owner Major Partners Original/Exclusive Content Strategy Verification Source Decision constraint Do not assume that a platform owns content simply because it streams it. Separate: Owned intellectual property Original productions Licensed exclusives Non-exclusive licensed content Live or catch-up broadcaster content Output Produce a verified ownership and content-strategy map showing the relationships between platforms, broadcasters, telecommunications companies, and content producers. | ||||
| 100% | 100% | 0% | 0% | |
How to passSelect approximately 8–10 of the most significant services and investigate how users actually access and pay for them. For each platform: Open the pricing or membership page. Record available subscription tiers. Identify differences in: Advertising Video quality Number of simultaneous streams Downloads Device compatibility Bundled services Check whether free trials, free access, advertising-supported tiers, or bundled memberships are available. Investigate partnerships with: Mobile carriers Broadband providers Device manufacturers E-commerce memberships Record whether live television, sports, or premium events require separate access. Use official platform pages as the primary source. Create a subscription comparison table: Platform Access Model Subscription Tiers Advertising Bundling Live Content Key Restriction Treat subscription prices and promotional offers as current observations and record the date or period displayed when possible. Decision constraint Do not compare services only by monthly price. Identify differences between: Standalone subscriptions Bundled access Ecosystem-based access Premium-event access Advertising-supported access Output Produce a verified streaming business-model comparison for the major platforms. | ||||
| 100% | 100% | 0% | 100% | |
How to passNow investigate how Korean audiences are actually using streaming and online TV. Use authoritative and reputable sources such as: https://www.kocca.kr/kocca/main.do https://www.kcc.go.kr https://www.kca.kr https://datareportal.com Company investor reports Official or established media-industry research Research indicators such as: Streaming adoption Online video consumption Mobile versus television viewing Popular content categories Domestic versus international content demand Sports and live-content consumption Subscription fatigue or multi-service usage Advertising-supported viewing Where market-share or usage figures are obtained from third-party research, record: The source Methodology if available Reporting period Whether the figure measures subscribers, monthly active users, viewing time, app usage, or another metric Do not combine different metrics as though they represent the same measurement. Create an audience and positioning table: Platform Available Audience Metric Measurement Type Core Audience Position Content Strength Source/Period Output Produce an audience-demand and competitive-positioning analysis that distinguishes verified metrics from broader market observations. | ||||
| 100% | 0% | 0% | 100% | |
How to passInvestigate what a company would need to understand before operating or distributing a streaming or online TV service in South Korea. Begin with official sources from: https://www.kcc.go.kr https://www.msit.go.kr https://www.kca.kr https://www.copyright.or.kr Research the current rules or official guidance relevant to: Online video and broadcasting services Content classification or ratings Copyright and licensing Personal-data and user-information handling Advertising disclosures Distribution through app stores and connected-TV ecosystems Telecommunications or broadcasting classifications where relevant Do not provide legal conclusions. Instead, identify: The relevant regulatory authorities. The official regulations or guidance that appear relevant. Areas where a new entrant would likely need legal or regulatory confirmation. Then investigate the distribution environment: Major mobile operating systems Smart TV ecosystems Local telecommunications companies Broadband/IPTV relationships Device partnerships App-store availability Create a market-entry constraint table: Area Relevant Authority/Platform Verified Requirement or Issue Potential Business Impact Requires Specialist Confirmation? Output Produce a regulatory and distribution readiness checklist for a hypothetical streaming-market entrant. | ||||
| 100% | 0% | 0% | 100% | |
How to passCombine the research into a structured market intelligence report. The final deliverable must contain: 1. Executive market overview Summarize: The overall structure of the streaming and online TV market Major competitive groups The relationship between global platforms and Korean companies Important consumer and distribution trends 2. Competitive landscape Include the 10–15 identified platforms, showing: Service Operator Market category Core audience Primary content advantage Business model 3. Ownership and content ecosystem map Explain the relationships between: Streaming platforms Korean broadcasters Telecommunications companies Technology and e-commerce companies Production companies and content suppliers Clearly distinguish verified ownership from partnership or licensing relationships. 4. Subscription and monetization comparison Compare the strongest 8–10 services based on: Pricing structure Advertising Bundling Device access Live content Premium or exclusive content 5. Audience and demand analysis Identify: Strongest available evidence about viewing behavior Important audience segments Major content preferences Differences between live TV, streaming, and mobile video Data limitations or conflicting measurements 6. Market-entry opportunity and challenge assessment Based on the research, identify possible: Content gaps Distribution opportunities Partnership opportunities Competitive barriers Regulatory issues requiring further investigation Do not claim that an opportunity is guaranteed. Clearly separate: Verified market facts Evidence-based inferences Questions requiring further validation 7. Source and verification appendix Organize sources into: Government and regulatory sources Official platform and company sources Broadcaster and telecommunications sources Industry research Secondary media reporting For every major conclusion, retain the underlying source so the research can be revisited and updated. | ||||
Failure modes on this task
11 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | — | 3 | 4 | 1 |
| Standing up the evidence1 mode | — | — | 2 | — |
| No signal in the judge’s words1 mode · the grader | — | — | — | 1 |