Chart data could not load; reload to try again.
Artificial Intelligence
Task-horizon progress
Record-setting task horizons and three trajectories, including modeled estimates.
Methodology
Assumptions
Task horizon is the human expert duration of a software task that an agent is predicted to complete with 80% success on METR's TH1.1 suite.[^2] It is not AI runtime or a claim about all kinds of work. The chart combines direct measurements with modeled estimates; converted TH1.0 values and ECI- and AECI-based values are estimates on the TH1.1 scale.
For ECI-based estimates, fit log TH1.1 p80 hours to ECI using 20 matched model versions from the saved tracker and Epoch data.[^1][^3] The source matches and exact inputs are in calibration.csv.
ln(p80_hours) = -22.20441287 + 0.14432844 × ECI
n = 20; R² = 0.970768; residual SD = 0.3128 log hoursThe ECI calibrators span 121.87–157.82. Estimates outside that range extrapolate. Epoch includes METR-derived benchmark evidence, so the correlation is not an independent validation of METR. Differences in scaffold, benchmark, and token budget add uncertainty. METR says horizons above 16 hours are unreliable with its current task suite.[^2]
Coverage
The task-horizon input contains all 724 scored Epoch source rows, represented by 259 distinct model/date/score entries, plus 23 METR TH1.1 p80 measurements, 31 TH1.0 p80 measurements, and 19 Anthropic AECI entries.[^1][^2][^3][^6] The chart selects one value for each matched model and date in this order: published TH1.1 p80, converted TH1.0 p80, ECI-based estimate, then AECI-based estimate. When the same source has more than one score for a model and date, the highest horizon is selected. These decisions leave 270 selected model/date entries from 332 inputs; all source entries remain in all-horizons.csv.
Most matches use the same model name and date. Explicit source-backed aliases reconcile GPT-4o's May 2024 version, Claude 3 Opus, o1, and GPT-4 Turbo Preview's January 2024 version when source labels or dates differ. For Claude 3 Opus and o1, the selected measurement date is used. The tracker dates GPT-3.5 Turbo Instruct to March 15, 2022; the chart instead uses September 18, 2023, the version release date in Epoch's model table.[^1][^3] Three entries in METR's TH1.1 file are labeled TH1.0. They are converted as TH1.0 rather than counted as independent TH1.1 measurements.[^2][^6] Other distinct dated versions are retained. The raw CSV retains all inputs and estimates.
The chart starts on January 1, 2023. By default, a dot is shown only when its horizon exceeds every model available on an earlier date, including before 2023. If models released on the same date tie for the new maximum, each is included. This gives 21 frontier dots on 21 dates. Show all models displays 268 selected entries: all dated entries from 2023 onward, plus four with no release date in the source: Claude Instant 1.1, Gemma 2 9B, GPT-5 Chat, and Gemma 2 27B. They appear in an Undated column only when all models are shown; they are excluded from the frontier and trend fits. No date is invented. All dots use the same visual style; the source of each value remains in the model data and this methodology.
The METR report also names an Unreleased 2022 model, but the saved AECI extraction provides no score and dated marker for it. It is not assigned an invented estimate.[^4]
Converting TH1.0 to TH1.1
Match the 16 named models measured on both TH1.0 and TH1.1. The explicit ID mapping is in script.py; the source measurements are in metr-results-1-0.yaml and metr-results.yaml.[^6][^2] Fit ordinary least squares with the natural log of TH1.0 p80 minutes as the input and the natural log of TH1.1 p80 hours as the output. The three TH1.0 results carried into the TH1.1 file are excluded. With H₀ in minutes and H₁ in hours:
ln(H₁) = -5.11084335 + 1.29457173 × ln(H₀)
n = 16; R² = 0.974065; residual SD = 0.272182 log hours
GPT-4 0125: H₀ = 1.913986 minutes → H₁ = 0.8386 minutesThe conversion is an estimate on the TH1.1 scale, not a TH1.1 measurement. METR's suites and agent scaffolds differ, and uncertainty in both versions is omitted. Three early models use converted TH1.0 values rather than treating a copied TH1.0 value as TH1.1. The fit's TH1.0 inputs span 1.37–40.04 minutes; conversions outside that range are extrapolations. The ECI score for GPT-4 0125 is shared by Epoch's broader model group and is not a separate measurement for that version.[^1]
Holdout comparison
For each model in a comparison, remove that model from every method's calibration, refit, then predict its measured TH1.1 p80. Score errors in log hours and report exp(RMSE): 1 is perfect and lower is better. This is model holdout, not a historical forecast. Each method first trains on all its available matched models except the held-out model; the models being scored are identical within each row. holdout_comparison.py reproduces the results.
Models with both TH1.0 and ECI: n = 15 TH1.0 1.347 ECI 1.376
Models with TH1.0, ECI, and AECI: n = 6 TH1.0 1.377 ECI 1.419 AECI 1.263When training is restricted to the same shared models too, TH1.0 and ECI score 1.351 and 1.354 on the 15-model comparison. TH1.0 wins 7 of those model holdouts and ECI wins 8. The TH1.0 advantage is small and not conclusive; on the six Anthropic models AECI scores best. The source order is a display policy, not a demonstrated accuracy ranking for every model. The held-out outcomes are excluded from fitting, but the published ECI and AECI inputs were built using full upstream benchmark snapshots, so they are not independent historical forecasts. Shared task suites, scaffolds, digitization, uncertain version matching, small samples, and benchmark saturation limit the comparison. In particular, the six-model AECI comparison does not validate extrapolation to internal frontier models.
Anthropic's index: assumptions and fit
AECI scores come from the same October fit shown in METR's November 5 assessment, using the tracker's extracted values and reported anchors.[^3][^4] The raw figure is included as aeci-main.png with a checked hash. Most scores are digitized to about 0.1 AECI; Opus 5.5 and internal Model 4 have reported anchors. Public dates are release dates. Internal dates are approximate dates read from the figure for availability to most Anthropic employees, not public releases or precise training completion dates.
Fit the log of exact METR p80 hours to AECI for all eight entries with an exact METR model ID, including the early Mythos Preview snapshot. The public Mythos Preview snapshot is a distinct entry. AECI is fitted directly to METR rather than treated as an Epoch score: an additional AECI-to-ECI bridge would add another model assumption, and the tracker reference uses the direct fit.[^3]
ln(p80_hours) = -22.68266456 + 0.1470880061 × AECI
n = 8; R² = 0.989430; residual SD = 0.221025 log hours
Model 2: AECI 174.04 → 18.4771 hours
Model 3: AECI 176.98 → 28.4734 hours
Model 4: AECI 178.60 → 36.1347 hoursThe AECI fit is reproduced from the saved scores and full-precision METR values, then applied to all 19 AECI entries before model selection. These are modeled horizons, not METR task measurements. The eight calibrators cover AECI 124.71–162.35; Models 2–4 are beyond that range. Their results rely on continued linearity in log time. The chart shows central estimates; the fit scatter is not a complete uncertainty range. Digitization, index error, vendor benchmark selection, saturation, small sample size, and METR's measurement limits add uncertainty.
Frontier trend lines
The single exponential and two-breakpoint lines are fitted to the same 21 frontier dates, from February 24, 2023 through September 25, 2026. Each date has equal weight. With H as task horizon in hours and t as days since January 1, 2025 divided by 365.2425, ordinary least squares minimizes squared errors in ln(H). The single exponential is:
ln(H) = -1.79770097 + 2.63350452 × tFor the two-breakpoint line, choose two dates that minimize the same squared error, with at least three frontier dates before, between, and after the breakpoints. Search a 14-day grid, then search daily within 14 days of the best grid pair. This deterministic search is an approximation, not a guarantee of the global daily optimum. The fitted breakpoints are April 10, 2024 and February 16, 2026:
ln(H) = -2.64045585 + 1.77457981 × t
+ 0.81277178 × max(0, t + 0.72828326)
+ 1.75218850 × max(0, t - 1.12527978)The lines are continuous at both breakpoints and become dashed after the last frontier date. The AI 2027 line is the separate scenario described below. The fitted lines extend to April 27, 2027; this extension is a projection, not a measured result. The curves depend on the chosen snapshot, model matching, priority rule, and record selection. The late frontier includes estimates above the range where METR says its task suite can measure reliably.[^2] The two-breakpoint line has more flexibility and is fitted to the same data used to choose its breakpoints, so its close fit to those points is not evidence that it will predict future results better. No confidence band is calculated here.
AI 2027 comparison
The dashed red line is a scenario, not a measured series or a new forecast. It uses the reconstruction in the saved tracker source, checked there against the authors' grading sheet.[^3][^5] Let u be the UTC decimal year minus 2025. Evaluate:
ln(seconds) = 6.36588017 + 1.75547434 u + 0.350855496 u²
+ 0.0462100721 u³ + 0.0144760767 u⁴
+ 0.0194798378 u⁵ + 0.00743193880 u⁶
+ 0.000851343348 u⁷
raw_hours = exp(ln(seconds)) / 3600
TH1.1_hours = raw_hours × rebase
rebase = Claude 3.7 Sonnet's measured TH1.1 p80 hours
/ raw_hours on April 3, 2025
≈ 0.783This anchors the older TH1.0 scenario scale to the best model at the scenario's publication date. A constant rebase is an approximation: the two task suites do not have a uniform ratio across models. The polynomial after the authors' graded window is a mathematical continuation. A model's position relative to the curve is therefore a comparison of estimates with a scenario, not evidence that its real-world capabilities are ahead of a forecast. The vertical dashed line marks the November 9 data snapshot.
Reproducing the complete model chart
Run python3 script.py in the extracted package. It emits the complete chart JSON, regenerates the priced data.csv and all-horizons.csv, selects the plotted points and frontier, fits the calibration and frontier trend lines, and reconstructs the scenario. Run python3 holdout_comparison.py to reproduce the holdout results. all-horizons.csv contains all 332 source entries with their measurement/estimate series and source; the JSON contains the 270 selected entries and 21 frontier points from 2023 onward. No unavailable price is filled and no model without a published score is given one. The source figures and frozen tracker files preserve the upstream extraction choices, but do not recreate the underlying private Anthropic benchmark experiments.