amorce · an outsider's lab on Timestral · J0

First forecast

Out of the box, are Timestral's bands honest on an ordinary hourly electricity series, and how fast is each size on a laptop CPU?

pre-publicverified by a fresh sessionseeds 0held

Pre-public: hypotheses committed before results in a private history, available on request.

What we expected (written before the run)

Each expectation was written with the hypotheses, before any number existed; next to it, the status of the hypotheses it rests on, from the results: 5 of 5 held.

The grey fan widens with the horizon and the black truth stays inside the darker band about four times out of five. held · H0.1
Moving the origin slider shifts the whole fan; the coverage table under it is one number per quantile level, close to the nominal level. held · H0.1 held · H0.2
Four latency bars, one per size; the 5M bar is the shortest and all four are below one second. held · H0.4

Figures

The series · one year of hourly national consumption, shown as daily means

RTE éCO2mix · national electricity consumption, France, hourly, 8,758 hourly steps from 2025-09-01 00:00 UTC to 2026-08-31 21:00 UTC (3 missing). The fans below use the hourly values; the origins live in the highlighted last third.

Timestral-5M · the quantile fan at each origin

One week of context (grey), then 96 forecast steps: 21 quantiles from 1 % to 99 %, darker toward the median, and the truth in black. The slider moves the origin; hours are relative to it. Coverage of the 80 % band over the 12 origins: 85.2 %.

Timestral-209M · the quantile fan at each origin

Coverage per level · share of truths under each quantile level

For a calibrated model each number is close to its level: 10 % of the truths under the 10 % quantile, and so on. Pooled over the origins and the horizon.

Calibration · the same numbers as a curve

Each point is one quantile level; a calibrated model sits on the nominal diagonal.

The four sizes on this series

Coverage of the 80 % band, per origin and size

Each cell is one origin × one size, over the 96 forecast steps; nominal 0.80.

Latency · one 96-step forecast from 1,024 steps, CPU, median of 5

arm64 laptop CPU, Darwin, torch on CPU with the forecaster's defaults (two passes: the context and its mirror). A measurement of this machine, not compared by `make reproduce`.

Hypotheses, written before the run

id	claim	measured	status
`H0.1`	The 80 % band (quantiles 0.10 to 0.90) of Timestral-5M covers the truth between 72 % and 88 % of the time over the 12 origins × 96 steps.	0.8516	held · metrics.coverage_80 ∈ [0.72, 0.88]
`H0.2`	The 98 % band (quantiles 0.01 to 0.99) of Timestral-5M covers at least 93 % of the truths.	0.9991	held · metrics.coverage_98 ≥ 0.93
`H0.3`	Fewer than 2 % of adjacent quantile pairs cross before sorting, for every size.	0	held · metrics.crossing_rate_max < 0.02
`H0.4`	Timestral-5M produces a 96-step forecast from a 1,024-step context in under 100 ms on the laptop CPU (median of 5); the 209M size is under 1 s.	13.49	held · metrics.latency_ms.Timestral-5M < 100
`H0.5`	Coverage of the 80 % band does not improve with size by more than 5 points between 5M and 209M on this series (size buys sharpness, not calibration).	-0.04688	held · metrics.coverage_80_gain_209M_vs_5M ≤ 0.05

What the page asserts (checked at build time)

claim	value	check
Every size returns a 96-step forecast from a 1,024-step context in under one second on this laptop CPU (median of the timed runs).	137	metrics.latency_ms.max < 1000
No adjacent quantile pair crosses, for any size: the head builds the 21 quantiles as median ± cumulative positive increments.	0	metrics.crossing_rate_max = 0.0
The 98 % band of every size covers at least 93 % of the truths.	0.987	metrics.coverage_98_min ≥ 0.93
The 80 % band of the 209M is narrower on average than the 5M's on this series.	0.9285	metrics.band80_width_ratio_209M_vs_5M < 1.0

What surprised us

Quantile crossing cannot happen: Timestral's head builds the 21 quantiles as the median plus and minus a cumulative sum of positive increments (monotone_quantiles in timestral/model.py), so H0.3 is held by construction, not by chance. The crossing rate is still measured on the output, and is exactly zero.
The weights' LICENSE (Modified MIT, with a revenue condition) is not fetched by the package, which downloads only config.json and model.safetensors; it was downloaded separately into the Hugging Face cache and read.
The portal serves one row per 15 minutes but consumption only at :00 and :30, and its timestamps stumble on the daylight-saving changes: one hour has no row, one hour is served twice with two different values. With one empty half-hour, that makes 3 NaN hours out of 8,758; nothing is imputed, and the 12 origins all fall in a clean stretch.
Coverage of the 80 % band swings from one origin to the next: from 22 % to 100 % for the Timestral-5M over its 12 windows of 96 hours. The pooled 85.2 % hides that spread; twelve origins is a small sample.
The size trend is not monotone on this series: CRPS 5M 871, 28M 903, 123M 815, 209M 820 MW; the 123M is the most accurate of the four here and the 28M the least. One series and twelve origins cannot rank the sizes; the bench of J1 and the baselines of J2 will.
The Timestral-5M's lower tail is too wide: 0.0 % of the truths fall under its 1 % quantile and 0.0 % under its 5 % quantile, while 91.5 % fall under its 95 % quantile (nominal 1, 5 and 95 %). Mean absolute calibration error over the 21 levels: 5M 2.4 pts, 28M 2.9 pts, 123M 4.8 pts, 209M 4.9 pts; averaged over the levels, the 5M is the closest to nominal on this series, tails apart.

Reproduce

make reproduce J=0
# $(PY) -m amorce.jalons.j0 --out results/J0.json