amorce · an outsider's lab on Timestral · J1

The trap bench

Used zero-shot as a normal-behaviour model on a simulated turbine, which of fourteen injected faults does Timestral see, how fast and at what false-alarm cost, against a ridge regression with a CUSUM and two one-line witnesses?

pre-publicverified by a fresh sessionseeds 1, 2, 3, 4, 5held

Pre-public: hypotheses committed before results in a private history, available on request.

Erratum · The second surprise listed as seen « beyond its noise » every fault detected faster than the noise whatever its status, and the page said every size « sees » the frank jump, though the 5M's status there is « noisy ». Both now say seen only for the status seen, and detected where only the delay counts; no number changed. « noisy » (detected within the delay budget, over the false-alarm budget) is a status set after the run: the pre-registered rule defines seen, late and missed only.

What we expected (written before the run)

Each expectation was written with the hypotheses, before any number existed; next to it, the status of the hypotheses it rests on, from the results: 1 of 7 held.

A matrix of fifteen rows (fourteen traps and the témoin) by eight columns (four Timestral sizes, then E1, E2 and the two one-liners); green = seen within 14 days, amber = seen late, grey = missed, red = false alarms; a click on a cell shows the bearing temperature of the trapped turbine, Timestral's 80 % band, the trap start and the first alert. layout, not a bet
The frank +15 °C jump (saut) is green across the four Timestral columns; the slow 0.1 °C-a-day ramp (rampe-lente) is grey for Timestral and green for E2. refuted · H1.3 refuted · H1.4
Under the matrix, one bar per column with the number of faults seen out of eleven and the spread across seeds, and the false alarms per year on the témoin. refuted · H1.5 refuted · H1.6

Figures

The bench · one row per scenario, one column per model or witness

Median over 5 seeds. Faults (physical or sensor): green = seen (within 14 days, at most 4 false alarms a year); amber = late (seen after 14 days) or noisy (detected within 14 days, but over 4 false alarms a year: a status set after the run, the pre-registered rule defines seen, late and missed only); grey = missed (never on a majority of seeds). Non-faults (curtailment, start-stop, early winter) and the healthy fleet: green = quiet, red = more than 4 false alarms a year. Each cell: median delay in days and false alarms per year. Click a cell: seed 1, turbine WT02.

Beyond the noise · on how many of the 5 seeds the fault was seen before the column's first alarm on the healthy fleet · added after the results, not pre-registered

Added after seeing the results, not pre-registered: the status rule counts any alarm after the start of a fault that lasts to the end of the third, so a column that fires every few weeks sees long faults by chance, and this measure subtracts that chance. Faults only. A column that raises an alarm every few weeks “sees” any long fault within a few weeks by chance; this matrix subtracts that chance: a 5 is a detection the healthy fleet never matches, a 0 or 1 is the background.

Per column · faults seen, and seen faster than the column's own noise on the healthy fleet · beyond noise added after the results, not pre-registered

The beyond-noise count was added after seeing the results, not pre-registered (see the matrix above for the reason). “Beyond noise”: status seen, and on at least 3 of the 5 seeds the fault's delay is shorter than the time to the column's first alarm on the healthy fleet after the same date. The healthy fleet's time to first alarm is the median over seeds and turbines; “never” when most seed-turbines stay quiet to the end of the third.

Faults seen out of 11 · within 14 days and under 4 false alarms a year (median over seeds) · and beyond the noise (added after the results, not pre-registered)

The beyond-noise bars were added after seeing the results, not pre-registered: a column that fires every few weeks sees long faults by chance, and these bars remove that chance. Two bars per column: the status count, then the same count once the detections no faster than the column's own noise on the healthy fleet are removed.

False alarms a year on the healthy fleet (nothing injected) · fleet rate, median over seeds

Four turbines, the test third of each seed. E1 judges each hour alone against a 95 % band: by construction it sounds about 5 % of the time.

Dispersion · on how many of the 5 seeds the fault was seen within budget

Faults only. A cell at 5 is a robust detection; at 2 or 3 the median hides a coin toss.

PIT histogram on the healthy fleet · where the truth falls among the 21 quantiles, test third, four turbines, all seeds

Ten bins; a calibrated model is flat at 0.10. Mass at the ends means bands too narrow; a hump in the middle, bands too wide.

The scenarios · what Paliers injects, on which turbine, from which hour

Hypotheses, written before the run

id	claim	measured	status
`H1.1`	On the témoin (nothing injected), the PIT of Timestral-5M on the monitored bearing averages between 0.45 and 0.55 over the test third (median over seeds and turbines), so its bands are centred on turbine temperatures as they were on electricity. Measured 0.434. With the four conditions as known covariates the median sits above the bearing: on seed 1, WT02, truth minus median averages -0.63 °C on the test third and -1.19 °C on the calibration third, for an error spread of 2.3 °C; without covariates the PIT is centred (0.515) but the spread is 11.8 °C. The conditions buy sharpness and cost a bias.	0.434	refuted · metrics.pit_mean_temoin.Timestral-5M ∈ [0.45, 0.55]
`H1.2`	On the témoin, every size raises at most 4 false alarms a year with the daily CUSUM (median over seeds of the fleet rate), so the rule is quiet on a healthy turbine. Measured 5M 24.8, 28M 17.2, 123M 19.5, 209M 9.8 false alarms a year on the healthy fleet. The daily score is not stationary: standardised on the calibration third (late spring and summer), where the mean PIT is 0.32, 0.38, 0.38, 0.38 for the four sizes, it reads the autumn's smaller bias as a shift, and a CUSUM with h = 5 fires every two to four weeks. E2's CUSUM limit, calibrated by replay for one false alarm a year on the same turbines, sits at 16.6 σ (median over seeds and turbines); h = 5 was a bet on a stationary score.	24.75	refuted · metrics.fa_per_year_temoin.max_over_sizes ≤ 4
`H1.3`	Every size sees the frank +15 °C jump (saut) within 2 days (median delay over seeds). Measured 2.6 days for the 209M (median over seeds); the three others detect the jump in 0.6, 0.6, 0.0 days. The 209M has the least excitable daily score of the four (9.8 false alarms a year on the healthy fleet) and its CUSUM needs a third day to cross h; the jump is detected by every size, one of them a day late on the bet.	2.58	refuted · metrics.delay_or_never_days.saut.max_over_sizes ≤ 2
`H1.4`	No size sees the slow 0.1 °C-a-day ramp (rampe-lente) within 14 days, because the rolling context absorbs a drift the model then forecasts as normal (median delay over seeds above 14 days, or never, for all four sizes). Measured 5M 9.6 d, 28M 11.6 d, 123M 6.6 d, 209M 8.6 d, all within the 14-day budget. The rolling context does not absorb a 0.1 °C-a-day drift: the truth keeps rising above a forecast that extrapolates the level, the daily score climbs and the CUSUM crosses h. The detection beats each column's own noise on the healthy fleet (first alarm 5M 14.1 d, 28M 18.6 d, 123M 9.6 d, 209M 19.1 d after the same date) on 4, 3, 3, 5 of 5 seeds: a detection, a slim one where it is 3 of 5. The bet was wrong for this ramp.	6.58	refuted · metrics.delay_or_never_days.rampe-lente.min_over_sizes > 14
`H1.5`	Size buys little on this bench - the 209M sees at most 2 more of the 11 faults than the 5M. Measured 5: the 209M has 5 faults seen to the 5M's 0. The gap is a false-alarm artefact: the 5M fires 24.8 times a year on the healthy fleet, so its 30 pre-trap days hold two false alarms on most seeds and every detection turns “noisy”; the 209M fires 9.8 times a year and its detections pass the budget. Beyond the noise the counts are 5M 0, 28M 5, 123M 4, 209M 5.	5	refuted · metrics.traps_seen_gain_209M_vs_5M ≤ 2
`H1.6`	Paliers' E2 (a ridge on the conditions with an EWMA and a CUSUM on the residual) sees at least as many of the 11 faults as the best Timestral size. Measured -3: E2 sees 8 faults, the 123M 11. But 7 of the 123M's detections are no faster than its own noise on the healthy fleet (19.5 false alarms a year, first alarm 9.6 days after the trap date), while E2's are all beyond noise (0.0 false alarms a year). Beyond the noise: E2 8, 123M 4.	-3	refuted · metrics.traps_seen_E2_minus_best_timestral ≥ 0
`H1.7`	The squared-difference one-liner sees the frozen sensor (capteur-fige) within 3 days (median over seeds), since a flat line has no hourly difference at all.	0.5833	held · metrics.delay_or_never_days.capteur-fige.sqdiff ≤ 3

What the page asserts (checked at build time)

claim	value	check
Fifteen scenarios: eleven faults (physical or sensor), three non-faults and the healthy fleet, each on every seed and every column.	11	metrics.n_fault_traps = 11
No adjacent quantile pair crosses in any forecast of any size (the head is monotone by construction).	0	metrics.crossing_rate_max = 0.0
Every Timestral size raises its first true alert on the frank +15 °C jump (frank jump) within 3 days (median delay over seeds), whatever its false-alarm rate.	2.58	metrics.delay_or_never_days.saut.max_over_sizes ≤ 3
E2 (Paliers' ridge with EWMA and CUSUM, limits calibrated by replay) raises under one false alarm a year on the healthy fleet.	0	metrics.fa_per_year_temoin.E2 < 1
E1 (the same ridge, each hour judged alone against a 95 % band) raises more than a hundred false alarms a year on the healthy fleet.	319	metrics.fa_per_year_temoin.E1 > 100
The two one-liners see the frozen sensor within a day (median over seeds).	0.5833	metrics.delay_or_never_days.capteur-fige.one_liners_max ≤ 1

What surprised us

A fault that lasts to the end of the third turns any later alarm into a “true positive”, so a column that fires every few weeks “sees” long faults within the budget by chance. The healthy fleet gives each column its own noise: its first alarm comes 5M 14.1 d, 28M 18.6 d, 123M 9.6 d, 209M 19.1 d, E1 0.9 d, E2 never, rolling variance 32.1 d, squared difference 42.6 d after the trap date (median over seeds and turbines). Faults seen within budget and faster than that noise on a majority of seeds: 5M 0, 28M 5, 123M 4, 209M 5, E1 0, E2 8, rolling variance 1, squared difference 1 of 11.
What Timestral sees beyond its noise (status seen, and faster than its noise on a majority of seeds), size by size: 5M: nothing; 28M: frozen sensor, load-dependent, fleet-wide jump, slow ramp, frank jump; 123M: load-dependent, fleet-wide jump, slow ramp, frank jump; 209M: frozen sensor, load-dependent, fleet-wide jump, slow ramp, frank jump. E2: intermittent, frozen sensor, load-dependent, fleet-wide jump, small offset, slow ramp, recalibration, frank jump. The frank jump and the fleet-wide jump are detected by every size in 5M 0.6/4.6 d, 28M 0.6/3.6 d, 123M 0.0/2.6 d, 209M 2.6/3.6 d (frank jump/fleet-wide jump, median delay of the first true alert); the 5M with too many false alarms for either to count as seen (“noisy”).
Mean PIT on the healthy turbines: 5M 0.434, 28M 0.471, 123M 0.487, 209M 0.471 (0.5 is centred). On the side check (5M, one seed, one turbine) the conditions as known covariates cut the error spread from 11.8 to 2.3 °C and move the PIT from 0.515 to 0.433: the model reads wind and power, forecasts five times sharper, and lands about a degree too hot.
The three non-faults (undeclared curtailment, undeclared stops, early winter) leave Timestral at roughly the healthy fleet's rate (the trapped turbine alone against the fleet): start-stop 5M 27, 28M 15, 123M 18, 209M 12; curtailment 5M 30, 28M 18, 123M 21, 209M 12; early winter 5M 33, 28M 18, 123M 24, 209M 12 false alarms a year, against 5M 25, 28M 17, 123M 20, 209M 10 on the healthy fleet. E2 stays at 0.0, 0.0, 0.0 on them.
The one-liners: the frozen sensor is seen in 0.6 d (log variance) and 0.6 d (squared difference), faster than any model; everything else is their noise. E1, each hour judged alone, is right about every fault in under three days and useless: 319 false alarms a year on the healthy fleet.
The size trend on this bench is not monotone: faults seen beyond noise 5M 0, 28M 5, 123M 4, 209M 5; false alarms on the healthy fleet 5M 24.8, 28M 17.2, 123M 19.5, 209M 9.8 a year. The 123M tops the status count; one seed in five decides most cells (the dispersion matrix).

Reproduce

make reproduce J=1
# $(PY) tools/export_paliers.py --series --jalon J1
# $(PY) -m amorce.jalons.j1 --out results/J1.json

The series come from Paliers, the author's simulator of a small wind fleet, which is private for now: `make reproduce J=1` needs it (and Node) to regenerate them; everything after the series is in this repository.