Publications

Large language models in safety-critical settings are evaluated almost exclusively by outcome accuracy. This paper argues that correctness is a necessary but radically insufficient measure: a correct answer may arise from genuine structural knowledge or from a brittle internal process that happened, by chance, to land on the right token. The two are indistinguishable at Level A1. This paper introduces the instruments to tell them apart.

Three instruments — ETC, ASCOL, and FEST — decompose aggregate accuracy into six epistemic regimes with distinct geometric signatures. The most important discovery underlying them is that the probability dynamics observed through the logit lens are not amplification events but rotational realignments in high-dimensional representational space. The proof is direct: norm ratios move by 0.002 while answer probabilities shift by 56 percentage points. Nothing was amplified. The analytical camera moved. This finding reframes what the field has been measuring with the logit lens for years.

A carrier–content decomposition follows from this: every transformer’s output is the product of a position-dependent carrier signal (derivable from frozen weights, zero data required) and a content signal representing actual knowledge. Accuracy differentials exceeding 20 percentage points between answer positions arise from this carrier — not from what the model knows. Inference-time correction exploiting the decomposition raises Llama-3-8B accuracy from 67.9% to 71.1% on 1,089 medical licensing questions without any modification to model parameters.

FEST analysis of 274 failures establishes that 260 (94.9%) are architecturally recoverable. Only 14 samples (1.29%) represent genuine knowledge gaps. The model’s true knowledge ceiling is 98.71%. The gap between the 67.9% baseline and that ceiling is not a measure of what the model does not know. It is a measure of how severely the inference architecture prevents the model’s knowledge from reaching the output.

The framework is validated across nine models (7B–14B parameters) from six suppliers, producing 3,246,256 attention probes and 6,411,576 layer probes across 283,289 inferences — production instrumentation, not a research prototype.