Research Paper Debunks Single-Metric Faithfulness in LLM Chain-of-Thought
Analysis of 10,276 reasoning traces across 12 major open-weight models reveals that classifier choice causes faithfulness scores to swing dramatically, with differences of up to 21.3 absolute percentage points. This finding directly contradicts the prevailing practice of reporting single-number metrics for model faithfulness, indicating the property is not an objective, stable attribute but a measurement-dependent construct.











