Beyond Surface Accuracy: New Benchmarks Expose Gaps in LLM Reasoning, Statefulness, and Alignment

This week's research points to a clear shift in LLM evaluation: moving beyond whether models say the right thing to whether their outputs correspond to real-world states, evidence, and intended behavior. ThinkingBox checks agent terminal backend state rather than generated sentences; SciSlopBench measures 'scientific slop' in AI papers; CARAT asks whether materials LLMs reason or recite; and StreamDecisionBench introduces in-force accuracy for streaming decisions. Together they reveal that models can appear correct while failing on substantive criteria—closing a ticket as resolved when the required state is hold, quoting a structural relation without using it, or letting outdated actions remain in force. Complementary work probes why these gaps occur. The contradiction study shows LLMs recover assignments from direct constraints but degrade in scientific prose, where expectations override evidence. The System Prompt Illusion finds that persona/formatting prompts reshape representations while safety prompts barely move, explaining jailbreak vulnerability. Context Confusion shows aligned data can induce narrow misalignment, implying training data alone cannot predict alignment. Collectively, this body of work underscores an industry need for evaluation frameworks that measure actual reliability, state consistency, evidence grounding, and temporal correctness—not just static accuracy.