"Please Remove All Mannered Prose" and Other LLM Incantations

The post opens by likening style prompts to the incantations in The Key of Solomon: precise phrases that, if said exactly right, harness supernatural powers—but a slight mistake can cause disaster. The author describes routinely adding modifiers such as 'Please be concise', 'avoid em dashes and semicolons', and 'restrict inline comments to 8 words or less' to steer LLM outputs. These style prompts work reasonably well but are unpredictable, and superficially similar phrasings can produce noticeably different outputs.

Anthropic recently recommended adding 'Please remove all mannered prose' to avoid the clichéd 'slop' tone, which reportedly works well but struck the author as oddly phrased. This motivates four empirical questions: (1) How specific is a style prompt's effect to its wording? (2) How consistent is the effect across different task types? (3) How does 'remove mannered prose' compare to opposite prompts like 'use mannered prose'? (4) Do these prompts actually achieve the intended effects? The author argues that rigorous understanding of prompt modifiers is important because most prompts are too specific or urgent to justify building dedicated eval sets, and better quantitative documentation from model makers would help.

The proposed investigation draws on the framework of Stolfo et al., 'Improving Instruction-Following through Activation Steering' (ICLR 2025), combining three tools: residual geometry (as in Stolfo et al. and Zou et al.), the logit lens (from nostalgebraist), and output stylometry using standard readability metrics (Flesch 1949 and Guiraud 1954). The excerpt cuts off just as the approach is being introduced, so the results are not yet presented here.