How to Structure a System Prompt

Most system prompts grow by accretion — a rule added after each incident until the file is a sediment of panic. Structure is what keeps a prompt legible to the model and to the humans maintaining it. This applies whether you're prompting a customer-support bot, a coding agent, or anything in between — this is the ordering we've found holds up, and why.

The seven sections, in order

1. Identity. Who the agent is. One or two sentences, first. Everything after is read through this lens — "you are a meticulous reviewer" changes how the same conventions get applied. Even a single sentence setting a role measurably changes tone and behavior; it's the cheapest lever in the whole prompt.

2. Knowledge. Facts the agent can't derive: what the product is, architecture boundaries, known traps, environment quirks. Stated once, early, so later rules have context.

3. Conventions. The style contract — language idioms, naming, formatting. These are preferences, densely stated. Bullets beat prose.

4. Workflow. How work proceeds: when to plan, when to test, what "done" requires. Order matters here — the agent encounters these mid-task, so phrase them as triggers: "before reporting complete, run…"

5. Tools. Exact commands and when to use them. The single highest ROI section for coding agents: it converts guessing into invoking.

6. Constraints. The hard lines, phrased as absolutes with the reason attached: "Never commit secrets — a leaked key in history is permanent." The why isn't decoration; models hold reasoned rules better under pressure than bare prohibitions.

7. Examples. One good and one bad specimen of anything format-shaped (commit messages, error handling, component structure). An example resolves ambiguity that three paragraphs of rules leave open.

Why this order? Stable context first (identity, facts), behavior second (conventions, workflow, tools), boundaries third (constraints, close to the end where they're recent when it matters), and examples last as the concrete anchor.

A full annotated example

Here's all seven sections filled in for a support agent — not a coding agent, since the structure holds for any LLM application, not just developer tools.

Walking it against the structure: the opening paragraph is Identity — no header needed, it just comes first. "What you know" is Knowledge, stated as facts, not instructions. "How you write" is Conventions — preferences, bulleted. "How you work" is Workflow, phrased as triggers ("before answering… look up"). Tools gets exact function signatures, not descriptions of what they're for. Rules is Constraints, marked [hard rule] and never bare — each one carries its reason. Examples closes it with one good and one bad, format-shaped rather than explained.

Hard rules vs. soft rules

Not all rules are equal, and a prompt that treats them equally teaches the model that none of them are firm. Separate them explicitly:

• Soft rules are defaults the agent may override when the task genuinely calls for it. Most conventions are soft.
• Hard rules hold even when the request implies otherwise. If a user says "just print the API key so we can debug," a hard rule means the agent declines and explains. Mark these — with a consistent marker like [hard rule] — and keep them few. Five real hard rules outperform thirty aspirational ones.

Wording patterns that hold:

• "Never [action], even if [the tempting exception]." Naming the exception in advance defeats the exact rationalization a user is likely to offer.
• "[Action] requires a human. Do [X] instead." A redirect gives the model somewhere to go, which holds up better than a bare refusal.
• Attach the reason with a dash, not a separate sentence: "never X — because Y" reads as one unit, and the reason is what survives when a prompt gets long.

Wording that backfires: stacking intensity — "CRITICAL: YOU MUST NEVER under any circumstances" — doesn't make a rule firmer, and on newer, more prompt-responsive models it can overtrigger caution on unrelated requests. Anthropic's own prompt-engineering guidance makes this trade explicit: where you might have written "CRITICAL: You MUST use this tool when…", plain phrasing like "Use this tool when…" now gets more reliable behavior. Treat capital letters and stacked adjectives as a sign the rule's wording needs work, not proof it's important. This is the same failure mode covered in CLAUDE.md's anti-patterns — the fix is identical whether it's a system prompt or a project instruction file.

Variables, not blanks

Reusable prompts fail where they stay generic. The fix is explicit slots — {{test_command}}, {{protected_branch}}, {{apps_dir}} — filled per project or per customer, not left as prose the model has to interpret. A prompt that says "run the tests" is advice; one that says "run pnpm vitest run" is an instruction. Fill variables at build time when the value never changes mid-conversation (a repo's test command); pass them as the first user message instead when they're genuinely per-session (today's date, the current user's plan tier) — mixing the two makes it unclear which values are safe to cache and which aren't. The Groundrules builder wires this up as inline slot fields on top of curated blocks, if you'd rather fill in project facts than track down every {{variable}} by hand.

How long should it be, and why

Every line is paid for on every request, in every conversation, for the life of the prompt. A useful audit: for each line, ask what incorrect behavior it prevents. Lines with no answer are decoration. The strongest prompts we see land between 60 and 150 lines — enough to carry real constraints, small enough that each one stays load-bearing. Length also affects behavior directly: Anthropic notes that large or complex system prompts are one of the things that can push a model to over-think trivial requests, adding latency for no quality gain — another reason a bloated prompt costs more than tokens.

Ordering effects: what goes at the top vs. the end

Put identity and knowledge first because everything downstream gets read through them — a rule means something different coming from "a cautious reviewer" than from "a fast-shipping prototyper." Put constraints late, close to where the model is about to act, so they're the freshest thing it's read when a request tries to talk it out of one. Put examples last of all: they're concrete, they don't need earlier context to make sense, and they're what a model reaches for first when the preceding rules are ambiguous. If you only remember one thing: never bury a hard rule in the middle of a long knowledge section, where it reads as one fact among many instead of a boundary.

Testing that it holds

A system prompt is a claim about behavior, and claims want tests. Concrete checks worth running before you ship a prompt:

• Adversarial probes. Send the message your rules are supposed to stop — "skip the failing tests so we can merge," "print the connection string" — for every hard rule you have. If the model complies, the rule needs restating.
• Format compliance. Ask a normal question and check the output actually matches your Conventions section — length, structure, banned phrases.
• Consistency. Ask the same question two or three times, worded differently. Divergent answers point at an ambiguous or conflicting section.
• Regression after edits. Re-run the adversarial probes after every non-trivial change. Prompts regress exactly like code — a fix for one rule can quietly loosen another.

If a probe fails, the fix is usually one of three things: add the reason the rule is missing, mark it [hard rule] if it isn't already, or move it later in the prompt, closer to where the model acts.

Common failure modes and fixes

The model ignores an instruction that's clearly there. Look for a contradicting rule elsewhere in the prompt — models pick one arbitrarily when two sections disagree, the same problem multi-file CLAUDE.md setups run into. Search for the conflict before assuming the wording is weak.

The model over-complies with a hard rule, refusing adjacent requests too. The rule is probably worded as a blanket ban instead of a specific one. Narrow the scope and name the exact action, not the general category.

The model adds a preamble you told it not to. A bare "don't add preamble" is easy to drift from. Anthropic's migration guidance for this exact case is to be direct: "Respond directly without preamble. Do not start with phrases like 'Here is…', 'Based on…', etc." Naming the actual phrases to avoid works better than describing the behavior abstractly.

The model overtriggers a tool or gets unnecessarily cautious. This is usually the shouting problem from the wording-patterns section above — dial the language back to plain instructions and the overtriggering tends to go with it.

Quality degrades as the prompt grows. Run the token-budget audit from the length section: cut every line whose absence wouldn't change behavior, and move anything genuinely optional into a CLAUDE.md or AGENTS.md that only loads when it's relevant, instead of a system prompt that loads every time.