Audience: anyone shipping prompts to production. Engineers, prompt
engineers, PMs writing copy that ends up in a model call,
non-technical operators tuning the GPT-N variant powering their
internal tools. The principles are model- and platform-agnostic; the
end-of-chapter notes show how PromptLab makes each one cheaper to
follow.

What this is: an opinionated playbook. There are 1,000 ways to
write a prompt; only a handful repeatedly hold up under load. This
guide is what's left after the rest get filed under "yeah, but"
caveats by anyone who's run a prompt for six months in production.

What this is NOT: a survey of prompting techniques (chain-of-
thought, ReAct, tree-of-thought, MoA, etc. all show up, but the goal
is when to reach for them, not how each works). For that, read the
papers, then come back.

---

1. What "optimised" actually means
2. Defining success before you write a word
3. The anatomy of a great prompt
4. Writing for the model in front of you
5. Few-shot, zero-shot, chain-of-thought — picking the right shape
6. Output format engineering
7. Reducing variance and hallucination
8. Cost optimisation
9. Latency optimisation
10. Iteration discipline
11. Cross-provider prompts
12. Anti-patterns to avoid
13. Production hardening
14. Continuous improvement loops
15. Pre-flight checklist before you ship

---

You can't optimise something you can't measure, and the word
"optimised" gets used to mean five different things at once. Pin one
down before you start.

The five axes every prompt tradeoffs across:

1. Quality — does the output do the thing it's supposed to do?
2. Cost — dollars per call, dollars per million calls.
3. Latency — wall-clock time from request to first usable output.
4. Consistency — does the same input produce the same useful
   output 100 times in a row?
5. Robustness — what happens when the input is slightly off, in a
   different language, adversarial, or empty?

You cannot maximise all five simultaneously. A prompt tuned for
Claude Sonnet 4.5 with detailed XML structure and few-shot examples
will beat the same task posed as a one-liner — but it costs more
per call and adds latency. A prompt tuned for cost (terse, no
examples, gpt-4o-mini) is cheaper but more variable.

The most common failure isn't picking the wrong tradeoff. It's not
realising you're making a tradeoff at all.

Decide your priority axis before you write the prompt. If
quality wins, you'll eat the cost of examples and a stronger model.
If cost wins, you'll cap output length and use a smaller model and
accept some variance. If latency wins, you'll stream and lean on
Groq or a small model. The rest of the chapters assume you've made
that choice.

How PromptLab helps with this

• Prompt Studio side-by-side cohort — run the same prompt
  against 3-5 models in parallel and see all five axes at once on
  every result card: output text (quality), cost ($0.0042), token
  usage, latency (1,840 ms). The tradeoffs aren't theoretical;
  they're a row of numbers you can compare.
• Eval rules capture the quality axis as a hard pass/fail so
  you can talk about quality in numbers ("eval pass rate 87% on
  Claude vs 71% on GPT-4o-mini") instead of vibes.
• Cost Analytics dashboard keeps the cost axis visible over
  time — when a prompt's cost-per-call drifts upward (because the
  model started returning longer outputs, or you bumped the
  context), it's a chart, not a surprise on the next bill.

---

A prompt is a function. Functions are easier to write when you know
what they return.

Write the success criteria first. Three forms, in order of
strength:

1. A scorable rubric — "For each generated bullet, the bullet
   must (a) be one sentence, (b) start with a strong verb, (c) name
   a specific number or proper noun." You can score this
   automatically.

2. A list of must-include and must-not-include strings —
   "Output MUST contain the article publication date in
   YYYY-MM-DD format. Output MUST NOT contain the phrase 'as an AI
   language model'."

3. Hand-graded examples — "Here are 5 inputs paired with the
   ideal outputs. New outputs should resemble these in length,
   tone, and structure."

Skip this step and you'll iterate by feel for weeks. You'll mistake
"the output looks good when I read it three times" for "the output
is good." Then a customer will report a regression and you'll have
no way to tell whether the prompt is worse or your taste has
drifted.

Concrete tip: keep a golden_set of 10-30 representative
inputs in version control next to the prompt, with the expected
behaviours noted. Run the prompt against the set whenever you make
a non-trivial change. The set doesn't need to be fancy — a JSONL
file works.

How PromptLab helps with this

• Eval rules are the first-class home for this. Every prompt
  attaches a list of evaluators (Contains / Regex / Length cap /
  Latency cap / Cost cap / JSON schema / LLM judge). Each
  evaluator runs on every interaction and surfaces pass/fail
  badges on the result card. The "scorable rubric" form maps
  directly to the LLM judge rule type — write the rubric in
  plain English, the Lab Assistant scores against it.
• Scenarios snapshot a prompt + cohort + fixed inputs + eval
  rules into a reproducible test setup. Every run against the
  scenario lands in an execution group with its eval pass rate
  recorded — that's your golden_set made executable.
• The pre-flight pass-rate trend in scenario history is the
  shortest-feedback-loop version of "did my latest edit break
  anything?": one chart, prompt-version on the x-axis, eval pass
  rate on the y-axis.

---

Good prompts almost always have the same six parts, in roughly the
same order. Skip parts at your own risk.

1. Role / persona             — who the model is supposed to be
2. Task                       — the single sentence describing the work
3. Context / inputs           — the data the task operates on
4. Output format              — what shape the response must take
5. Constraints                — what the model must / must not do
6. Examples                   — 1-3 ideal input/output pairs (optional but powerful)

Role / persona. "You are a senior copy editor at a technology
publication. Your job is to…" — sets the prior over what
"reasonable" output looks like. Effect is real but small. Don't
spend more than two sentences here.

Task. "Summarise the article in exactly 5 bullets, each
highlighting a distinct fact." — the verb plus the specific
deliverable. Should be one sentence. If you need two sentences, your
task is two tasks; split the prompt.

Context / inputs. "Here is the article: {{articleText}}" — the
data plug. Use named placeholders ({{varName}}) so the calling
code is explicit about what's substituting. Wrap large inputs in
delimiters (<article>...</article> or triple-backticks) so the
model knows where the user data ends and your instructions resume.

Output format. "Return strict JSON matching the schema below.
No prose before or after." — be ruthlessly explicit. "JSON" alone
is a suggestion. "Strict JSON, no prose, no markdown fences, no
explanation" is a directive. If the format is JSON, paste a
1-line schema example.

Constraints. "Each bullet must be one sentence under 15 words.
Do not include marketing language. If the article doesn't contain
enough material for 5 bullets, return fewer." — the negative
space. Anything you've ever wished a model didn't do, write here.

Examples. "Input: . Output: <ideal 5
bullets>." — 1-3 pairs. Examples are the single most effective
intervention you can make on prompt quality, and they're the most
under-used. They're also the costliest in tokens, so they live
behind the "do I need this?" question.

Anti-pattern: the kitchen sink

Tempting to dump everything: 12 paragraphs of role, 8 examples, 30
constraints, 4 layers of nested instructions. The model gets
confused, output quality falls, you pay 4x in tokens. Trim
mercilessly. The shortest prompt that hits the rubric is the right
prompt.

How PromptLab helps with this

• "✨ Compose with AI" drafts the six parts for you from a
  plain-English task description. Fill the form (task, output
  format, constraints, optional examples), pick a target model,
  and the Lab Assistant returns a prompt with role/task/context/
  format/constraints/examples already organised. You then edit by
  hand instead of starting from a blank editor.
• Per-model variants in the Compose result mean the same task
  is structured differently for Claude (XML tags), GPT-4o
  (structured chat + JSON schema), and Gemini (markdown sections).
  You don't have to know each model's preferences; the assistant
  encodes them.
• Diff view in the Optimize sheet highlights when a structural
  change (e.g. moving constraints before examples) actually helps
  vs. when it's neutral. Visual diff + projected eval delta makes
  the costliness of structural changes legible.

---

Models have personalities. The same prompt produces different
outputs not because one model is better, but because each model has
been RLHF'd into preferring certain inputs.

Rough prior over the big four

• Claude (Anthropic) — loves XML-style tags
  (<task>, <context>, <rules>). Strong at long context, tool
  use, faithful instruction following. Tends to be verbose unless
  you cap it. Best general-purpose default in 2026.
• GPT-4o / 5 (OpenAI) — prefers structured chat (system
  message + user message) and response_format: { type: "json_object" } or json_schema for guaranteed JSON.
  Function/tool calling is excellent. Tends to over-hedge ("I'm not
  able to…" for benign requests) — push back with explicit
  "you can answer this directly" instructions.
• Gemini (Google) — markdown-formatted prompts work well;
  numbered lists and ## section headings are interpreted
  faithfully. Strong vision, very long context windows. Sometimes
  confidently wrong on edge facts; ground heavily.
• Llama / Qwen / Mistral on Groq — open-weights, no RLHF
  consistency across providers. Tool use varies (Llama 4 yes,
  Mixtral no). Sub-second latency is the differentiator.

Concrete differences that matter

Choice	Claude likes	GPT-4o likes	Gemini likes
Section delimiter	<section> tags	## Section markdown	## Section markdown
Output forcing	"Respond ONLY with…"	response_format API param	"Respond ONLY with…"
Example boundary	<example> tags	Input: ... Output: ...	numbered list
Long context	works fluidly	needs structure	works fluidly
Refusal style	rare; clear	frequent; hedged	sometimes silent

These priors don't make a prompt right or wrong. They reduce the
distance between "the prompt I wrote" and "the prompt the model
was trained to expect." Smaller distance → better output for free.

When to break model conventions

If your prompt is going across multiple models in a comparison
batch (Migration Wizard, A/B test, fallback chain), do NOT lean
into one model's syntax. A Claude-leaning XML prompt against
GPT-4o produces mediocre GPT output, and you'll wrongly conclude
GPT is worse at the task. For multi-model use, write
provider-agnostic markdown prompts and accept slightly lower
peak quality on each model in exchange for fair comparison.

How PromptLab helps with this

• "Translate for this model" banner — when you add a model to
  a cohort whose provider differs from how the prompt was authored
  (e.g. Claude XML prompt + you add GPT-4o), a non-blocking banner
  offers to translate the prompt into the new provider's
  conventions. One click → diff view → apply. No hand-rewriting.
• "Optimize for cohort" mode in the Optimize sheet — when
  you're running a fair comparison across providers, this returns
  ONE provider-agnostic prompt that all cohort models can interpret
  reasonably, plus per-model interpretation notes warning you about
  each model's likely take.
• Compose with AI per-model variants (covered in Chapter 3)
  bake in the model-specific syntax preferences so you're not
  guessing.

---

Three families of prompting technique. Each costs more tokens and
latency than the previous. Pick the cheapest one that hits your
rubric.

Zero-shot

Just the instructions. "Summarise this article in 5 bullets." No
examples, no reasoning steps. Cheapest, fastest. Default to this.
Most "I need few-shot examples" intuitions are wrong — modern
frontier models are excellent at zero-shot for well-defined tasks.

Few-shot (1-5 examples)

Adds 1-5 input/output example pairs. Use when:

• The task has a style (tone, structure, format) that's hard to
  describe in instructions but easy to show.
• The output is structured and the schema has subtleties (e.g.
  "extract entities as { canonical_name, aliases[] } where
  canonical_name is the most-formal version, but if there's no
  formal version use the most-frequent variant").
• The task is unusual enough that the model lacks a strong prior
  (a domain-specific classification, a niche output format).

Don't use few-shot when:

• The instructions can describe the rule clearly.
• The task is common (summarisation, translation, code generation
  for popular languages).
• Token budget is tight — examples are expensive.

Chain-of-thought

Asks the model to reason step-by-step before producing the answer.
"Think through the problem step by step. Then output the answer
inside <answer> tags."

Use when:

• The task requires multi-step reasoning (math, logic, complex
  filtering, multi-hop questions).
• You can tolerate the latency hit (CoT roughly doubles the
  output, doubling the latency and cost).
• You'll parse the final answer out (don't dump CoT directly to
  end users).

Don't use CoT for:

• Pure recall tasks ("what's the capital of France?" — the model
  knows; reasoning steps add cost without quality).
• Highly latency-sensitive UX (CoT is fundamentally slower).
• Reasoning models (grok-4, deepseek-reasoner, o4) — they
  reason internally without explicit CoT prompting. Asking
  explicitly sometimes makes them worse.

Hybrid: structured chain-of-thought

Ask for reasoning IN a structured field, parse it out:

{
  "reasoning": "step 1... step 2... step 3...",
  "answer": "the final answer"
}

You get CoT quality benefits + clean parsing. Best of both worlds
when JSON output is acceptable.

How PromptLab helps with this

• Eval-rule pass-rate scorecard is how you justify the
  cost-quality tradeoff. Run zero-shot against the golden set; if
  pass-rate is ≥90%, ship. If 60-90%, try few-shot. Below 60%, try
  chain-of-thought or a stronger model.
• The Compose with AI form has a dedicated example pairs
  field — give the assistant 1-3 hand-crafted examples and it
  weaves them into the generated prompt with the proper delimiters
  for the target model.
• Cost-per-call comparison in the result card makes the
  cost-of-CoT explicit. You're choosing between "$0.001 / call,
  78% pass rate" and "$0.004 / call, 91% pass rate" — both
  numbers are right there.

---

The model's output format is half the prompt's job. Get it right
and parsing becomes trivial; get it wrong and you'll write a
500-line regex that breaks every Tuesday.

Three output styles, in order of reliability

1. Strict JSON via provider-enforced schema (best when
   available)

   • OpenAI: response_format: { type: 'json_schema', json_schema: { schema, strict: true } }
   • Mistral: same as OpenAI
   • Anthropic: tool-use with a forced tool_choice is the
     equivalent
   • Google: response_mime_type: 'application/json' +
     response_schema

   The provider validates the response server-side. If validation
   fails, the API errors. You never receive malformed JSON.

2. JSON-by-instruction (next best)

   • "Respond ONLY with valid JSON matching this schema. No prose
     before or after. No markdown fences."
   • Works on every provider, including those without strict mode.
   • Failure mode: occasional markdown fence (```json ... ```)
     wrapping. Add a trim step.

3. Free-form text with markers (when JSON isn't right)

   • "Output the answer between <answer> and </answer> tags."
   • Use for human-readable output, prose, lists.
   • Pair with a regex/parser that extracts the marked region.

When NOT to force JSON

JSON is not always the right shape. If your output is naturally
prose (a summary, a translation, a creative draft), forcing it into
{ "summary": "..." } is just overhead. Use free-form text. Save
JSON for genuinely structured data.

Schema design tips

• Tighter schemas produce fewer errors. { tags: string[] }
  is fine; { tags: string[] } with a description "lowercase
  hyphenated kebab-case" reduces post-processing.
• Avoid nested optional fields. Flatten. Models miss optional
  fields more often than you'd hope.
• Enums beat strings for discrete categories.
  { sentiment: 'positive' | 'negative' | 'neutral' } is more
  reliable than { sentiment: string }.
• Don't ask for fields the model can't compute. "Confidence
  score 0-100" sounds useful; in practice the model returns 85
  basically every time. Self-reported confidence is mostly noise.

How PromptLab helps with this

• The outputDefinition field on every prompt has a
  first-class slot for outputJsonTemplate. Set it once; the
  router enforces strict-schema mode where the underlying provider
  supports it. You get the strongest available guarantee
  automatically.
• The JSON schema eval rule validates every output against
  your schema and surfaces a hard pass/fail badge. If schema
  conformance drifts (the model started omitting a field), you see
  it on the result card immediately, not when a downstream
  consumer breaks.
• The Lab Assistant Compose form has an output schema field
  — paste the example shape, the assistant builds the strict
  instructions + provider-specific format directives into the
  generated prompt for you.

---

Two related problems. Variance is "the same prompt produces
different outputs across runs." Hallucination is "the output
contains a confident-sounding statement that isn't true."

Reducing variance

• Drop temperature. 0.0 is deterministic-ish (not fully
  deterministic across providers, but close). For tasks where
  there's a single correct answer (extraction, classification,
  formatting), temperature 0.0-0.2 is the right zone.
• Use a fixed seed when the provider supports it (OpenAI,
  Mistral via random_seed, DeepSeek). Same seed + same
  temperature + same prompt = same output ≈99% of the time.
• Constrain output strictly. Strict JSON schema, length cap,
  enum-typed fields — every constraint is a degree of freedom the
  model can't use to produce different outputs.

Reducing hallucination

Hallucinations come from the model's training data filling in gaps
when the prompt is under-specified. Three cures:

1. Ground the prompt. Don't ask "what's the population of
   Helsinki?" — paste the Wikipedia paragraph and ask the model to
   read it. Retrieval-augmented prompting (RAG) is hallucination
   prevention dressed up.

2. Tell the model it's allowed to refuse. "If the answer
   isn't in the provided context, respond Not in source." This
   is the single most effective hallucination prevention technique.
   Models will lean confidently into nonsense if not given an
   honourable exit.

3. Ask for citations. "For each claim, cite the source URL."
   Forces the model to ground each statement. Sonar / Tavily /
   citations-aware prompts are a strong default for any
   information-retrieval-style task.

Anti-pattern: high temperature for "creativity"

Temperature 1.0+ for creative writing is folk wisdom; in 2026 it's
mostly wrong. Modern models don't need temperature to be creative
— they need a creative instruction ("write 3 distinct angles, each
with a different tone"). High temperature buys you variance, not
creativity. Variance + creativity = output you can't predict OR
reuse.

How PromptLab helps with this

• Per-prompt model parameters (temperature, topP, seed)
  are first-class fields on the spec. Set once, applied on every
  call.
• Replay in the Interactions explorer re-executes a past
  interaction with the same inputs against the current active
  version. Hard regression test for "did v4 reduce variance?":
  replay 100 v3 interactions against v4 and compare the eval pass
  rates.
• Perplexity / Tavily / Cohere with documents are
  first-class providers — the entire grounding-via-search story is
  reachable through one canonical request shape. The router
  surfaces metadata.citations on every grounded call so the
  Interactions explorer renders them inline.
• Contains/Not contains eval rules catch hallucination
  patterns. Configure "Output must NOT contain 'as an AI language
  model'" or "Output must contain the article publication date"
  and the rule fails loudly the first time it slips.

---

The unit cost of an LLM call is the product of:

total_cost = input_tokens   × input_rate
           + output_tokens  × output_rate
           + cached_tokens  × cached_rate
           + per_request_fee

Each term has its own optimisation lever.

Input tokens — usually the big spend

The input is the prompt template + variables. Wins, in order of
impact:

1. Drop unused context. If you're stuffing the entire user
   profile into every call but only the email is used, drop the
   profile. (You'd be amazed.)

2. Move unchanging context to the cached prefix. Anthropic
   prompt caching, OpenAI prompt caching, DeepSeek cache-hit
   pricing all charge cached input at 10-25% of fresh input. A
   30k-token instruction prefix that costs $0.09/call drops to
   $0.01/call once cached.

3. Summarise long history. For multi-turn conversations,
   summarise turns >5 ago into a 2-sentence summary; keep only the
   last 5 turns verbatim. Halves input tokens with minimal quality
   loss.

4. Use a smaller model where the task allows. GPT-4o-mini at
   $0.15/$0.60 vs Claude Sonnet at $3/$15 — 20x ratio. If
   GPT-4o-mini hits your eval pass-rate, ship it.

Output tokens — sneaky and chatty

• Cap max_tokens. Hard ceiling on cost-per-call. Set it to
  ~1.5x what you actually need; revise based on real distribution.
• Tell the model to be terse. "Output exactly 5 bullets, each
  one sentence." Models default to verbose; instructions matter.
• Avoid CoT in production unless eval gain justifies it. CoT
  doubles or triples output tokens.

Per-request fees

Some providers (Tavily, Scrappey, OpenAI Batch processing
overhead) charge per-request flat fees on top of token cost. For
high-volume flows, batching multiple logical calls into one model
call (when feasible) is a real saving.

The Batch API trap

OpenAI Batch API (and equivalents) is 50% off but async with
24h SLA. Tempting for offline workloads. Don't use it for
anything user-facing — the 24h variance is fatal in production
UX. Use it for nightly bulk jobs only.

How PromptLab helps with this

• Cost Analytics dashboard breaks spend down by provider,
  model, prompt, scenario, time. The "spend by model" pie chart
  surfaces over-spending faster than the bill does.
• Cache Hit Potential card measures duplicate-call rate over
  the last 30 days — if 40% of your calls are duplicates of
  recent calls, the prompt-response cache (Phase 2 — coming) will
  save you 40% on inference. Until Phase 2 ships, the card tells
  you the savings are sitting on the table.
• Per-model pricing in the Project Models page makes the
  smaller-model-trial trivial: pick gpt-4o-mini instead of
  gpt-4o, run the scenario, look at the eval pass rate. Decision
  in 5 minutes.
• DeepSeek's 50% off-peak window (16:30–00:30 UTC) is
  enforced via the time-window-discount pricing rule on each
  DeepSeek model. Schedule batch workloads to off-peak hours and
  the cost engine applies the discount automatically — no code
  branching.
• Anthropic prompt caching (Tier-3, Future) will land natively
  with cache_control extraction in the request shape and
  cost-attribution from usage.cache_creation_input_tokens /
  usage.cache_read_input_tokens. Until then, configure caching
  in the providerOptions pass-through and the router forwards it
  verbatim.

---

Different from cost. Cost is "dollars per million calls"; latency
is "seconds per call." User-facing UX usually cares about latency
more.

Where latency comes from

total_latency = network_RTT
              + provider_queue_time
              + time_to_first_token (TTFT)
              + tokens × inverse_throughput

Each component has its own lever.

TTFT and inverse throughput — the model choice

• Groq (open-weights LPU) is the fastest. Llama 3.3 70B in
  ~300ms TTFT, ~600 tokens/sec. 5-10x faster than the same weights
  on cloud GPU.
• Cerebras (similar idea, smaller catalog) is competitive.
• gpt-4o-mini, gemini-2.5-flash, claude-haiku are the fastest
  frontier-lab options. ~500ms TTFT, ~200-400 tokens/sec.
• gpt-4o, claude-sonnet, gemini-pro — ~1-2s TTFT, ~80-150
  tokens/sec.
• o4, deepseek-reasoner, grok-4 (reasoning models) — multiple
  seconds of reasoning before any visible output.

If latency is the binding constraint, reach for Groq or a
mini-tier model first. Optimising the prompt buys you 10-20%;
switching to a faster model buys you 5-10x.

Streaming — the perceived-latency cheat

If your UX renders tokens as they arrive, the user sees
"something happened" at TTFT (~500ms) instead of waiting for the
full response (~5s). Same total time; very different felt
experience. Stream wherever the UX permits.

Async patterns

• Long-running workloads (deep research, video generation,
  music gen): submit-then-poll. Don't block a request thread for
  30+ seconds; submit a job, poll for completion, render the
  result.
• Batchable workloads (nightly summaries, bulk classification):
  use OpenAI Batch API or equivalent. Cheaper but high latency.
• User-facing: streaming sync. Never batch.

Output length is latency

max_tokens=2000 and max_tokens=200 differ by 10x in latency at
the same throughput. Cap output as tightly as your UX needs.

How PromptLab helps with this

• Latency on every result card. Side-by-side cohort comparison
  shows latency for each model in milliseconds. Picking the
  cheapest-acceptable model for a latency-bound flow becomes a
  15-second exercise, not a research project.
• Provider Health dashboard (Phase D — coming) will surface
  P50/P95/P99 latency canaries hourly per provider × model so a
  model going slow is visible before it shows up in your error
  budget.
• Latency cap eval rule — set a hard cap (e.g. "P95 must be
  <2s") and the rule fails loudly when a model regresses. Catch
  drift before users do.
• Async media job lifecycle is built into the router and SDK —
  the mode: 'async' response with jobId polling is canonical
  for FAL, Replicate, Mureka, and future video / music providers.
  The desktop app will get the same UI when the async-job UI lands.

---

Most prompts don't fail the first time you write them. They fail
on iteration 4 — when you've made enough changes that the gains of
some edits have been undone by the costs of others, but you can't
remember which.

The discipline

1. Version every change. Every save = new version. Old
   versions stay queryable forever.
2. Run a fixed golden_set against each version. The
   pass-rate trend is your truth.
3. Don't change two things at once. "I rewrote the prompt
   and switched models, now it's better" tells you nothing about
   which change drove the gain. Change one variable per iteration.
4. Keep a changelog per version. "v3 → v4: added 'do not
   refuse' instruction; pass rate +6%." Past you talking to
   future you.
5. Pin canary deploys to a specific version. When you're
   ready to ship v4, route 10% of traffic to it for a day. Roll
   forward only if pass rate matches or beats v3 in production.

When to roll back

When v(n) eval pass rate drops below v(n-1) for any reason. Not
"I'll fix it forward" — roll back, then investigate. Production
quality should never regress while you debug.

When to start a fresh prompt

When you've changed >50% of the prompt content in the last
three versions, or when the pass rate has been trending down for
three versions. Don't keep band-aiding; start over with what
you've learned.

How PromptLab helps with this

• Idempotent versioned PUT — every prompt save is a new
  version with a changelog field, the previous version stays
  queryable. Roll back is one click.
• Active version pinning + canary version param. Your
  production code calls client.execute('article-summariser', {...}) and gets the active version by default; pass version: '3' to pin a canary to the last known good. 10/90 traffic
  splits live in your code, not in PromptLab UI, but the
  Interactions explorer makes the eval pass rate by version
  trivial to query.
• Scenarios are the golden_set infrastructure. Save a
  scenario once; every prompt version run against it lands in the
  same execution-group history with eval rate. Trend chart for
  free.
• labAssistantHistory[] on every scenario captures every
  Lab Assistant action (Optimize, Refine, Translate) with the
  before/after diff and applied/discarded outcome. Your prompt's
  changelog is partially auto-generated.

---

Two scenarios, different rules.

Scenario A — single-provider prompt

You've decided the prompt ships on Claude Sonnet 4.5. Lean into
Claude. XML tags, longer context, more examples, Anthropic prompt
caching. Your peak quality wins are 10-30% in eval pass rate vs a
provider-agnostic version.

Scenario B — multi-provider cohort

The prompt needs to work on Claude AND GPT-4o AND Gemini (e.g. a
fair comparison flagship, a fallback chain, a cost-tier dispatch).
Provider-leaning syntax becomes a confound — you can't tell
whether GPT did worse because GPT is worse or because the prompt
was Claude-shaped.

For B, write provider-agnostic markdown:

• Section headings as ## SECTION (all three accept).
• Examples as ### Example 1 followed by Input: / Output:.
• Output format declared as plain text, "Respond ONLY with valid
  JSON matching this schema:" + a 1-line example.
• No XML tags, no provider-specific structured-output API
  parameters in the prompt (let the request-level params handle
  enforcement separately per model).

Accept that peak quality drops 5-15% on each model versus a
provider-leaning prompt. The fairness of the comparison is worth
it.

Migration moments

When you decide to migrate a prompt from one provider to another
(GPT-4 → Claude Sonnet, Llama-on-OpenRouter → Llama-on-Groq), the
prompt almost never works as-is. The provider conventions baked
into the original prompt become noise on the new model.

The migration playbook:

1. Run the original prompt against the new model, gather a baseline
   eval pass rate.
2. Translate the syntax into the new model's preferred shape.
3. Run translated prompt against the new model, gather a translated
   pass rate.
4. Optimise the translated prompt for the new model
   (provider-leaning version of B).
5. Compare optimised pass rate vs original-on-original. Migrate
   only if optimised matches or beats original.

How PromptLab helps with this

• "Translate for this model" banner is automated step 2 of the
  migration playbook. One click per provider.
• "Optimize for one model" mode is automated step 4. Pick the
  target model, the Lab Assistant rewrites the prompt for that
  model's strengths.
• "Optimize for cohort" mode is automated for scenario B.
  Returns one prompt that all cohort models can interpret +
  per-model interpretation notes.
• Migration Wizard flagship (in flight) composes the whole
  playbook into a guided multi-step UI.

---

Patterns that look helpful but make prompts worse.

"Be sure to think carefully"

Adds cost and latency without measurably changing quality on
modern models. The model is already trying its best. This is
1-shot magical thinking dressed up as instruction.

Over-roleplay

"You are an EXPERT senior architect with 30 years of experience.
You ALWAYS produce the best possible output. You NEVER make
mistakes. You think LIKE A GENIUS." Every adjective is a wasted
token. Two sentences of role is the ceiling.

Negation overload

"Don't be vague. Don't use jargon. Don't be too short. Don't be
too long. Don't include marketing speak. Don't refuse…"
Negative instructions are weak; the model has trouble inverting
them. Reframe positively where possible. "Use plain language."
"Aim for 200-300 words."

Threats and bribes

"This is VERY IMPORTANT. I will tip you $200 if you do this
right. Lives depend on it." Mostly folklore at this point. Modern
models don't condition on these. They feel like they should work.
They don't.

"Step-by-step" on already-reasoning models

Asking o4 or deepseek-reasoner to "think step by step" can
make output worse — they reason internally; explicit CoT
instructions interfere with the model's own reasoning trace. Drop
the CoT instruction for reasoning models.

Prompt explosion

Iteration 1 was 80 words. Iteration 12 is 1,800 words. Every
iteration added a constraint to fix some edge case. The base
quality has now degraded under the weight of the constraints.
Trim ruthlessly. Delete every constraint that isn't paying for
itself in eval pass rate.

"JSON" without strictness

"Return JSON." — vague. The model returns JSON wrapped in
markdown fences with a chatty preamble. "Respond ONLY with valid
JSON. No prose. No markdown fences. No explanation." — strict.
Or use response_format API param.

Asking the model what to do

"What should I include in the summary?" — that's not a prompt,
that's a planning conversation. Get the planning out before you
write the prompt. The prompt is the verdict, not the deliberation.

How PromptLab helps with this

• Optimize sheet does a hard pass over your prompt and
  highlights cruft. The diff shows what got cut, the rationale
  explains why.
• Diff view makes prompt explosion visible — when v12 is
  visibly 5x longer than v3 and the eval pass rate is the same,
  the chart tells you to roll back to v3.
• Lab Assistant has rated thumbs-up/down feedback that builds
  preference data for future versions of the assistant. Marking a
  bad suggestion as "down" makes the next assistant better at
  spotting that anti-pattern.

---

The prompt works in dev. Now make it survive in production.

Fallback chains

The primary model goes down. Or hits a rate limit. Or a single
call returns something unparseable. Your code should:

• Retry once on transient failures (5xx, 429 with Retry-After).
• Fall back to a secondary model on persistent failures. Log
  the fallback so you can quantify how often it triggers.
• Surface a graceful error to the user only as a last resort.

A reasonable default fallback chain for a Claude-primary prompt:
claude-sonnet-4.5 → gpt-4o → claude-haiku (smaller / cheaper
but probably available).

Timeouts

Set per-call timeouts. Without one, a hung provider call hangs
your service. Reasonable defaults:

• Chat completion (small model): 10s
• Chat completion (frontier): 30s
• Reasoning model (o4, deepseek-reasoner): 90s
• Media generation (FAL, Replicate): submit-poll, no synchronous
  timeout

Cost caps per call

The model returns 50,000 tokens because the prompt was malformed
and there was no max_tokens. The bill is $1.50 for that one
call. Now imagine 10,000 of those.

max_tokens is your cost cap. Always set it. Fail closed.

Rate limit awareness

Each provider has rate limits. They're per-key, often per-model,
sometimes per-account. Plan for 429s; respect Retry-After headers;
back off exponentially.

PII / sensitive data

Don't put production secrets in prompts. Don't put customer PII
in prompts unless you've designed for it (DPA in place, retention
policy, user consent). Log scrubbing matters.

Observability minimum bar

Per call, log:

• prompt id + version
• model + provider
• input token count
• output token count
• cost
• latency
• status (success / failed / cancelled)
• eval rule pass/fail
• request id from the provider

Minimum bar for being able to debug a customer-reported issue 3
weeks from now.

How PromptLab helps with this

• fallbackModelCodes field on every prompt spec. Set the
  fallback chain at registration; the router executes the
  cascade automatically. metadata.isFallback: true is logged on
  the interaction so fallback rate is queryable.
• Per-prompt modelParameters.maxTokens — set once, applied
  on every call. No way to forget the cap on individual call
  sites.
• Provider Health dashboard (Phase D — coming) catches
  rate-limit spikes before users do. SLO alerts trigger when P95
  latency or failure rate breaches threshold.
• Interactions explorer captures everything in the
  observability list above on every interaction, automatically.
  Filter by status / cost / latency / model / version → debug
  customer issues in seconds, not hours.
• Cost cap eval rule fails the rule loudly if a single call
  exceeds the cap. Catch malformed-prompt cost explosions on the
  first call, not after 10,000.

---

A prompt that's good today is mediocre in six months. The world
moves: customer expectations rise, new models ship, the input
distribution drifts. A prompt without a feedback loop slowly rots.

The feedback sources

1. Eval rule failures over time. A rising failure rate is the
   earliest signal that something is off.
2. User thumbs-up/down on outputs. If your product surfaces
   AI-generated outputs to users, capture explicit ratings. The
   roll-up is your truest quality signal.
3. Cost-per-success drift. Cost going up while pass rate stays
   flat means the model is producing more tokens for the same
   quality — your prompt may be drifting verbose.
4. Latency drift. Same.
5. New-model release events. Every frontier-lab release is a
   forcing function: re-baseline your prompts against the new
   model to see if there's a free quality gain.

The cadence

• Weekly: glance at eval pass-rate trend, cost trend.
• Monthly: run the golden set against the active prompt and
  compare to last month. Investigate any drop.
• Quarterly: re-baseline against new-model releases. Migrate
  to a stronger model if it's free quality.
• Per incident: root-cause any user-reported regression with a
  before/after replay.

The metric trap

Don't optimise the metric you can measure at the expense of the
outcome you actually want. Eval pass rate going up while user
NPS goes down means your eval is poorly designed. Reset.

How PromptLab helps with this

• Eval rule pass-rate over time is queryable directly from
  the Interactions explorer with date-range filters. Per-version
  trend lines fall out.
• Lab Assistant rating telemetry — every assistant suggestion
  carries a thumbs-up/down. The aggregate is the assistant's
  quality dashboard. The same pattern applies if you surface AI
  outputs in your own product: pipe ratings back into a feedback
  collection tied to interactions, and you get a per-prompt
  quality signal at scale.
• Replay runs old interactions against the current prompt
  version. "Did v5 fix the failures from last month?" → load the
  failed interactions → click Replay → check the pass rate. 60
  seconds.
• Migration Wizard (flagship) automates the new-model
  re-baseline. Pick a candidate model, paste the existing prompt,
  the wizard runs the playbook (Translate → Optimize → eval
  comparison) and gives you a "ready to migrate / not yet"
  verdict.

---

The 60-second checklist to run before promoting a prompt to
production.

[ ] Success criteria written down (rubric or eval rules)
[ ] golden_set has ≥10 representative inputs, all pass
[ ] Output format is strict (JSON schema, or marker tags)
[ ] max_tokens cap set
[ ] Temperature explicit (don't rely on default)
[ ] Fallback model configured
[ ] Latency under your UX budget at P95
[ ] Cost-per-call known and acceptable
[ ] No PII / secrets in prompt template
[ ] Versioned with changelog
[ ] Active version pinned, canary plan documented
[ ] Eval rules attached and passing
[ ] Provider account budget alert configured

If any box is unchecked, you don't ship. You can write down a
deliberate exception ("we are knowingly accepting variance for v1")
but the box is checked because the conversation happened, not
because nobody thought about it.

Worked example — pre-flight for a real prompt

A customer support summariser:

• ✅ Success criteria: Output must contain (a) the customer
  name, (b) the issue category, (c) at most 3 action items, in
  strict JSON.
• ✅ golden_set: 25 anonymised support transcripts in JSONL,
  each with the expected JSON output. Current pass rate: 23/25
  (92%). Acceptable.
• ✅ Output format: OpenAI strict JSON schema.
• ✅ max_tokens: 400 (output JSON for this task is ~200-300).
• ✅ Temperature: 0.0 (extraction task, no creativity needed).
• ✅ Fallback: primary gpt-4o-mini, fallback
  claude-haiku-4.
• ✅ Latency: P95 1.8s on gpt-4o-mini. Budget is 3s. Pass.
• ✅ Cost: $0.0008 / call avg. Volume 50k/month → $40/month.
  Acceptable.
• ✅ PII: transcripts already redacted at ingest; prompt
  doesn't request email / phone in the output.
• ✅ Versioning: v3, changelog "added action-item cap of 3
  after v2 was returning 7-item dumps".
• ✅ Canary: code routes 10% to v3, 90% to v2 for first 24h.
• ✅ Eval rules: JSON schema validation, length cap on
  action-items array, "Output must contain customer name field
  non-empty" Contains rule.
• ✅ Budget alert: project-level $200/month cap with email
  alert at 80%.

Ship it.

How PromptLab helps with this

• Save Prompt dialog is the natural enforcement point. Most
  of the checklist items map directly to fields:
  • Success criteria → eval rules attached on the prompt
  • Output format → outputDefinition.outputJsonTemplate
  • max_tokens → modelParameters.maxTokens
  • Temperature → modelParameters.temperature
  • Fallback → fallbackModelCodes
  • Versioning → automatic on save, with changelog field
  • PII / secrets → not enforced; this remains your discipline
• Project-level budget alerts in the Cost Analytics dashboard
  cover the last item.
• Pre-flight scenario run — save the golden set as a scenario,
  click "Run All" before activating a new version. You see the
  pass rate before you ship, not after.

---

"Should I use few-shot?"

Eval pass rate at zero-shot:
  ≥ 90%       → ship zero-shot
  80-90%      → try one-shot
  70-80%      → try few-shot (3 examples)
  60-70%      → try few-shot + chain-of-thought
  < 60%       → switch to a stronger model first; few-shot
                 won't fix a model mismatch

"Should I use a stronger model?"

gpt-4o-mini / claude-haiku / gemini-flash on golden set:
  pass rate ≥ 90%   → ship the cheap model
  pass rate 80-90%  → try the frontier model; ship if
                       pass rate ≥ 95%, otherwise stick with cheap
  pass rate < 80%   → use frontier model, accept the cost

"Should I use a reasoning model?"

Task involves multi-step reasoning, logic, math, or planning:
  user-facing latency budget ≤ 5s   → use frontier non-reasoning + CoT in prompt
  user-facing latency budget > 5s   → use o4 / grok-4 / deepseek-reasoner
  offline batch                     → use the reasoning model regardless

"Should I cache?"

Same large prefix (>2k tokens) used across many calls:
  yes always — Anthropic / OpenAI / DeepSeek cache rates are 10-25% of fresh
Different prefix every call:
  no — caching adds latency; doesn't help

---

You have this problem	Reach for this in PromptLab
"Don't know where to start"	✨ Compose with AI
"Output is good on Claude but bad on GPT"	Optimize for cohort
"Output is good but I want it tuned for one model"	Optimize for one model
"One specific thing is wrong (e.g. missing date)"	Refine
"I migrated to Claude and lost quality"	Translate banner
"Don't know why this run returned weird output"	Explain this run
"Eval rule fails sometimes — what should I change?"	Inline coach hints
"What did I save under? What was the prompt last week?"	Versions tab + Replay
"Did v4 actually improve over v3?"	Scenarios + execution-group history
"Is this prompt drifting expensive?"	Cost Analytics + Cache Hit Potential
"Production logged something weird"	Interactions explorer + Replay
"Want to know if migrating to a new model is safe"	Migration Wizard (flagship, in flight)

---