What is prompt versioning?

Q: What happens when a model provider silently updates the underlying model?

If your prompt version uses a floating alias, a provider update can change your function's behavior without any action on your part. Pinning a specific model identifier in your version snapshot protects against this — if behavior drifts, you have a record of exactly which model and prompt ran.

Q: How does prompt versioning relate to model fallback?

They are complementary. Versioning ensures the right logic ships to production; automatic model fallback ensures it keeps running if the primary model has an outage. An AI Function Runtime handles both: the function is version-gated before it ships and auto-routed to a backup model if the primary goes down.

Prompt versioning tracks every change to an LLM prompt—instructions, model, parameters, schema—as an immutable snapshot you can evaluate, compare, and roll back.

Prompt versioning is the practice of recording every change to an LLM prompt — its instruction text, model identifier, temperature, and output schema — as an immutable, named artifact that can be compared, evaluated against a baseline, and rolled back when a change regresses production quality.

Why Git alone doesn't version your AI feature

Your application code is versioned. Your AI feature's behavior isn't — not by default.

Git records the diff when you edit src/prompts/classify.ts. What it doesn't capture is the full execution contract of that call: which model version ran, at which temperature, with which system prompt, against which output schema. Swap any one of those and your AI feature's output can shift substantially — with zero lines of application code changing.

The OpenAI developer community has documented repeated cases where behavior shifted after a default model was updated under the same API identifier — developers on Azure reported behavior changes in GPT despite identical setup: unchanged prompts, unchanged temperature, pinned model strings. The versioning problem isn't only about the edits your team makes; it's also about changes that happen to you.

What a prompt version actually captures

A well-structured version is a snapshot of the complete execution context, not just the text string. That means:

System prompt and message template — the full instruction text, including any dynamic slots
Model identifier — pinned to a specific release (e.g., gpt-4o-2024-08-06), not a floating alias
Parameters — temperature, max tokens, top_p, stop sequences
Output schema — the JSON structure or format the model is expected to return
Dependencies — tool definitions, RAG configuration, or injected context that affects output

In a multi-agent workflow, the output of one function is the input of the next. You're not versioning a text string — you're versioning the behavioral contract of an entire step.

The promotion gate: evaluate before you ship

Versioning without evaluation is labeled chaos. The value of a version number is that it gates promotion: you don't push v4 to production until it demonstrably outperforms v3 on the tests that matter.

What that looks like in practice:

fn eval classify --version v4 --threshold 0.95

Running classify@v4 against golden dataset (120 examples)…
  accuracy:   0.97  ✓  (v3 baseline: 0.94)
  p95 latency: 188ms
  cost/call:  $0.0021
  status: PASS — ready to promote

The gate answers three questions before anyone calls this function in production: does the new version produce better output? Does it stay within cost tolerance? Does it clear the threshold you defined — not the one you eyeballed after the fact?

Prompt versioning as a first-class function concern

The problem with bolt-on prompt versioning tools is that versioning is only one piece. You also need the function to stay callable during a model outage, to run in a consistent environment, and to surface per-version metrics after weeks of real traffic. Solving versioning in isolation leaves the other three unsolved.

Functional AI treats each prompt, agent, or multi-agent workflow as a function. Versioning is built in, not bolted on. When you deploy a new version:

fn deploy classify@v4

That version is:

Evaluated before it reaches production — v4 has to pass your quality threshold; there is no other path to production
Callable without redeploying your app — your application calls fn run classify, and the runtime resolves the current certified version
Observable per-version — metrics (latency, cost, pass rate) break down by version so you know if v4 drifts three weeks into production
Auto-failed-over — if the underlying model has an outage, Functional AI switches to a backup and keeps serving; your application doesn't know or care

The practical consequence: when the engineer who wrote classify@v3 leaves the team, nothing breaks. v3 is still running. v4 will go through the same evaluation gate before anyone can promote it. The institutional knowledge lives in the function definition — not in one person's head.

That's what Engineering Ownership looks like for AI features. The function is a durable artifact, not a pile of prompt strings scattered across branches.

FAQ

What is prompt versioning?

Prompt versioning is the practice of tracking every change to an LLM prompt — its instructions, model identifier, parameters, and output schema — as an immutable named version, so teams can compare versions, evaluate before promoting, and roll back when something regresses. It is the foundation of any production prompt management workflow.

How is prompt versioning different from storing prompts in Git?

Git versions your application code, not the behavioral contract of your AI feature. A prompt version captures the full execution context as a single immutable snapshot — model identifier, temperature, schema, instruction text — and gates promotion on passing an evaluation against a known baseline. Git records the diff; a prompt versioning system gates the deploy.

What should a prompt version include?

At minimum: the system prompt and message template, a pinned model identifier (not a floating alias), generation parameters (temperature, max tokens), and the expected output schema. For agents or multi-step workflows, include tool definitions and any retrieval configuration that affects output — because a schema change in one step can silently break downstream steps.

Why do I need an evaluation gate before promoting a prompt version?

LLMs are non-deterministic. A change that looks correct in a playground can silently degrade output across thousands of real requests. An evaluation gate runs the new version against a golden dataset and blocks promotion until it meets a defined quality threshold — the same way CI blocks a code merge on a failing test.

What happens when a model provider silently updates the underlying model?

If your prompt version uses a floating alias (e.g., gpt-4o without a date suffix), a provider update can change your function's behavior without any action on your part. Pinning a specific model identifier in your version snapshot protects against this. If behavior drifts, you have a record of exactly which model and prompt combination ran.

How does prompt versioning relate to model fallback?

They're complementary. Versioning ensures the right logic ships to production; automatic model fallback ensures it keeps running if the primary model has an outage. An AI Function Runtime handles both: the function is version-gated before it ships and auto-routed to a backup model if the primary goes down — without any change to how your application calls it.

Functional AI is currently in private beta. If you're building AI features and want version-controlled, evaluated functions without stitching the infrastructure together yourself, join the waitlist at fnai.dev.