September 26, 2026
Optimizing AI prompts effectively can transform how language models tackle complex tasks. The GEPA framework introduces a structured approach to prompt evolution, combining multi-component prompts, actionable feedback, and validation techniques. This guide explores how to implement GEPA for arithmetic problem-solving, demonstrating its ability to refine weak initial prompts into high-performing solutions.
Reflective prompt optimization isn’t just about tweaking words—it’s about creating a dynamic system where AI models learn from failures and adapt. By leveraging GEPA, developers can systematically improve model outputs while maintaining control over the optimization process. Below, we’ll walk through the entire workflow, from setup to validation, to showcase how this method elevates AI performance.
To begin, you’ll need to install **GEPA** and **LiteLLM**, two critical components for prompt optimization. These tools streamline model interactions and feedback loops, making it easier to refine prompts iteratively.
“python
pip install -q gepa litellm
import os, re, json, random, getpass, textwrap
import litellm
import gepa.optimizeanything as oa
from gepa.optimizeanything import (
optimizeanything, GEPAConfig, EngineConfig, ReflectionConfig,
)
litellm.suppressdebuginfo = True
if not os.environ.get(“OPENAIAPIKEY”):
os.environ[“OPENAIAPIKEY”] = getpass.getpass(“Enter your OpenAI API key: “)
“
GEPA relies on two distinct models:
– A **task model** (e.g., `openai/gpt-4o-mini`) to solve problems.
– A **reflection model** (e.g., `openai/gpt-4.1`) to analyze failures and suggest improvements.
Setting a metric-call budget (e.g., MAXMETRICCALLS = 100) ensures the optimization process remains efficient without excessive API usage.
A well-structured dataset is the foundation of effective prompt optimization. For this tutorial, we’ll generate arithmetic word problems covering four scenarios:
– Discount calculations
– Travel distance computations
– Wallet balance adjustments
– Chained numerical operations
Each problem includes a programmatically generated correct answer, ensuring consistency and reliability during evaluation.
“python
def makeproblems(n, seed=0):
rng = random.Random(seed)
out = []
for in range(n):
t = rng.choice([“discount”, “travel”, “wallet”, “chain”])
if t == “discount”:
unit = rng.choice([40, 60, 80, 120])
qty = rng.choice([5, 6, 8, 10])
disc = rng.choice([10, 20, 25, 50])
total = unit qty
gold = total – total disc // 100
q = (f”A shop sells notebooks at {unit} rupees each. You buy {qty} ”
f”notebooks and get a {disc}% discount on the total bill. ”
f”How many rupees do you pay in total?”)
# Additional problem types (travel, wallet, chain) follow similar structure
out.append({“question”: q, “answer”: gold})
return out
“
After generating 18 problems, we shuffle and divide them into:
– A **training set** (12 problems) for optimization.
– A **validation set** (6 problems) to test generalization.
This split ensures the optimized prompt performs well on unseen data.
GEPA’s effectiveness hinges on its ability to provide **structured feedback**. The system prompt combines:
– **Instructions** for solving the problem.
– **Output format rules** to standardize responses.
“python
def buildsystemprompt(candidate: dict) -> str:
return (f”{candidate[‘instructions’]}nn”
f”OUTPUT FORMAT RULES:n{candidate[‘formatrules’]}”)
“
The evaluator checks if the model’s response matches the expected answer. If not, it generates **actionable feedback** explaining the error, which GEPA uses to refine the prompt.
“python
def evaluate(candidate: dict, problem: dict) -> dict:
systemprompt = buildsystemprompt(candidate)
userprompt = problem[“question”]
response = litellm.completion(
model=TASKLM,
messages=[
{“role”: “system”, “content”: systemprompt},
{“role”: “user”, “content”: userprompt},
],
)
# Extract and compare the model’s answer with the gold standard
# Return structured feedback if incorrect
“
GEPA’s configuration includes:
– **Engine settings** (e.g., model selection, call limits).
– **Reflection parameters** to guide feedback generation.
“python
config = GEPAConfig(
engine=EngineConfig(
taskmodel=TASKLM,
reflectionmodel=REFLECTIONLM,
maxmetriccalls=MAXMETRICCALLS,
),
reflection=ReflectionConfig(
feedbackformat=”structured”,
),
)
“
GEPA iteratively tests candidate prompts, using feedback to evolve both **instructions** and **output format rules**. This multi-component approach ensures comprehensive improvements.
“python
optimizedprompt = optimizeanything(
config=config,
initialprompt=seedprompt,
trainset=trainset,
evaluator=evaluate,
)
“
After optimization, we evaluate the **baseline prompt** and **optimized prompt** on the validation set. Key metrics include:
– **Accuracy** (percentage of correct answers).
– **Consistency** (adherence to output format rules).
The evolutionary process typically yields:
– Higher accuracy due to refined instructions.
– More reliable outputs thanks to stricter format rules.
– Reduced ambiguity in problem-solving approaches.
GEPA (Generative Evolutionary Prompt Architecture) is a framework that uses structured feedback and iterative testing to refine AI prompts. It helps language models improve by learning from mistakes and adapting instructions dynamically.
Multi-component prompts separate instructions from output format rules, allowing GEPA to optimize each element independently. This leads to more precise and consistent model responses.
Held-out validation involves testing the optimized prompt on a separate dataset not used during training. This ensures the improvements generalize to new, unseen problems.
The GEPA framework revolutionizes AI prompt optimization by combining structured feedback, multi-component prompts, and held-out validation. Starting with a weak seed prompt, developers can systematically refine instructions and output rules to achieve superior performance. This method not only enhances accuracy but also ensures consistency across diverse problem types.
By following this step-by-step guide, you can implement GEPA for arithmetic problem-solving or adapt it to other AI tasks. The key takeaways include:
– The importance of a deterministic benchmark dataset for reliable evaluation.
– The power of structured feedback in guiding prompt evolution.
– The necessity of validation on held-out data to confirm generalization.
As AI models grow more sophisticated, frameworks like GEPA will play a crucial role in unlocking their full potential.
