feat(gepa): add tool description optimization for multi-agent systems

[Ju-usc]

Summary

Addresses #8706 which requested GEPA to optimize tool descriptions. This PR expands on that to enable comprehensive ReAct module optimization with joint optimization of all four ReAct components: react instructions, extract instructions, tool descriptions, and tool argument descriptions.

When optimize_react_components=True, GEPA discovers all dspy.ReAct modules in your program (including nested multi-agent systems) and uses a specialized reflection prompt to jointly optimize how agents reason, select tools, and extract answers from execution trajectories. All ReAct components are optimized together based on shared execution traces, enabling the reflection LM to generate cohesive instructions since it sees how components work together (not optimized in isolation). This addresses the ReAct trajectory prefix duplication issue (gepa-ai/gepa#97).

Fully backward compatible - Default optimize_react_components=False preserves existing behavior.

Issue

Closes #8706 - Original request was to enable GEPA to optimize tool descriptions. This PR expands on that to optimize all four ReAct components jointly (react instructions, extract instructions, tool descriptions, and tool argument descriptions) for more effective agent optimization.

Changes

Core Implementation

• Add optimize_react_components parameter to GEPA (default False for backward compatibility)
• Unified ReAct module optimization - Treats each dspy.ReAct as one module with react/extract/tools as subcomponents, respecting both GEPA's module-level abstraction and DSPy's ReAct module design
• Efficient reflective dataset - Single trajectory per ReAct execution shared across all components (eliminates duplicate trajectory formatting for separate components)
• ReActModuleProposer with dynamic signatures - Specialized proposer that generates output fields for each tool/parameter, enabling selective optimization
• ReAct module discovery - Traverse program via named_sub_modules() to find all dspy.ReAct instances (supports deeply nested multi-agent architectures)
• Component serialization - Serialize ReAct modules as JSON configs containing react/extract instructions and tool schemas
• Intelligent routing - Direct ReAct components to ReActModuleProposer, regular predictors to default/custom proposers
• Component updates - Apply optimized react/extract instructions, tool descriptions, and tool argument descriptions back to ReAct modules (propagates arg_desc to tool.args for prompt rendering)

Testing

• 8 comprehensive tests covering:
  • Single ReAct module detection
  • Multi-ReAct workflow discovery (mixed ReAct + non-ReAct modules)
  • Nested orchestrator-worker patterns (hierarchical agents)
  • Program building with optimized components
  • Reflective dataset creation with trajectory feedback
• All tests validate: Discovery logic, JSON serialization, component routing, program reconstruction

Documentation

• GEPA_Advanced.md - Complete ReAct optimization guide:
  • What gets optimized (4 components with selective optimization)
  • When to use (5 common failure patterns)
  • How it works (discovery → serialization → optimization → application)
  • Usage examples (basic agent + multi-agent system)
  • Custom proposer interface with reference implementation
• overview.md - Brief introduction linking to advanced guide
• Reflection prompt documentation - Explains progressive optimization philosophy

Usage Example

Basic ReAct Agent

import dspy

# Define tools
def search_web(query: str) -> str:
    return f"Search results for: {query}"

search_tool = dspy.Tool(search_web, name="search", desc="Searches")

# Create agent
agent = dspy.ReAct("question -> answer", tools=[search_tool])

# Optimize with GEPA
gepa = dspy.GEPA(
    metric=my_metric,  # Your evaluation metric
    reflection_lm=dspy.LM(model="gpt-5"),
    optimize_react_components=True,
    auto="medium"
)

optimized = gepa.compile(agent, trainset=trainset, valset=valset)

Multi-Agent System

import dspy

def search_web(query: str) -> str:
    return f"Search results for: {query}"

def calculate(expression: str) -> float:
    return eval(expression)

search_tool = dspy.Tool(search_web, name="search", desc="Searches")
calc_tool = dspy.Tool(calculate, name="calculator", desc="Computes")

class ResearchAssistant(dspy.Module):
    def __init__(self):
        super().__init__()
        self.researcher = dspy.ReAct("query -> findings", tools=[search_tool])
        
        def delegate_research(query: str) -> str:
            return self.researcher(query=query).findings
        
        research_tool = dspy.Tool(delegate_research, name="research", desc="Delegates")
        self.assistant = dspy.ReAct("question -> answer", tools=[research_tool, calc_tool])
    
    def forward(self, question):
        return self.assistant(question=question)

# Optimizes ALL ReAct modules and tools
gepa = dspy.GEPA(
    metric=my_metric,
    reflection_lm=dspy.LM(model="gpt-5"),
    optimize_react_components=True,
    auto="medium"
)

optimized = gepa.compile(ResearchAssistant(), trainset=trainset, valset=valset)

Key Features

Joint Optimization:

• React instruction and tool descriptions are optimized together based on execution traces
• The reflection LM sees how components work together (not optimized in isolation)
• Can generate more cohesive instructions across all components

Selective Optimization:

• Reflection LM returns None for components that should stay unchanged
• Only components that need improvement are updated
• Enables progressive refinement across GEPA iterations

Multi-Agent Support:

• Automatically discovers nested ReAct modules
• Optimizes parent and sub-agent modules cohesively
• Handles complex delegation patterns

[Ju-usc]

Apologies for accidentally closing #8927

Thank you for the thorough review, @LakshyAAAgrawal! I'll address your feedback:

1. Since tools are categorically different from prompts, they should use a different reflection meta prompt. The default reflection meta prompt is shown here https://dspy.ai/api/optimizers/GEPA/GEPA_Advanced/#default-implementation, whereas I assume that the tool must use somewhat different meta prompt. Can you implement a propose_new_texts method that mimics the default_proposer shown in the link above for all prompts, but calls to a tool description specific prompt/signature for tool evolution.
2. Can you also add some description to the documentation, explaining that this feature is beneficial for React agents.
3. (This is not a requirement to merge the PR) Would it be possible to add a simple and short tutorial demonstrating the use and performance improvement via tool evolution?

I'll start working on items 1 and 2 and update the PR soon. Please let me know if you have any specific preferences for the tutorial format!

[LakshyAAAgrawal]

Thanks a lot! For the tutorial, I think you can follow the current GEPA tutorial format (load a dataset, show an example from the dataset, build a dspy program, evaluate the baseline program on testset, run GEPA with new optimization settings, show the optimized programs' prompts and tool descriptions, and finally evaluate the optimized program).

Hopefully we should be able to see a nice and large gain on agentic tasks with this amazing contribution by you!

[Ju-usc]

Hi @LakshyAAAgrawal,

I've implemented the tool-specific proposer as requested! Here's what's included:

1. Tool-Specific Proposer Implementation ✅

• Added GenerateImprovedToolDescriptionFromFeedback signature with a specialized reflection prompt
• Implemented ToolProposer and SingleComponentToolProposer following the MultiModalInstructionProposer pattern
• Routing logic in DspyAdapter that directs tools to ToolProposer and signatures to custom/default proposers
• Fully backward compatible with existing custom instruction proposers

2. Documentation ✅

• Added comprehensive section to GEPA_Advanced.md
• Explains when to use tool optimization (ReAct agents, multi-agent systems)
• Includes usage examples for both simple and nested agent architectures
• Documents how to inspect optimized tool descriptions

Reflection Prompt Design:
The tool-specific prompt is intentionally open-ended to avoid prescriptive patterns that might lead to local minima. It asks the LM to identify patterns in successful/unsuccessful tool usage and extract domain-specific information, without suggesting specific heuristics.

Before I create a short tutorial (item #3), would you have any feedback on:

• The reflection prompt design - is it general enough? Any improvements you'd suggest?
• The implementation approach - does the routing logic make sense?
• The documentation - anything unclear or missing?

Any feedback would be helpful before I invest time in the tutorial. Thank you!

[Ju-usc]

wait there is a bug in the implementation working on it to fix. Also test has to be fixed.

[LakshyAAAgrawal]

which tool to use, when to use it, and how to use it. All three are captured by the description.

[LakshyAAAgrawal]

Let's avoid the word "fundamentally". One can imagine that all of tool descriptions can (and many times do) simply included in the system prompt itself.

[LakshyAAAgrawal]

Please also add a corresponding entry in GEPA Overview, that links to this file/section.

[LakshyAAAgrawal]

One should consider using this, when they use dspy.Tool anywhere in the DSPy program. Here are a few scenarios for using dspy.Tool:

[LakshyAAAgrawal]

I don't think we need to inform users about backward compatibility here. It should be implicit that there should be no behaviour changes for any program not containing dspy.Tool.

[LakshyAAAgrawal]

Add a link to GEPA Advanced/Tool section

[LakshyAAAgrawal]

Edge case: What should happen when user tries to provide both a custom proposer, and enables optimize_tool_descriptions

[LakshyAAAgrawal]

This is a slight deviation from this PR, but would be a large enhancement (feel free to ignore):

1. Create 2 fields, self.instruction_proposal_signature and self.tool_proposer, which are initialized to the default InstructionProposalSignature and ToolProposerSignature.
2. Take an argument from dspy.GEPA that can override the default signature values.

[LakshyAAAgrawal]

Is this robust? Might it be better to use isinstance or some other way?

[LakshyAAAgrawal]

Tool use. Also suggest identifying any failure modes of the tool?

[LakshyAAAgrawal]

Dear @Ju-usc,

This is a great PR. Thanks a lot! I have tried to be overly critical and made too many nits. Feel free to ignore if you disagree with something. Let me know if you'd like me to address anything!

Regarding the meta prompt, overall I think it looks great. However, I suggest that as you build the tutorial, you may find that the reflection prompt needs tweaking, or the content exposed in reflective_dataset for the tool may be lacking or need improvement. This is going to be an empirical exercise, which will guide what works in the reflection meta prompts. ! Looking forward to the tutorial on this too!

You may already have thoughts about what you'd like to show in the tutorial, but if not, you may consider building off (https://kargarisaac.medium.com/building-and-optimizing-multi-agent-rag-systems-with-dspy-and-gepa-2b88b5838ce2) by @kargarisaac.

[LakshyAAAgrawal]

@Ju-usc, if Option 2 will be detrimental to optimization quality, I think we should preserve option 1, i.e., allow for full generality, but leverage additional information from dspy.ReAct when available.

But the real question is that is there a way to achieve the same optimization quality even with option 2, the generalization.

[chenmoneygithub]

@Ju-usc @LakshyAAAgrawal From my perspective the quality should be on par if we capture the tool trace correctly, because ReAct itself is just a way of interacting with tools. But definitely need to run experiments to find it out.

There is one prerequisite we need to build - capture tool usage in dspy.settings.trace. I see two viable options here:

1. Use the callback to inject the tool trace into the dspy.settings.trace, which is only enabled when running GPEA + optimize_tool=True.
2. Explicitly add tool calling inputs/outputs to dspy.settings.trace

I like 1 better because that doesn;t affect existing behavior. In general, I would like to avoid shipping an optimization algorithm that has many special logic for dspy.ReAct, which will create a maintenance blackhole for us in the long run. For example, it is subject to broken without awareness when ReAct code is changed.

@Ju-usc This work is very challenging, but super cool. We really appreciate your contribution here.