pet clinic agent issue

[sguruvar]

@awsapm can you triage this issue

Problem: Agent Hallucination Despite Service Errors

Your nutrition agent is recommending products for pets that don't exist in your database, even after receiving error responses from the nutrition service.

Key Evidence from Logs:
"Do we have ferret-specific kibble in stock? What brands do we recommend for ferrets?"
Tool Result: "Nutrition info for ferret: Error: Nutrition service could not find information for pet: ferret"

Despite this error, the agent continues providing recommendations.

Supported vs. Unsupported Pet Types:

✅ Available Products (200 responses):
• Dog (1,201 requests)
• Cat (1,149 requests)
• Bird (782 requests)
• Hamster (644 requests)
• Snake (627 requests)
• Lizard (406 requests)

❌ Unavailable Products (404 responses):
• Rabbit (1,520 requests - highest failure!)
• Turtle (369 requests)
• Ferret (369 requests)
• Guinea pig (348 requests)
• Parrot (182 requests)
• Bearded dragon (167 requests)
• Fish (120 requests)

Critical System Issues:

1. Agent Error Handling Failure:
   • Pet clinic agent: 100% error rate, 70.27% fault rate
   • Nutrition agent: 13.46% error rate, 1,425 error occurrences
   • Bedrock throttling exceptions causing timeouts

2. Agent Hallucination:
   • Agents provide recommendations despite receiving "Error: Nutrition service could not find information" responses
   • No proper error handling when nutrition data is unavailable

Immediate Fixes:

1. Update Agent Prompts:
   • Configure agents to explicitly state when products aren't available
   • Add validation to prevent recommendations for unsupported pets

2. Fix Error Handling:
   • Implement proper fallback responses when nutrition service returns 404
   • Add circuit breaker patterns for failed service calls

3. Database Alignment:
   • Either add nutrition data for missing pets (rabbit, ferret, turtle, etc.)
   • Or update agent knowledge to only include supported pets

The root cause is that your agents are hallucinating recommendations instead of properly handling service errors when nutrition data is unavailable.

[github-actions]

🎯 Application observability for AWS Investigation Complete

Investigation completed successfully! Here are the results:

---

Root Cause Analysis

Agent Hallucination Issue Confirmed: Bedrock agents are providing nutrition recommendations despite receiving 404 errors from the nutrition service.

Key Findings from Trace Analysis:
• Pet clinic agent: 100% error rate with Bedrock throttling exceptions
• Nutrition service returns proper 404s for unsupported pets (rabbit, ferret, turtle, etc.)
• Agents ignore error responses and hallucinate product recommendations

Critical Fixes Required

1. Agent Prompt Engineering
   • Update agent instructions to explicitly handle nutrition service errors
   • Add validation logic: "If nutrition service returns error, inform user no products available"
   • Remove hallucinated product knowledge from agent context

2. Error Handling Implementation
   • Implement circuit breaker pattern for failed nutrition service calls
   • Add proper fallback responses for 404 errors
   • Configure agents to acknowledge service limitations

3. Data Alignment
   • Either expand nutrition database to include missing pets (rabbit, ferret, turtle, guinea pig, parrot, bearded dragon, fish)
   • Or update agent knowledge base to only reference supported pets (dog, cat, bird, hamster, snake, lizard)

4. Bedrock Throttling
   • Implement exponential backoff for Bedrock API calls
   • Add request rate limiting to prevent throttling exceptions
   • Consider upgrading Bedrock provisioned throughput

Immediate Action Items

• Update agent prompts in /scripts/agents/ directory
• Add error validation logic to nutrition agent responses
• Implement proper 404 handling in agent tool responses
• Test agent behavior with unsupported pet queries

The core issue is agents are trained to be helpful but lack proper error boundary handling when external services fail.

---

✅ Status: Complete
👤 Requested by: @sguruvar
🔗 Workflow: View details

Powered by AI Agent

[sguruvar]

@awsapm can you try to fix the issue • Nutrition service returns proper 404s for unsupported pets (rabbit, ferret, turtle, etc.)

[github-actions]

🎯 Application observability for AWS Investigation Complete

Investigation completed successfully! Here are the results:

---

Root Cause Analysis

The nutrition service correctly returns 404 errors for unsupported pets (rabbit, ferret, turtle, etc.), but the AI agents are programmed to be helpful and provide recommendations even when the service fails.

Key Issues Identified

• Agent Error Handling: The get_nutrition_data() function in nutrition_agent.py catches 404 errors but returns generic error messages instead of properly handling unsupported pets

• Hallucination Problem: Agents ignore service errors and provide fabricated product recommendations from their training data

• Limited Pet Database: The nutrition service only supports 6 pet types (cat, dog, lizard, snake, bird, hamster) but agents receive queries for many more

Required Fixes

1. Update Agent Error Handling
   • Modify get_nutrition_data() in /pet_clinic_ai_agents/nutrition_agent/nutrition_agent.py to explicitly handle 404s
   • Add validation to prevent hallucinated responses when service returns errors
   • Update system prompt to acknowledge service limitations

2. Expand Nutrition Database
   • Add missing pet types to /pet-nutrition-service/db-seed.js: rabbit, ferret, turtle, guinea pig, parrot, bearded dragon, fish
   • Ensure comprehensive coverage of common pets

3. Improve Agent Instructions
   • Update system prompts to explicitly state: "If nutrition service cannot find information for a pet type, inform the user that we don't currently have nutrition data for that specific pet"
   • Remove ability to hallucinate product recommendations

4. Add Circuit Breaker Pattern
   • Implement proper fallback responses for service failures
   • Add timeout and retry logic for nutrition service calls

The core issue is that agents prioritize being helpful over being accurate, leading to hallucinated recommendations when the underlying service properly returns 404 errors for unsupported pets.

---

✅ Status: Complete
👤 Requested by: @sguruvar
🔗 Workflow: View details

Powered by AI Agent

[sguruvar]

@awsapm can you create a PR to help fix this?

[github-actions]

🎯 Application observability for AWS Investigation Complete

Investigation completed successfully! Here are the results:

---

Created PR #201 to fix the agent hallucination issue. The solution addresses both the immediate problem and prevents future occurrences:

Key Fixes

• Enhanced error handling - Agents now properly detect 404 responses and acknowledge service limitations
• Prevented hallucination - Updated system prompt emphasizes accuracy over helpfulness, prohibiting generic recommendations
• Expanded database - Added 7 missing pet types (rabbit, ferret, turtle, guinea pig, parrot, bearded dragon, fish) with proper nutrition data
• Structured responses - Modified get_nutrition_data() to return error flags for proper validation

Result

Agents will now provide accurate recommendations for 13 supported pets and properly acknowledge limitations for unsupported ones, eliminating the hallucination of product recommendations when the nutrition service returns 404 errors.

---

✅ Status: Complete
👤 Requested by: @sguruvar
🔗 Workflow: View details

Powered by AI Agent