Concept: Reasoning & Problem-Solving Agents
Overview
This example demonstrates how to configure an LLM as a reasoning agent capable of analytical thinking and quantitative problem-solving. It shows the bridge between simple text generation and complex cognitive tasks.
What is a Reasoning Agent?
A reasoning agent is an LLM configured to perform logical analysis, mathematical computation, and multi-step problem-solving through careful system prompt design.
Human Analogy
Regular Chat Reasoning Agent
───────────── ──────────────────
"Can you help me?" "I am a mathematician.
"Sure! What do you need?" I analyze problems methodically
and compute exact answers."
The Reasoning Challenge
Why Reasoning is Hard for LLMs
LLMs are trained on text prediction, not explicit reasoning:
┌───────────────────────────────────────┐
│ LLM Training │
│ "Predict next word in text" │
│ │
│ NOT explicitly trained for: │
│ • Step-by-step logic │
│ • Arithmetic computation │
│ • Tracking multiple variables │
│ • Systematic problem decomposition │
└───────────────────────────────────────┘
However, they can learn reasoning patterns from training data and be guided by system prompts.
Reasoning Through System Prompts
Configuration Pattern
┌─────────────────────────────────────────┐
│ System Prompt Components │
├─────────────────────────────────────────┤
│ 1. Role: "Expert reasoner" │
│ 2. Task: "Analyze and solve problems" │
│ 3. Method: "Compute exact answers" │
│ 4. Output: "Single numeric value" │
└─────────────────────────────────────────┘
↓
Reasoning Behavior
Types of Reasoning Tasks
Quantitative Reasoning (this example):
Problem → Count entities → Calculate → Convert units → Answer
Logical Reasoning:
Premises → Apply rules → Deduce conclusions → Answer
Analytical Reasoning:
Data → Identify patterns → Form hypothesis → Conclude
How LLMs “Reason”
Pattern Matching vs. True Reasoning
LLMs don’t reason like humans, but they can:
┌─────────────────────────────────────────────┐
│ What LLMs Actually Do │
│ │
│ 1. Pattern Recognition │
│ "This looks like a counting problem" │
│ │
│ 2. Template Application │
│ "Similar problems follow this pattern" │
│ │
│ 3. Statistical Inference │
│ "These numbers likely combine this way" │
│ │
│ 4. Learned Procedures │
│ "I've seen this type of calculation" │
└─────────────────────────────────────────────┘
The Reasoning Process
Input: Complex Word Problem
↓
┌────────────┐
│ Parse │ Identify entities and relationships
└────────────┘
↓
┌────────────┐
│ Decompose │ Break into sub-problems
└────────────┘
↓
┌────────────┐
│ Calculate │ Apply arithmetic operations
└────────────┘
↓
┌────────────┐
│ Synthesize│ Combine results
└────────────┘
↓
Final Answer
Problem Complexity Hierarchy
Levels of Reasoning Difficulty
Easy Hard
│ │
│ Simple Multi-step Nested Implicit │
│ Arithmetic Logic Conditions Reasoning│
│ │
└─────────────────────────────────────────────┘
Examples:
Easy: "What is 5 + 3?"
Medium: "If 3 apples cost $2 each, what's the total?"
Hard: "Count family members with complex relationships"
This Example’s Complexity
The potato problem is highly complex:
┌─────────────────────────────────────────┐
│ Complexity Factors │
├─────────────────────────────────────────┤
│ ✓ Multiple entities (15+ people) │
│ ✓ Relationship reasoning (family tree)│
│ ✓ Conditional logic (if married then..)│
│ ✓ Negative conditions (deceased people)│
│ ✓ Special cases (dietary restrictions)│
│ ✓ Multiple calculations │
│ ✓ Unit conversions │
└─────────────────────────────────────────┘
Limitations of Pure LLM Reasoning
Why This Approach Has Issues
┌────────────────────────────────────┐
│ Problem: No External Tools │
│ │
│ LLM must hold everything in │
│ "mental" context: │
│ • All entity counts │
│ • Intermediate calculations │
│ • Conversion factors │
│ • Final arithmetic │
│ │
│ Result: Prone to errors │
└────────────────────────────────────┘
Common Failure Modes
1. Counting Errors:
Problem: "Count 15 people with complex relationships"
LLM: "14" or "16" (off by one)
2. Arithmetic Mistakes:
Problem: "13 adults × 1.5 + 3 kids × 0.5"
LLM: May get intermediate steps wrong
3. Lost Context:
Problem: Multi-step with many facts
LLM: Forgets earlier information
Improving Reasoning: Evolution Path
Level 1: Pure Prompting (This Example)
User → LLM → Answer
↑
System Prompt
Limitations:
- All reasoning internal to LLM
- No verification
- No tools
- Hidden process
Level 2: Chain-of-Thought
User → LLM → Show Work → Answer
↑
"Explain your reasoning"
Improvements:
- Visible reasoning steps
- Can catch some errors
- Still no tools
Level 3: Tool-Augmented (simple-agent)
User → LLM ⟷ Tools → Answer
↑ (Calculator)
System Prompt
Improvements:
- External computation
- Reduced errors
- Verifiable steps
Level 4: ReAct Pattern (react-agent)
User → LLM → Think → Act → Observe
↑ ↓ ↓ ↓
System Reason Tool Result
Prompt Use
↑ ↓ ↓
└───────────Iterate──┘
Best approach:
- Explicit reasoning loop
- Tool use at each step
- Self-correction possible
System Prompt Design for Reasoning
Key Elements
1. Role Definition:
"You are an expert logical and quantitative reasoner"
Sets the mental framework.
2. Task Specification:
"Analyze real-world word problems involving..."
Defines the problem domain.
3. Output Format:
"Return the correct final number as a single value"
Controls response structure.
Design Patterns
Pattern A: Direct Answer (This Example)
Prompt: [Problem]
Output: [Number]
Pros: Concise, fast Cons: No insight into reasoning
Pattern B: Show Work
Prompt: [Problem] "Show your steps"
Output: Step 1: ... Step 2: ... Answer: [Number]
Pros: Transparent, debuggable Cons: Longer, may still have errors
Pattern C: Self-Verification
Prompt: [Problem] "Solve, then verify"
Output: Solution + Verification + Final Answer
Pros: More reliable Cons: Slower, uses more tokens
Real-World Applications
Use Cases for Reasoning Agents
1. Data Analysis:
Input: Dataset summary
Task: Compute statistics, identify trends
Output: Numerical insights
2. Planning:
Input: Goal + constraints
Task: Reason about optimal sequence
Output: Action plan
3. Decision Support:
Input: Options + criteria
Task: Evaluate and compare
Output: Recommended choice
4. Problem Solving:
Input: Complex scenario
Task: Break down and solve
Output: Solution
Comparison: Different Agent Types
Reasoning Tools Memory Multi-turn
───────── ───── ────── ──────────
intro.js ✗ ✗ ✗ ✗
translation.js ~ ✗ ✗ ✗
think.js (here) ✓ ✗ ✗ ✗
simple-agent.js ✓ ✓ ✗ ~
memory-agent.js ✓ ✓ ✓ ✓
react-agent.js ✓✓ ✓ ~ ✓
Legend:
- ✗ = Not present
- ~ = Limited/implicit
- ✓ = Present
- ✓✓ = Advanced/explicit
Key Takeaways
- System prompts enable reasoning: Proper configuration transforms an LLM into a reasoning agent
- Limitations exist: Pure LLM reasoning is prone to errors on complex problems
- Tools help: External computation (calculators, etc.) improves accuracy
- Iteration matters: Multi-step reasoning patterns (like ReAct) work better
- Transparency is valuable: Seeing the reasoning process helps debug and verify
Next Steps
After understanding basic reasoning:
- Add tools: Let the agent use calculators, databases, APIs
- Implement verification: Check answers, retry on errors
- Use chain-of-thought: Make reasoning explicit
- Apply ReAct pattern: Combine reasoning and tool use systematically
This example is the foundation for more sophisticated agent architectures that combine reasoning with external capabilities.