Skip to main content
When AI isn’t working as expected, the problem usually falls into a few predictable categories. This guide helps you diagnose and fix common issues quickly.

​
Inconsistent Results

​
Problem: Different Output Each Time

You run the same prompt on the same document but get different results. Common Causes:
  • Prompt is too vague or open-ended
  • Missing essential context
  • Relying on AI to infer rather than instruct
Solutions: 
Add specific constraints: "Focus only on payment terms"
Include clear context: "We are the vendor with low leverage"
Define output format: "List exactly 3 issues in order of importance"
Test: Run the improved prompt 3 times. Results should be nearly identical.

​
Problem: Quality Varies Wildly

Sometimes the AI nails it, sometimes it’s completely off. Common Causes:
  • Prompt works for some document types but not others
  • Context requirements vary by situation
  • Model updates affecting performance
Solutions:
  • Create document-specific prompts rather than universal ones
  • Add conditional logic: “If customer is government entity, then…”
  • Test prompts regularly and update as needed

​
Wrong Focus

​
Problem: AI Focuses on Wrong Issues

The AI obsesses over formatting while missing major liability issues. Common Causes:
  • No priority guidance provided
  • Equal weight given to all issues
  • Missing risk framework
Solutions: 
"Ignore formatting and stylistic issues"
"Focus only on provisions creating financial risk over $10,000"
"Prioritize: 1) Liability 2) Payment 3) IP rights - ignore everything else"

​
Problem: Too Much or Too Little Detail

Getting novels when you need summaries, or single sentences when you need analysis. Common Causes:
  • No length specification
  • Audience unclear
  • Purpose undefined
Solutions: 
"One sentence per issue"
"Detailed analysis with specific clause references"
"Executive summary: 3 bullet points maximum"

​
Context Problems

​
Problem: AI Doesn’t Understand the Situation

The AI gives generic advice that doesn’t fit your specific circumstances. Common Causes:
  • Insufficient context provided
  • Wrong context emphasized
  • Contradictory information given
Solutions: Build context systematically: 
1. Who you are: "Small startup vendor"
2. Document type: "Enterprise MSA"
3. Leverage: "Very low - competitive RFP"
4. Constraints: "Cannot accept uncapped liability per insurance"

​
Problem: AI Ignores Critical Context

You mentioned you’re in healthcare but the AI ignores HIPAA requirements. Common Causes:
  • Context buried in long prompt
  • Competing contexts confusing priority
  • Key context mentioned too late
Solutions:
  • Lead with critical context
  • Repeat important constraints
  • Use headers to organize context clearly

​
Output Format Issues

​
Problem: Wrong Format for Needs

Getting prose when you need a table, or bullets when you need an email. Common Causes:
  • Format not specified
  • Conflicting format instructions
  • AI defaulting to its preferences
Solutions: Be explicit about format: 
"Create a table with these exact columns: Issue | Risk | Action"
"Draft as email suitable for sending to opposing counsel"
"Bullet points only - no prose paragraphs"

​
Problem: Information Scattered

Important points buried throughout long responses. Common Causes:
  • No structure requested
  • Multiple asks in one prompt
  • AI trying to be comprehensive
Solutions: 
"Organize by: 1) Critical issues 2) Medium priority 3) Minor points"
"Start with your recommendation, then explain"
"Group all payment-related issues together"

​
Accuracy Issues

​
Problem: AI Misses Important Provisions

Critical clauses aren’t being caught by the AI. Common Causes:
  • Rule/prompt too narrow
  • Keywords not matching document language
  • Important sections in unexpected places
Solutions:
  • Broaden search terms: “audit, inspection, review, examination”
  • Check entire document: “Review all sections including exhibits”
  • Be explicit: “Check for liability caps anywhere in document”

​
Problem: False Positives

AI flags issues that aren’t really problems. Common Causes:
  • Prompt too broad
  • Missing negative prompting
  • No threshold for materiality
Solutions: 
"Only flag if liability exceeds $100,000"
"Ignore standard industry terms"
"Do not flag: payment terms, notice provisions, standard definitions"

​
Performance Issues

​
Problem: AI Takes Forever

Response times are unusually long. Common Causes:
  • Overly complex prompts
  • Multiple large documents
  • Requesting too much in one prompt
Solutions:
  • Break into smaller requests
  • Process documents separately
  • Simplify prompt structure

​
Problem: AI Cuts Off Mid-Response

Responses end abruptly without completing the task. Common Causes:
  • Output length limits reached
  • Token limits exceeded
  • Complex chains causing timeouts
Solutions:
  • Request more concise output
  • Break into multiple prompts
  • Ask for summary first, details separately

​
Model-Specific Issues

​
Problem: Prompt Works in ChatGPT but Not Claude

Different AI models behave differently. Common Causes:
  • Models trained differently
  • Format preferences vary
  • Instruction interpretation differs
Solutions:
  • Adjust formatting for specific model
  • Test prompts on target platform
  • Use model-appropriate structure (hashtags for GPT, XML for Claude)

​
Problem: Quality Degraded After Update

Previously good prompts now failing. Common Causes:
  • Model improvements changing behavior
  • Prompt exploiting previous quirks
  • Overfitting to old model version
Solutions:
  • Simplify and clarify prompts
  • Remove workarounds for old issues
  • Test and update prompt library

​
Quick Diagnostic Checklist

When something’s not working, check:
  1. Context: Is it clear who you are and what you need?
  2. Specificity: Are instructions precise or vague?
  3. Format: Did you specify how you want output?
  4. Scope: Is the AI looking at the right things?
  5. Constraints: Are there clear boundaries?
  6. Priority: Does the AI know what matters most?

​
Emergency Fixes

  1. ​
    When Nothing Works

Start completely fresh with the simplest possible version: 
"I'm the vendor. Find the three biggest risks in this agreement."
Then gradually add complexity back.

  1. ​
    When Pressed for Time

Use pre-built templates or playbooks rather than custom prompts. Even if not perfect, they’re tested and reliable.

  1. ​
    When Results Are Dangerous

If AI suggests something that seems legally problematic, always verify against source material and apply professional judgment.

​
Prevention Strategies

  • Test Early and Often Don’t wait until the big deal to discover your prompt doesn’t work.
  • Document What Works When you find a good prompt, save it immediately with notes on why it worked.
  • Build Gradually Start simple and add complexity only as needed.
  • Stay Updated AI models evolve. What worked last month might need adjustment today.

​
Remember

Most AI problems stem from communication issues, not AI limitations. The AI can’t read your mind – it only knows what you tell it. When troubleshooting, start by examining your prompt before blaming the model. Clear instructions beat clever prompts every time.