Shim vs Alternatives

Compare Shim to OutputFixingParser, LLM retries, custom parsing, and try/catch error handling. See real benchmarks for latency, cost, and success rates.

Solution	Latency	Cost per Repair	Success Rate
Shim	8-50ms	$0.001	99.7%
OutputFixingParser	1,500-3,000ms	$0.03	94%
Full LLM Retry	3,000-8,000ms	$0.05-$0.10	~100%
try/catch only	0ms	$0	0%
Custom Regex	1-5ms	$0	60-80%

OutputFixingParser (LangChain)

Sends broken JSON back to the LLM for fixing

Latency

1,500-3,000ms

Cost

$0.03 per repair

Success Rate

94%

Pros

Built into LangChain
Eventually consistent

Cons

200x slower than Shim
Requires additional LLM call
Unpredictable latency
Higher cost per repair
Can fail if LLM returns invalid JSON again

Migration Example

# Before
from langchain.output_parsers import OutputFixingParser

parser = OutputFixingParser.from_llm(llm, base_parser)
result = parser.parse(broken_json)  # 1,500-3,000ms

# After
from shim import ShimClient

shim = ShimClient(api_key="sk_...")
result = shim.repair.create(content=broken_json, schema=schema)  # 8-50ms
parsed = base_parser.parse(result.repaired)

Full LLM Retry

Retry the entire LLM request when JSON is invalid

Latency

3,000-8,000ms

Cost

$0.05-$0.10 per retry

Success Rate

~100% (eventually)

Pros

Guarantees valid output (eventually)
Simple to implement

Cons

500x slower than Shim
Expensive (full LLM call cost)
Poor user experience (long wait)
Can retry multiple times
Not suitable for streaming

Migration Example

// Before
let result;
let attempts = 0;
while (attempts < 3) {
  try {
    const response = await openai.chat.completions.create({...});
    result = JSON.parse(response.choices[0].message.content);
    break;
  } catch (e) {
    attempts++;
  }
}

// After
import { ShimClient } from 'shim-sdk';
const shim = new ShimClient({ apiKey: 'sk_...' });

const response = await openai.chat.completions.create({...});
const repaired = await shim.repair.create({
  content: response.choices[0].message.content,
  schema: mySchema,
  mode: 'strict'
});
result = JSON.parse(repaired.repaired);  // Always valid

try/catch with JSON.parse

Catch errors but no repair logic

Latency

0ms

Cost

Success Rate

0% (errors propagate)

Pros

Built-in to JavaScript/Python
Zero external dependencies

Cons

Provides no recovery mechanism
Breaks production when LLM outputs malformed JSON
Requires manual error handling
No streaming support
No schema validation

Migration Example

// Before
try {
  const data = JSON.parse(llmOutput);
  // Use data...
} catch (e) {
  console.error('Invalid JSON:', e);
  // Now what? ❌
}

// After
import { ShimClient } from 'shim-sdk';
const shim = new ShimClient({ apiKey: 'sk_...' });

const result = await shim.repair.create({
  content: llmOutput,
  schema: mySchema,
  mode: 'strict'
});

if (result.valid) {
  const data = JSON.parse(result.repaired);
  console.log('Confidence:', result.confidence); // 'high' | 'medium' | 'low'
  // Use data... ✅
} else {
  console.error('Repair failed:', result.errors);
}

Custom Regex Parsing

Hand-written regex to strip markdown/fix syntax

Latency

1-5ms

Cost

Success Rate

60-80% (brittle)

Pros

Fast
No external dependencies

Cons

Brittle (breaks on edge cases)
No schema validation
No confidence scoring
Requires extensive testing
Hard to maintain
Misses complex syntax errors

Migration Example

// Before
function fixJSON(broken) {
  let fixed = broken.replace(/```json\n|\n```/g, '');
  fixed = fixed.replace(/,\s*([\]}])/g, '$1');  // Trailing commas
  fixed = fixed.replace(/([{,])\s*([a-zA-Z_][a-zA-Z0-9_]*)\s*:/g, '$1"$2":'); // Unquoted keys
  return fixed;  // Hope for the best 🤞
}

// After
import { ShimClient } from 'shim-sdk';
const shim = new ShimClient({ apiKey: 'sk_...' });

const result = await shim.repair.create({
  content: broken,
  schema: mySchema,
  mode: 'strict'
});

console.log(result.confidence); // Know if repair is safe
console.log(result.warnings);   // See what was fixed

Ready to switch?

Get 1,000 free repairs/month. No credit card required. Migrate in under 5 minutes.