LLM Inference Hosting Decision Tool

AI inference is the fourth infrastructure pillar — as fundamental as compute, storage, and networking — and on AWS the cost of getting it wrong is real: Bedrock on-demand pricing spans a 143× range, and most enterprises underestimate inference cost by 3× in year one.

This tool turns the LLM Token Economics decision framework into an interactive guide. Tell it about your workload — model, region, monthly token volume, latency needs, traffic pattern, and team readiness — and it walks you through the same decision tree the paper lays out, then quantifies the trade-off with live AWS pricing. It answers one question: for your workload, should you run on Amazon Bedrock, self-host on EC2 GPUs, or go hybrid — and roughly where your break-even sits.

Everything runs in your browser. Bedrock token prices come straight from AWS’s public, CORS-enabled Price List API (refreshable live with one click); per-region model availability and EC2 GPU pricing are captured from the AWS API at build time. No data about your workload ever leaves the page.

AWS · Token Economics

Bedrock, Self-Host, or Hybrid?

A guided decision tree backed by live AWS pricing. Answer a few questions about your workload and see the recommended hosting strategy and where your cost break-even lands.

—Bedrock models
—region
143×price spread
Loading AWS pricing snapshot…

Your workload

Model availability and pricing vary by region.
M tokens / month
Your naive estimate of input + output tokens per month. Amplify it below for realistic demand. Decision thresholds anchor to this model's real break-even.
50% output / 50% input. Output tokens often cost more.
0% — verbose prompts, redundant retrieval, missing caches, wrong-sized models. The paper's estimate: 40–60% is recoverable.
Proprietary models have no self-host option on AWS — Bedrock only.
Network round-trip to Bedrock adds 50–200 ms per call.
50% — below 50% favours managed Bedrock.
Expected demand (production is often 10–20× the prototype)
1× — tool calls, retries & sub-agents multiply calls per task (single task → up to 100×).
+0× — retrieved docs add 3–10× input tokens at naive injection.
+0× — a 200-token query can emit 2,000–20,000 thinking tokens.
+0% — re-sent on every call; grows non-linearly in multi-turn chats.
Advanced cost assumptions
65% — capacity headroom: the fleet is sized so demand lands at this share of peak throughput. The single biggest self-host cost lever.
0% — prompt-cache hit ratio (Bedrock 90% off cached reads).
0% — async work that tolerates a 24h delay (Bedrock 50% off).
Estimated discount band vs on-demand (steady/batch patterns; bursty keeps elasticity). Applies to the Optimised tier.
0% — route simple queries to cheaper models (paper: 40–70% achievable).
Loaded cost ÷ 12. Geographic variability is huge — see the paper's disclaimer.
Halving weight bytes halves VRAM and roughly doubles decode tok/s (bandwidth-bound).
KV cache grows linearly with context — long contexts can dwarf the weights.
Concurrent sequences per replica (continuous batching). More batch = more tok/s but more KV VRAM.
Floor only — the fleet grows automatically when volume needs more.

Decision tree

The full framework from the paper. Your answers light up the path down to the recommendation below.
Decision Bedrock Self-Host Hybrid
—
Configure your workload
Adjust the inputs above and the recommendation will update live.

Cost crossover

Monthly cost vs. token volume (log scale). Dots mark where self-host overtakes each Bedrock tier; the blue line is your current volume.
Bedrock On-Demand Bedrock Optimised Self-Host Your volume