AI-agent observability and evals · Verdict-first · OTLP-native
Know which agents are getting better — and which are just getting expensive.
Morse turns labeled production outcomes, spend, and separate sampled quality signals into a ranked performance brief — with traces and evidence on demand.
Prefer to look before you commit? See the performance brief ↓
Python and TypeScript SDKs for OpenAI Agents, LangGraph, and Anthropic — or send OTLP directly from any OpenTelemetry-instrumented service, no SDK required.
Morse estimates prompt-cache savings per provider and per agent, with the estimate labeled.
Your AI bill is climbing. Are your agents getting better?
You built agents that qualify leads, summarize documents, route support tickets. They run. But are they getting cheaper, better, or quietly worse?
Model consoles show spend. Traces show executions. Eval results show sampled behavior. None, on its own, ranks which production agent needs your attention now.
Success starts with the production outcome your team maps and confirms. Without that label, a plausible response or clean trace is not proof the agent did its job.
Comparing agents means lining up those outcomes with spend and change over time, then checking sampled quality evidence without pretending the signals are one fused score.
When something moves, the supporting trace may implicate a prompt, model, tool, dependency, or infrastructure signal. That is evidence for review — not automatic proof of root cause.
The missing layer is a first-pass brief that tells you where to look before you start trace spelunking.
Performance brief first. Traces on demand.
A verdict is a ranked performance brief — not a blended cost × eval metric and not a root-cause claim. Morse keeps each evidence lane visible so you can judge what moved and what still needs review.
1. Measure production outcomes.
You map what success means. Morse reports labeled or confirmed production outcomes, CPSO, and production success trends. Applicable default scorers add sampled quality signals on eligible traces, shown separately from business success. Coverage means the share of pricing-eligible spans Morse successfully priced.
2. Prioritize what needs review.
The brief ranks agents and cohorts that deserve attention, explains which evidence drove that priority, and identifies unsupported conclusions. Failures, low sampled scores, and novel paths grow the review queue without silently becoming test cases.
3. Verify against the evidence.
Open the linked cohorts and traces to confirm the outcome mapping, inspect the change, and decide what to ship next. When revision data is present, Morse aligns the evidence to that revision; it does not claim the deploy caused the movement.
- Context attribution. Context growth can point to a likely producer span on a best-effort basis.
- Memory comparison. Memory-on versus memory-off success comparisons are directional, not causal.
- Cache estimates. Prompt-cache savings are estimated with provider-specific pricing and labeled as estimates.
Intelligence automation
Less eval setup. More first-pass review.
Morse samples eligible traces, applies relevant defaults, and prioritizes the review queue. You confirm the success mapping and decide what becomes a test.
Eligible samples get a first pass.
Applicable default scorers evaluate eligible sampled traces. Their quality signals stay separate from the production outcomes your team labels and confirms.
The review queue finds exceptions.
Failures, low sampled scores, and novel paths add candidates to a capped, deduplicated review queue. You decide which examples become durable test cases.
You define success. Morse organizes the evidence.
Your team maps the business outcome and confirms labels. Morse performs the first-pass review and ranks what deserves attention.
Revisions align when traces carry them.
When revision metadata is present, Morse aligns outcome, cost, and sampled quality evidence to that revision for comparison — without claiming causation.
When something breaks
When your agent breaks at 3am, start from a hypothesis — not 800 spans.
When an alert fires — cost spike, failure-rate jump, latency regression, quality drop — an AI agent assembles a first-pass investigation before you open the page: it samples failed traces, compares them with a baseline, checks recent changes, and correlates available infrastructure signals. You get a model-generated hypothesis, model confidence, and the supporting signals — or a Needs review result when support is thin. Typically for about a nickel per investigation.
A verification aid for the performance brief — not an asserted root cause.
Step 1/7 2:14am. Failure rate on triage-agent crosses its threshold.
A model hypothesis with supporting signals.
The investigator can compare baselines, recent changes, failed-trace samples, and correlated signals. It presents the signals that informed the hypothesis for review.
Honest about uncertainty.
If the evidence is thin or conflicting, the result is marked Needs review and the correlated data is handed back without a root-cause claim.
Your cost cap, your control.
Per-investigation cost cap (default $0.05), per-org controls, and the investigation itself is a Morse trace you can inspect — not a black-box summary.
Quality & reliability
Run agents like production infrastructure.
Quality, with real statistics.
Found a bad response in production? Save that exact trace as a test case in one click. Write your own LLM-judge scorer in the UI — no deploy required. Quality verdicts come from real statistics — confidence intervals, multiple-comparison correction — not an eyeballed score that moved.
The SRE toolkit, built for agents.
Threshold alerts cover failure rate, cost, latency, and sampled quality signals. Define error-budget SLOs and custom business metrics (“extraction accuracy above 95%”) as monitored targets alongside the production evidence.
And a copilot on every page that answers questions about your own agents in plain English — from your real traces, not a canned FAQ.
For Claude builders
Building on Claude? This is your observability layer.
Morse instruments the Anthropic SDK through a targeted client wrapper — capturing streaming, tool use, and prompt-cache tokens without routing traffic through a proxy — and the Claude Agent SDK is available in the onboarding picker.
Prompt-cache economics, visible.
Anthropic's cache pricing rewards well-structured prompts — Morse estimates cache savings with provider-specific pricing and highlights low hit rates.
Memory instrumentation that understands Claude apps.
Typed memory operations captured through a client wrap — compare their cost and latency, plus directional success rates for runs with and without memory.
Honest in the core views.
Core economics, eval, and context views distinguish reported values from estimates; unsupported metrics stay empty or inconclusive. Coverage is the share of pricing-eligible spans successfully priced.
Off the critical request path.
The SDK batches telemetry asynchronously and is designed to fail silent, so telemetry delivery errors do not interrupt the agent request. The SDK is open source, so you can inspect what it sends.
Hardened for the attacks LLM tools actually face.
Prompt-injection scanning, output classification, and canary tokens protect Morse's own Copilot and judge paths — not a scanner over your agents' traffic. Tenant isolation is covered by a dedicated test suite. Full GDPR export and deletion. Security →
Getting started
Three steps. That's it.
Get the first trace flowing in minutes. A defensible brief starts after success mapping and enough production evidence.
Sign up & get your API key
Create a free account and copy your API key from Settings → API Keys. No credit card required.
MORSE_API_KEY=mhq_xxxxxxxxxxxxxxxxxxxxInstall the SDK & add your key
pip install morse-aiimport morse_ai
# One line — reads MORSE_API_KEY from your environment
morse_ai.init()
# Cost, quality, and traces appear as your agent runsSee the verdict, then dig deeper
As labeled outcomes and eligible sampled traces arrive, CPSO, context signals, and separate quality signals populate their applicable views. Start with the ranked worklist, then open a trace when you need to inspect the evidence.
Integrations
Fits the stack you already have — on both ends.
Traces arrive the way you already emit them. Evidence and alerts leave through the tools you already use. No proprietary agent format in the middle, and nothing you have to rip out to try it.
Data in
Send it with the SDK.
Python and TypeScript SDKs cover OpenAI Agents, LangGraph, LangChain, and the Anthropic and OpenAI SDKs directly — plus the Vercel AI SDK in TypeScript. Pick the adapter at onboarding, add the documented wrapper and key, and typed agent, LLM, tool, HTTP, and database spans start flowing. Telemetry batches asynchronously; delivery failures are designed not to interrupt the agent request path.
Or send nothing new at all.
Already emitting OpenTelemetry? Skip the SDK. Point your existing OTLP HTTP exporter at Morse and you're done — any OTLP-compatible instrumentation in any language. (HTTP ingest only today — not gRPC.) Ingestion is protocol-agnostic, so the wire format is yours, not ours.
Evidence out
Ask from your editor, over MCP.
Morse serves a native MCP endpoint over JSON-RPC. Point Claude Code, Cursor, or your own agent at it with an API key and ask about costs, production success trends, sampled quality signals, and investigation hypotheses — read-only, org-scoped, without opening a browser.
Alerts where your team already is.
Configure Slack, PagerDuty, email, and in-app channels by severity so urgent and informational alerts follow different delivery paths.
Framework-agnostic isn't a tagline here. It's the ingestion design.
Pricing
Simple pricing. No surprises.
No credit card required for free tier. Cancel anytime.
Free
- ✓5 agents
- ✓75,000/mo billable spans (≈12,100 runs)
- ✓30 days retention
- ✓2 alerts
- ✓Unlimited seats
Starter
- ✓15 agents
- ✓475,000/mo billable spans (≈76,600 runs)
- ✓60 days retention
- ✓15 alerts
- ✓Unlimited seats
Pro
- ✓50 agents
- ✓1,600,000/mo billable spans (≈258,000 runs)
- ✓90 days retention
- ✓50 alerts
- ✓Unlimited seats
- ✓Team roles & workspaces
Scale
- ✓Unlimited agents
- ✓4,800,000/mo billable spans (≈774,000 runs)
- ✓180 days retention
- ✓500 alerts
- ✓Unlimited seats
- ✓Team roles & workspaces
We only bill the spans your code authored. Infra spans (HTTP, DB) get a separate 10× allowance on every tier and never count against your quota.
Plan limits are real and enforced, with a usage dashboard showing exactly where you stand — not a number someone checks later. Not sure which tier? Estimate your volume →
Limited design-partner offer
Get a founder-reviewed performance brief in 48 hours.
For a limited number of design partners, Morse's founder reviews your labeled production outcomes, CPSO, success trends, and applicable sampled quality signals — then sends a ranked brief with what to inspect next. The 48-hour clock starts after your success mapping is confirmed and sufficient telemetry is ready for review.