Tomosu runs a small set of agents across the places where reliability is won or lost: the editor, the pull request, the running system and the post-incident review. They read the same evidence an experienced engineer would, score it on one scale, propose the fix, and feed what production teaches back into the next check. This page walks through that loop end to end.
Files are scored as they are written and every pull request is scored before review. Findings arrive with a proposed fix and a projected PRI change.
A change that would lower the PRI below your policy does not merge quietly. The check explains which dimension moved and what would restore it.
Runtime signals are matched to known reliability patterns and to your own incident history. Routine incidents are resolved automatically with an evidence trail; the rest are escalated with context attached.
Each escalation becomes a new check upstream, in the editor and on the PR, so the same class of failure is caught before it reaches production again.
Pre-merge is where most reliability debt is created and where it is cheapest to remove. The agents work in the two places developers already are: the editor and the pull request.
Every finding the agents raise is delivered as a recommendation: a self-contained unit you can judge, accept or decline without opening another tool. The same recommendation shows up in the editor, on the pull request, on the dashboard and in Claude through the MCP server.
Why it matters. The call on line 41 of payment_client.py has no timeout. Under provider latency it holds a worker per request until the pool is exhausted. The same pattern caused incident #92 on this service in March.
Error handling, backoff, resource limits, concurrency guards and the shortcuts AI assistants tend to leave behind.
Vulnerable or abandoned packages, infrastructure-as-code that removes redundancy or alerts, missing pod disruption budgets and the like.
Missing metrics or traces on critical paths, and for agentic code: step budgets, tool-result checks and prompt-injection containment.
The Production Reliability Index (PRI) is a 0 to 100 score for a repository or a service. It is built from seven sub-indexes that each measure one aspect of reliability, so a single number is still explainable: you can always see which dimension moved and why.
How likely this code is to break, based on structural signals and real production behavior.
The gap between what development expected and what production delivered.
Severity-weighted adherence to your coding, security and observability standards.
Live production health: error rate, latency and resource anomalies, aggregated per service.
Churn and hotspot density. Files that keep re-triggering the same issues are penalized.
Frequency and safety of releases. Small, confident batches are rewarded.
The interrupt tax on engineering: repeat ticket rate, tier-level time to resolve, senior on-call load.
Added to the roll-up only for codebases that run AI agents. Explained further down this page.
| Sub-index | Score |
|---|---|
| FI Fragility | 62 |
| DI Drift | 70 |
| GC Governance | 74 |
| RS Runtime Signals | 58 |
| CV Code Volatility | 66 |
| DV Deployment Velocity | 71 |
| EEI Escalation | 64 |
| AR Agent Reliability | no data |
| PRI roll-up | 66 |
Each sub-index is scored on the same 0 to 100 scale from the evidence available for it, then rolled up into the PRI. The weighting is calibrated per organization to reflect its architecture, maturity and risk tolerance, so two teams with the same findings can reasonably land on different scores. The trend over time matters more than any single reading.
Accepting a proposed fix raises the dimension it belongs to. Merging a regression lowers it. Runtime signals, resolved incidents and escalations feed the production-side dimensions as they happen. A dimension with no real signal yet, for example Runtime Signals before any telemetry is connected, is left out of the roll-up rather than guessed, so a first scan is honest about what it can and cannot see.
Most repositories start in the Fragile band, typically between 26 and 34, because timeouts, retry budgets, resource limits and observability are rarely complete. Accepting the recommendations from that first scan usually brings the same repository to 65 to 74 before anything new is merged. From there the score moves with every change and every production signal.
A reliability score is only useful if it is current. Tomosu re-scores at every point where reality changes, so the number on the dashboard is the number for the code that is running now.
| Week | PRI | Event |
|---|---|---|
| 1 | 31 | first scan |
| 2 | 64 | fixes accepted |
| 3 | 70 | |
| 4 | 73 | |
| 5 | 64 | deploy regression |
| 6 | 76 | resolved by agent |
| 7 | 81 | |
| 8 | 84 | guardrail holds |
Every recalculation keeps its inputs, so any score on the dashboard can be traced back to the findings, signals and changes that produced it.
A finding is only actionable when you know what it touches. The Production Context Graph is the model the agents build and maintain so that a line of code, the service it runs in, the dependency it calls and the incident it caused are one connected picture.
More systems now contain AI agents of their own: tool-calling loops, retrieval pipelines, multi-agent workflows. They fail in ways classic services do not. When a scan detects an agentic codebase, Tomosu adds an Agent Reliability sub-index to the PRI and the agents review these failure classes alongside the rest.
For codebases without agents the sub-index simply has no data and is left out of the roll-up.
In production the agents watch the signals your observability stack already collects, match them to the graph and to known reliability patterns, and act according to a tiered policy you control.
Known patterns with a known safe remedy, for example a saturated connection pool after a deploy. The agent applies the remedy through a canary, verifies the signal recovers and records what it did.
The pattern is recognized but the remedy touches code or configuration. The agent prepares the change, applies it within policy, and leaves the evidence trail for review. The senior engineer is informed, not paged.
No matching pattern or a remedy outside policy. The agent escalates with the causing change, the affected path in the graph and the signals attached, so the engineer starts from the answer rather than the alert.
Every resolution and escalation updates the Escalation Index and feeds the learning loop below.
Generic rule sets check for failures someone else had. Tomosu gets stricter where you actually fail.
When an incident is resolved, the agent records the pattern that caused it and the fix that held. That pattern becomes a check in the editor and on every pull request across the repositories that share the pattern, with the original incident linked as the reason.
A guardrail is only worth keeping if the failure stops recurring. Recurrence is tracked per rule, and the PRI reflects it: repositories protected by rules that hold trend up; a rule that does not prevent recurrence is surfaced for review.
Over weeks the first-scan picture changes. Fixes accepted pre-merge stop the common failures, production resolutions handle the rest, and the learned guardrails mean the same class of problem is not paid for twice.
Repository and observability connections start read-only. Writing a fix, holding a merge or applying a runtime remedy is enabled per repository and per tier, by you.
Fixes are proposals until accepted. Merge gates are your policy. Escalations go to a person with the evidence attached. Nothing bypasses review that you have not explicitly automated.
Findings, scores, fixes, resolutions and escalations keep their inputs, so any verdict can be audited later. Your code is never used to train models.
Read-only, no credit card, a PRI score and the ranked gaps in under five minutes. Add the editor, the pull request check and the runtime connection when you want the whole loop.
