Two open-source projects, one 3-line cleanup bug, one 8-year-old memory leak, and what happens when you scan before users do.
Most reliability bugs are not exotic. They are ordinary code that works on a laptop and fails politely at scale. The usual way to find them is to ship, wait, get paged, and write a postmortem. Tomosu is built around a different loop: scan the code (and every PR) for the missing reliability rule before anyone is on call.
That is a claim, so here is the evidence. Tomosu published two case studies where they rewound open-source repos to the code as it stood before the fix, handed Tomosu a single file with no bug report, and checked whether it named the real defect.
Both bugs were readable in the source long before production exposed them. In two blind scans, with the bug report and the fix withheld, Tomosu named the real defect from a single pre-fix file.
- IBM ContextForge: a
finallyblock wheredb.close()ran only ifdb.commit()succeeded. PRI 27 out of 100, rated Critical. - Celery: a dict of unresolved child tasks that nothing ever removed from. PRI 30 out of 100, rated Critical.
- The common rule: if you take a finite production resource, or keep long-lived state, the release and the bound must survive every failure path.
Case 1: IBM ContextForge and the finally that was not
ContextForge is an open-source AI gateway for MCP, A2A, REST and gRPC. Every request opens a database session from a finite PostgreSQL pool. In a January 2026 load test with 3,000 concurrent users, between 42 and 81 connections sat “idle in transaction”, each held for up to 120 seconds, until the pool ran dry.
The bug, in mcpgateway/middleware/token_scoping.py:
# before (issue #1885)
finally:
db.commit() # can raise
db.close() # never reached if commit raises# after (PR #1886)
finally:
try:
db.commit()
finally:
db.close() # always runsReads like a punchline: a finally block that is not guaranteed to finish. It is not a stupid mistake either; the intent is obviously right and a reviewer skimming the diff would nod it through. After the fix, under the same load, stuck connections went to zero.
The blind scan
Tomosu checked out commit cf5adcd5d, gave the scanner that one 755-line file, and nothing else. Result: PRI 27/100, Critical, with a finding describing an unsafe session lifecycle where an operation before close() can fail and leave the connection unreleased. The rule underneath, in Tomosu’s words: if you acquire a finite production resource, its release must survive every relevant failure path. Read the full ContextForge case study.
Case 2: the Celery leak that took eight years
In June 2018 a user reported that Celery’s event monitor “unboundedly leaks memory” (issue #4832). The culprit was a plain dict, State._tasks_to_resolve, which remembers child tasks whose parents were never seen. Nothing ever removed those entries. The fix, merged August 31, 2026 (PR #10534), was one line: swap the dict for an LRUCache(max_tasks_in_memory).
Blind scan of the pre-fix file (730 lines, one file): PRI 30/100, Critical, 7 findings. Finding #7 said events referencing never-seen parents “can accumulate indefinitely” and recommended bounding the index with max_tasks_in_memory. Mechanism, consequence and invariant, all named. Same idea as the eventual fix, different implementation. Read the full Celery case study.
The reusable lesson fits on a sticky note. For every long-lived collection ask: what inserts, what removes, is the removal guaranteed on failure, and what is the hard bound? If the last two answers are “no” and “none”, production owns the risk.
What it looks like inside one repo: five days
Tomosu’s own walkthrough follows a single repo through its first week. Numbers are from their published demo.
The honest takeaway is the dip. Clearing every finding got the repo to 82, and production traffic still knocked it to 63. A scan finds what the code says; production finds what the traffic does. Tomosu’s pitch is that the same system reads both, and the Day 5 fix landed as a one-file PR.
- Cleanup that can be skipped
- State with no bound
- Missing retries, timeouts
- Weak failure paths
Cost: a PR comment
- Pool exhaustion at 3,000 users
- Retry storms (68.3% retries)
- Drift after a traffic spike
- Slow leaks, found years later
Cost: an incident
Fig 6. Same bugs, different price tag.
Try it on your own repo
Suggested first experiment: scan one service you already know is fragile and see whether the top finding matches your scar tissue. If it does, that is your use case. If it does not, you have lost five minutes.
What to take with a grain of salt
- Both case studies are written by Tomosu, who chose the bugs. The criteria were locked before scanning, but two hits is not a benchmark.
- Scores are per file; the PRI is Tomosu’s own index, not an industry standard.
- Tomosu’s outcome figures (68% fewer engineering tickets, 50% fewer post-deployment failures, 73% fewer support escalations) are labelled “estimated upper bound” for mid-size SaaS teams. Treat as a ceiling, not a forecast.
- The PRI arc and customer numbers come from Tomosu’s own demos.
Free, no card. The scan is read-only and returns a PRI with findings by category.
A load test tells you the system broke at 3,000 users. A reliability scan tells you which three lines were going to break it. Scan a repo free →
