How to Prioritize Risky Pull Requests

Not every pull request needs the same amount of engineering attention.

A documentation change and a change to a shared production dependency may both go through the same pull request workflow, but the evidence needed before merging them can be very different.

The challenge is deciding where additional review, testing, or investigation is actually useful. A practical approach is to prioritize pull requests using several signals together:

• Change scope
• Production impact
• Dependencies
• Testing
• Incident history
• Runtime behavior
• Blast radius
• Deployment conditions

The goal is not to create a perfect ranking of every pull request. It is to help engineers spend more attention where a change has more potential production impact or more uncertainty.

What does it mean to prioritize risky pull requests?

Prioritizing risky pull requests means identifying changes that may require more engineering attention before they are merged or deployed. That attention could mean:

• Additional code review
• More testing
• Dependency investigation
• A smaller change
• A feature flag
• A gradual rollout
• Additional monitoring
• A deployment plan
• A discussion with the relevant service owner

A pull request does not need to be labeled “bad” to deserve additional attention. It is a question of production reliability rather than correctness alone. The useful question is:

What evidence do we have, and what does it suggest we should do next?

For the previous step in this workflow, see How to Identify High Risk Pull Requests.

Start with production impact

The first question should be:

What could this change affect if it behaves unexpectedly?

Look at:

• Customer facing services
• Critical workflows
• Shared infrastructure
• Shared libraries
• Databases
• Authentication
• Payments
• Core APIs
• Internal systems

A change to an isolated internal tool may need less additional investigation than a change to a shared component used throughout the production environment.

This does not mean one type of change should always be merged or blocked. It means the production context should influence how much evidence is needed.

1. Look at change scope

Start with the pull request itself. Consider:

• Files changed
• Components changed
• APIs modified
• Database changes
• Configuration changes
• Infrastructure changes
• Shared code

Diff size can provide context, but it should not be the primary way to prioritize pull requests.

A small change can have broad consequences if it touches a shared dependency. A large change can have limited production impact if it is isolated.

2. Map dependencies

Next, identify what depends on the changed component.

The pull request may be small. The potential impact is not.

Dependency relationships are therefore one of the most useful signals when deciding which changes deserve additional attention. For a deeper look, see How to Assess the Blast Radius of a Code Change.

3. Consider production usage

Dependency count alone does not tell the entire story. Consider how those dependencies are actually used. Useful context includes:

• Request volume
• Number of production consumers
• Customer facing usage
• Business critical workflows
• Internal versus external users
• Geographic scope

A shared component used by several low traffic internal tools can have a different production impact from one used by the main customer API. The architecture and actual usage matter.

4. Look at testing evidence

Testing provides evidence about how much of the changed behavior has been verified. Consider:

• Unit tests
• Integration tests
• End to end tests
• Regression tests
• Load tests
• Staging verification
• Manual testing

A useful question is:

What important behavior remains unverified?

For example, a change to a shared API might have excellent unit test coverage while having limited integration coverage with its production consumers.

That does not automatically make the pull request unsafe. It identifies an area where additional evidence may be useful.

5. Check incident history

Past incidents can provide valuable context. Ask:

• Has this component caused incidents before?
• Have similar changes caused problems?
• Has the service required rollback recently?
• Are there recurring failure patterns?
• Has the affected dependency been unstable?

Previous incidents do not prove that the current change will cause another incident. They provide historical evidence that can help determine how much additional investigation is appropriate.

6. Look at current runtime behavior

Current production behavior can also influence prioritization. Check relevant:

• Error rates
• Latency
• Resource usage
• Dependency failures
• Alerts
• Runtime anomalies

A pull request touching a service that is already experiencing reliability problems may deserve additional investigation. A stable service with the same code change may provide a different context.

This is where observability data can become useful for change analysis. See Production Reliability vs Observability.

7. Consider recent change activity

Change frequency can provide another signal. Look at:

• Recent commits
• Recent deployments
• Related pull requests
• Recent refactoring
• Multiple concurrent changes
• Changes to related dependencies

A component undergoing several changes at the same time can be harder to reason about. Again, this is context rather than a universal rule.

8. Assess the blast radius

The blast radius describes what could be affected if the change behaves unexpectedly. Map:

Then ask:

• How many services are affected?
• Are customer facing systems involved?
• Are critical workflows involved?
• Can the impact be isolated?
• Can the change be rolled back?

See How to Assess the Blast Radius of a Code Change for the detailed process.

9. Consider deployment conditions

Two identical changes can have different operational exposure depending on how they are deployed. Consider:

• Feature flags
• Canary releases
• Gradual rollout
• Regional rollout
• Automated rollback
• Manual rollback
• Monitoring during deployment

For example, a change with broad potential impact may have stronger controls when released gradually than when deployed everywhere simultaneously. Deployment conditions should therefore be part of prioritization.

A practical prioritization framework

Instead of asking whether a pull request is simply “risky” or “safe,” consider four questions:

This creates a more useful model:

The result does not need to be a single universal score. It can simply determine what the team should investigate next.

Example: two pull requests

Consider two hypothetical changes.

Looking only at lines changed would put B first. Looking at production context produces a different picture.

This is why prioritization should use multiple signals rather than diff size alone.

What should happen after identifying a risky pull request?

The next step should depend on why the pull request requires attention.

The purpose of prioritization is to direct engineering effort toward the right action.

Should teams assign every pull request a risk score?

A score can be useful when it summarizes evidence that engineers would otherwise have to gather manually. But a number should not replace the evidence behind it. For example:

The explanation is often more useful than the number itself. Teams should be able to understand why a change received additional attention.

Production Reliability Index

Tomosu uses the Production Reliability Index (PRI) to bring multiple reliability signals into a broader view of changes and production systems. The platform describes PRI as a master score built from seven sub indices:

• Fragility Index
• Drift Index
• Governance Compliance
• Runtime Signals
• Code Volatility
• Deployment Velocity
• Escalation Index

These signals are calibrated to the organization and used across development, pre merge, and production.

The purpose is to bring multiple pieces of reliability context together rather than relying on a single signal such as diff size, test status, or dependency count.

Try the Production Reliability Index →

A practical pull request prioritization checklist

Before deciding how much attention a pull request needs, ask:

The main idea

Pull request prioritization should not be about finding a magic number. It should help answer:

Which changes need more attention before they reach production, and why?

A useful process connects change, dependencies, production usage, testing, history, runtime behavior, blast radius, and deployment controls into one view of the required attention. That gives engineers a way to spend more time where the available evidence suggests more investigation is needed.

For the broader workflow, continue with How to Assess Production Reliability Before Deployment. For how to turn these signals into measurements, see How to Measure Software Change Reliability, and for the full pre merge workflow see Pre Merge Reliability Analysis.

Frequently asked questions

Pull request prioritization isn't about finding a magic number. It's about knowing which changes need more attention before they reach production, and why. The Production Reliability Index brings those signals together so the reasoning stays visible. Try the Production Reliability Index →