Key takeaways
- Use three or four severity levels and define each one by its effect on the change.
- State which levels block a merge and which can be deferred to a follow-up ticket.
- Anchor every level to evidence a reviewer can point to, not to personal preference.
- Separate severity from tone so a blocking comment reads as a decision, not a judgment.
- Audit a sample of reviews each quarter to confirm the scale is applied consistently.
Choose a small set of levels
Three levels are enough for most pods: blocking, should-fix, and optional. Blocking means the change cannot merge until the issue is resolved. Should-fix means the change may merge but the issue needs a ticket and an owner. Optional covers style and preference that the author may decline with a reason.
A four-level scale can work when the product has regulated or safety-critical behavior. The extra level separates a correctness defect from a security or data-integrity risk that needs a named approver. Adding more levels rarely adds precision. Reviewers stop distinguishing them, and authors start treating every level as blocking.
Define blocking versus non-blocking
Write the blocking rule in the review guide, not in a comment thread. A blocking issue changes behavior in a way that is wrong, unsafe, or unverifiable. Examples include a missing authorization check, a migration without a rollback, or a test that cannot fail. The rule should be short enough that a reviewer can apply it without a meeting.
Non-blocking issues still need a destination. A should-fix comment becomes a follow-up ticket with an owner and a target sprint. An optional comment may be accepted or declined in the thread. If the team never creates the follow-up ticket, the scale becomes a way to postpone work indefinitely. Track the tickets so the deferral is visible.
Anchor each level to evidence
A severity label is only useful when it points at something observable. Ask the reviewer to cite the line, the behavior, and the evidence that the behavior is wrong. A comment that says "risky" without a reproduction or a contract is a preference dressed as a finding. The scale should require the same evidence at every level.
Give reviewers a short prompt in the review template: what happens, who is affected, and how the author can verify the fix. This prompt does most of the work. It slows a careless blocking comment and strengthens a genuine one. It also gives the author a fair basis to disagree without arguing about seniority.
Separate severity from tone
Severity describes the change, not the author. A blocking comment can be direct and still respectful. Write "this path returns the cached value after logout; please clear the cache or add a test that proves the session is invalid" instead of "this is sloppy". The first version tells the author what to fix and how to prove it.
Ask reviewers to avoid rhetorical questions and absolute words such as always or never unless they are literally true. These habits read as judgment in text and often trigger a defensive reply. A neutral description of behavior keeps the thread focused on the code. The reviewer's time is better spent on the evidence than on the phrasing.
Handle disagreement and escalation
Some blocking comments will be disputed, and the scale needs a path for that. The author replies with evidence, and the reviewer either withdraws the block or names the specific concern that remains. If both sides still disagree, a named technical owner decides within one working day. The decision is recorded in the review, not in a side channel.
Escalation should not be a punishment. It is a way to resolve a genuine difference without stalling the queue. Keep the escalation list short and name the owner in advance. A pod that escalates every third review has a scale problem, not a people problem. Review the pattern monthly and adjust the definitions.
Use the scale in the review record
Record the highest severity found, the number of blocking items, and whether the change merged. This small record lets the pod see whether reviews are catching real issues or generating noise. It also gives the buyer a view of review depth without exposing individual performance. Keep the fields short enough to fill in seconds.
Use the record in the weekly review to discuss recurring causes. A cluster of blocking security comments may point to a missing guideline or a training gap. A cluster of optional style comments may mean the coding standard is not written down. The scale is working when the team can explain its patterns and change one of them.
Limitations and review cadence
A severity scale cannot make a weak review strong. It only makes the reviewer's judgment easier to read and to challenge. If the team lacks tests or clear contracts, many comments will be guesses at any severity. Fix the evidence base before adding more process to the scale.
Review the definitions each quarter and after any serious defect escape. Ask the reviewers which level is hardest to assign and why. Ask the authors which comments were unclear. Change one definition at a time and measure whether the threads get shorter. A stable scale that people trust is more valuable than a detailed one they ignore.
Questions about assessing Philippine developers
How many severity levels should a small pod use?
Three levels cover most teams: blocking, should-fix, and optional. Add a fourth only when a regulated or safety-critical behavior needs a separate approval path.
Should style comments block a merge?
Usually not. Style belongs in the coding standard and should be deferred unless it makes the change hard to verify. A style issue that hides a defect is a correctness issue, not a style issue.
What if the author disagrees with a blocking comment?
Ask for the evidence, then escalate to a named technical owner within one working day. Record the decision in the review so the queue does not stall on an unresolved thread.
