Types of Peer Reviews

There are many different types of peer reviews called by many different names in the software industry. Peer reviews go by names such as inspections, team reviews, technical reviews, walk-throughs, pair reviews, pass-arounds, ad-hoc reviews, desk checks, and others. However, I have found that most of these can be classified into one of three major peer review types:

• Desk Checks: A desk check is a process where one or more peers of a work product’s Author reviews that work product individually. Desk checking can be done to detect defects in the work product and/or to provide engineering analysis. The formality used during the desk-checking process can vary. Desk checking can be the most informal of the peer review processes or more formal peer review techniques can be applied. Desk checking can be a complete peer review process in and of itself, or it can be used as part of the preparation step for a walk-through or inspection.
• Walk-Throughs: A walk-through is the process where one or more peers of a work product’s Author meet with that Author to review that work product as a team. A walk-through can be done to detect defects in the work product and/or to perform engineering analysis. The formality used during the walk-through process can also vary. An example of a very informal walk-through might be an Author holding an impromptu “white board” walk-through of an algorithm or other design element. In an informal walk-through there may be little or no preparation. In a more formal walkthrough, preparation is done before the team meeting typically through the use of desk checking. Typically preparation is left to the discretion of the individual Reviewer and may range from little or no preparation to an in-depth study of the work product under review. During the walk-through meeting, the Author presents the work product one section at a time and explains each section to the reviewers. The Reviewers ask questions, make suggestions (engineering analysis) or report defects found. The Recorder keeps a record of the discussion and any suggestions or defects identified. After the walk-through meeting, the Recorder produces the minutes from the meeting and the Author makes any required changes to the work product to incorporate suggestions and to correct defects.
• Inspections: An inspection is a very formal method of peer review where a team of peers, including the Author, performs detailed preparation and then meets to examine a work product. The work product is typically inspected when the Author thinks it is complete and ready for transition to the next phase or activity. The focus of an inspection is only on defect identification. Individual preparation using checklists and assigned roles is emphasized. Metrics are collected and used to determine entry criteria in the inspection meeting as well as for input into product/process improvement efforts.

While inspections are always very formal peer reviews, the level of formality in desk checks and walk-throughs varies greatly depending on the needs of the project, the timing of the reviews, and the participants involved.

The type of peer review that should be chosen depends on several factors. First, inspections are focused purely on defect detection. If the Author is looking for engineering analysis and improvement suggestions (for example, reducing unnecessary complexity, suggesting alternative approaches, identifying poor methods or areas that can be made more robust), a desk check or walk-through should be used. The maturity of the work product being reviewed should also be considered when selecting the peer review type. Desk checks or walk-throughs can be performed very early in the life of the work product being reviewed. For example, as soon as the code has a clean compile or a document has been spell-checked. In fact, white-board walk-throughs can be used just to bounce around very early concepts before there even is a work product. However, inspections are performed once the author thinks the work product is done and ready to transition into the next phase, or activity in development. Staff availability and location can also be a factor. If the peer review team is geographically dispersed, it can be much easier to perform desk checks than walk-throughs or inspections. However, the use of modern technology including web-based meetings and video conferencing makes long-distance peer review meetings much more feasible. Economic factors such as cost, schedule, and effort should also be considered. Team reviews tend to cost more and take longer than individuals reviewing separately. More formal peer reviews also tend to cost more and take longer. However, the trade-off is the effectiveness of the reviews. Team peer reviews take advantage of team synergy to find more defects and more formal reviews also typically are more thorough and therefore more effective at identifying defects. One final factor to consider when choosing which type of peer review to hold is risk. I will discuss risk-based peer reviews in another article.

Table 1: Differences Between the Three Main Categories of Peer Reviews