Customers sometimes ask us how large their reviews should be. It is a difficult question to answer because it depends a lot on the review culture of the team and the nature of the content being reviewed. Opinions vary widely on what the optimal size should be.
Creating many tiny reviews is not optimal because there is some static overhead for each review (setting up participants and metadata, and so forth). There is also a mental penalty for "context switching" when a reviewer changes their attention between two different reviews.
Creating a few giant reviews is not optimal because there is a limited amount of time a reviewer can truly concentrate on a review. Our studies have shown the maximum time to be about 90 minutes. If the review is too large to process in 90 minutes then the review is less effective - defects will be missed and time will be wasted as the reviewer's attention wanders.
Based on our research, we recommend the following Optimal Size for reviews, with the understanding that one size does not fit all:
-
3 participants.
-
15 files, 3 versions each (3 changelists).
-
2 overall defects, 5 file defects.
-
30 file conversations.
-
8 comments per conversation.
Collaborator is designed to give the best experience at this size. However, we do try to support a wide range of sizes so that customers can do what works best for them. Note though that reviews that are more than an order of magnitude larger than this Optimal Size will start to perform sluggishly.