😴 🧙🌈 ʕ•ᴥ•ʔ

The following article covers Google code review process and their in-house code review tool, Critique, and why software engineers at Google love Critique features so much.

https://engineercodex.substack.com/p/how-google-takes-the-pain-out-of


A lot of ex-Google engineers talk about how much they miss Critique, Google’s code review tool, out of all the internal tools they leave behind.

[Source: X/Twitter](https://twitter.com/nsthorat/status/1728211545165828162)

Source: X/Twitter

Another Reddit comment laments about how they “miss [Critique] so bad”, listing off various features they miss, like its “attention set,” which is explained below.

This tracks with Google’s own findings. Internally, 97% of Google software engineers are satisfied with Critique.1

But what is Critique exactly and what makes it so good? 

How does this pair with Google’s actual process of code review?

In this article, I dive into:

While the SWE Book covers the basics of Critique, new blog posts and research by Google show new developments in their code review processes, such as using AI to make automatic suggestions and improvements to code changes.

Google’s Code Review Guidelines

To understand how Google writes clean, maintainable code, I wrote about Google’s readability process here.

Google’s guidelines for a good code review include:

For a changelist (CLs are Google’s version of pull requests, or PRs) to go through, it must have 0 unresolved comments, a LGTM (Looks Good To Me) from at least one reviewer, and 2 types of approval:

One person can be the LGTM-er and approver all at once.

Subscribe

Critique: Google’s Code Review Tool

Critique is Google’s code review tool, allowing engineers to efficiently review and submit code changes.

Critique’s UI, from SWE Book

They also have a diff view between the current codebase and the proposed changes:

The diff view, from Resolving Code Review Comments with ML

Importantly, Google’s recent publications in 2023 show they have comprehensive AI-powered code review tools in-house now (as shown above). 

When reviewers leave comments on code, Critique will show suggested ML-powered edits, which mean that the author of the code review just has to click one button to address the comment in entirety.

Recent developments, based on Google research papers, tell us that Google is improving developer productivity with AI-powered code review tools where possible.

The Code Review Flow

The SWE Book covers Critique in detail with over 5000 words, but I summarize the key points here and have provided a link for those who want to dive deeper into it. 

Stage 1: Create a Change

A CL (or Pull Request) is created in Google’s in-house code editor Cider, which is “tightly integrated with Critique and other in-house Google tools”, leading to higher developer productivity.

Critique integrates feedback channels for analysis writers.

Reviewers have the option to click “Please fix” on an analysis generated comment as a signal that the author should fix the issue.

Either authors or reviewers can click “Not useful” in order to flag an analysis result that is not helpful in the review process. (Source)

Stage 2: Request Review

When a pull request is ready to be reviewed, the code author adds reviewers and sends it to them officially for review.

When a PR/CL is sent in for review, “presubmits” are run if they haven’t already on the current snapshot of code. This means that everyone involved in the review knows whether or not the code breaks anything.

From SWE Book

Code reviews can also be anonymized, where the code author can be kept anonymous from the reviewer. However, Google didn’t find much of an useful difference between anonymous code reviews and “real” code reviews.3

Using research to make code review more equitable

Stages 3 and 4: Understanding and Commenting on a Change

There’s a dashboard for review statuses and an “attention set” which lets people involved in a certain code review who is the current person being waited upon.

The attention set is commonly a highly-regarded feature by Google engineers.

The Critique dashboard, from SWE Book

Stage 5: Change Approvals

As noted above, for a review to go in, it needs an LGTM from at least one reviewer. It needs 0 unresolved comments, though a code author can mark a comment resolved themselves when they reply. It also needs approvals from owners of the part of the codebase the code is going into, along with a readability approval.

As mentioned before, this can all be done by one reviewer.

Stage 6: Committing a Change

Changes are submitted and committed within Critique itself.

Critique is valuable even after changes are submitted.

“Google researchers found strong evidence that Critique’s uses extend beyond reviewing code. Change authors use Critique to examine diffs and browse analysis tool results. In some cases, code review is part of the development process of a change: a reviewer may send out an unfinished change in order to decide how to finish the implementation. Moreover, developers also use Critique to examine the history of submitted changes long after those changes have been approved.”

Modern Code Review Stats at Google

Google conducted a study on code review at the company specifically. I’ve listed some interesting stats from their paper.

Change Authoring Frequency:

Review Frequency:

Time Spent Reviewing Per Week:

Initial Feedback Waiting Time:

Overall Review Process Time:

“Previous studies have found that the number of useful comments decreases and the review latency increases as the size of the change increases. Size also influences developers’ perception of the code review process; a survey of Mozilla contributors found that developers feel that size-related factors have the greatest effect on review latency.” (Source)

Furthermore, the number of comments received per change decreases with seniority.

Source

Another stat for you: 100% of Engineer’s Codex readers report higher satisfaction after reading. So subscribe, it’s free :)

Subscribe

Why is Critique so loved by Googlers?

Most Googlers and ex-Googlers are big fans of Critique.

Internally, 97% of Google software engineers are satisfied with Critique.4

I asked 7 Googlers about why they prefer Critique over tools they may have used in the past, like GitHub.

They noted:

A Reddit comment lists more:

A great quote on static analysis:

A qualitative study of 88 Mozilla developers found that static analysis integration was the most commonly-requested feature for code review.

Automated analyses allow reviewers to focus on the understandability and maintainability of changes, instead of getting distracted by trivial comments (e.g., about formatting).

Thoughts and Takeaways

While many of these features are available in other tools today, I believe it is the tight integration and extreme “personalization” of the tooling towards a Google-specific workflow and codebase that makes it so loved.

At the same time, that means it’s not realistic for every company to replicate Critique and related tools exactly. For example, some of their tooling seems specific to challenges created by their monorepo structure.

While Critique itself will never be open-sourced, Gerrit is a similar tool to Critique. It’s an open-source code review tool also created and maintained by Google.

However, I think Google does put a lot of effort and thought into developer productivity. They publish their research freely and there are useful takeaways from their work.

1 - Modern Code Review: A Case Study at Google

2 - Modern Code Review: A Case Study at Google

3 - https://research.google/pubs/pub50130/

4 - Modern Code Review: A Case Study at Google

#reads #engineering codex #google #code review #critique