This week the Pixel 2 and Pixel 2 XL, Google’s latest smartphones, arrive in shops. Both gadgets include options like Now Playing, the Google Assistant, and the best-rated smartphone digicam ever, in line with DXO.

We designed Pixel 2’s digicam by asking how we are able to make the digicam in your Pixel 2 act like SLRs and different large cameras, and we’ve talked earlier than in regards to the tech we use to try this (reminiscent of HDR+). Today we’re highlighting a brand new function for Pixel 2’s digicam: portrait mode.

With portrait mode, you may take photos of your family and friends (that features your pets too!) that hold what’s necessary sharp and in focus, however softly blur out the background. This helps draw your consideration to the folks in your photos and retains you from getting distracted by what’s within the background. This works on each Pixel 2 and Pixel 2 XL, on each the rear- and front-facing cameras.

girl-with-the-orange-hat-s.jpg

Pictures with out (left) and with (proper) portrait mode. Photo by Matt Jones

Technically, blurring out the background like that is an impact known as “shallow depth of subject.” The large lenses on SLRs will be configured to do that by altering their aperture, however smartphone cameras have fastened, small apertures that produce photos the place every part is kind of sharp. To create this impact with a smartphone digicam, we have to know which components of the picture are distant in an effort to blur them artificially.

Normally, to find out what’s distant with a smartphone digicam you’d want to make use of two cameras shut to one another, then triangulate the depth of assorted components of the scene—similar to your eyes work. But on Pixel 2 we’re capable of mix computational images and machine studying to do the identical with only one digicam.

How portrait mode works on the Pixel 2

Portrait mode begins with an HDR+ image the place every part is sharp and high-quality.

Next, our know-how must determine which pixels belong to the foreground of the picture (an individual, or your canine) and which belong to the background. This is named a “segmentation masks” and it’s the place machine studying is available in. We skilled a neural community to take a look at an image and perceive which pixels are folks and which aren’t. Because photographs of individuals might also embody issues like hats, sun shades, and ice cream cones, we skilled our community on near one million photos—together with photos with issues like these!

Just creating two layers—foreground and background, with a tough edge in between them—isn’t fairly sufficient for all photos you’d wish to take; SLRs produce blur that will get stronger with every fraction of an inch farther from the factor that’s in sharp focus. To recreate that look with Pixel 2’s rear digicam, we use the brand new Dual Pixel sensor to look by means of the left and proper sides of the digicam’s tiny lens on the identical time—successfully giving us two cameras for the value of 1. Using these two views we compute a depth map: the gap from the digicam to every level within the scene. Then we blur the picture based mostly on the mixture of the depth map and the segmentation masks.

The outcome? Portrait mode.

girl-with-coffee.jpg

Pictures with out (left) and with (proper) portrait mode. Photo by Sam Kweskin

Portrait mode works slightly in a different way on the front-facing digicam, the place we aren’t capable of produce a depth map the identical manner we do with the extra highly effective rear-facing digicam. For selfies, we simply use our segmentation masks, which works significantly properly for selfies since they’ve easier compositions.

bikeride-selfie-comp-s.jpg

Selfie with out (left) and with (proper) portrait mode. The front-facing digicam identifies which background pixels to blur utilizing solely machine studying—no depth map. Photo by Marc Levoy

When and how you can use portrait mode

Portrait mode on the Pixel 2 is computerized and straightforward to make use of—simply select it out of your digicam menu then take your image. You can use it for photos of your mates, household, and even pets. You can even use it for every kind of “close-up” photographs of objects such flowers, meals, or bumblebees (simply don’t get stung!) with background blur.

flower-comp-s.jpg

Close-up image with out (left) and with (proper) portrait mode. Photo by Marc Levoy

Here are some ideas for how you can take nice portraits utilizing any digicam (and Pixel 2 as properly!):

  • Stand shut sufficient to your topics that their head (or head and shoulders) fill the body.
  • For a bunch shot the place you need everybody sharp, place them on the identical distance from the digicam.
  • Put far between your topics and the background.
  • For close-up photographs, faucet to focus to get extra management over what’s sharp and what’s blurred. Also, the digicam can’t deal with issues nearer than a number of inches, so keep no less than that distant.

To be taught extra about portrait mode on the Pixel 2, watch this video by by Nat & Friends, or geek out with our our in-depth, technical submit over on the Research weblog.

How Google Built the Pixel 2 Camera