The digital camera on the brand new Pixel 2 is packed stuffed with nice hardware, software program and machine studying (ML), so all you might want to do is level and shoot to take superb photographs and movies. One of the applied sciences that helps you are taking nice photographs is HDR+, which makes it attainable to get glorious photographs of scenes with a wide variety of brightness ranges, from dimly lit landscapes to a really sunny sky.

HDR+ produces lovely pictures, and we’ve advanced the algorithm that powers it over the previous yr to make use of the Pixel 2’s software processor effectively, and allow you to take a number of photos in sequence by intelligently processing HDR+ within the background. In parallel, we’ve additionally been engaged on creating hardware capabilities that allow considerably higher computing energy—past current hardware—to convey HDR+ to third-party pictures purposes.

To broaden the attain of HDR+, deal with essentially the most difficult imaging and ML purposes, and ship lower-latency and much more power-efficient HDR+ processing, we’ve created Pixel Visual Core.

Pixel Visual Core is Google’s first custom-designed co-processor for client merchandise. It’s constructed into each Pixel 2, and within the coming months, we’ll flip it on by a software program replace to allow extra purposes to make use of Pixel 2’s digital camera for taking HDR+ high quality photos.

pixel visual core

Magnified picture of Pixel Visual Core

Let’s delve into the small print for you technical people on the market: The centerpiece of Pixel Visual Core is the Google-designed Image Processing Unit (IPU)—a totally programmable, domain-specific processor designed from scratch to ship most efficiency at low energy. With eight Google-designed custom cores, every with 512 arithmetic logic items (ALUs), the IPU delivers uncooked efficiency of greater than 3 trillion operations per second on a cellular energy finances. Using Pixel Visual Core, HDR+ can run 5x sooner and at lower than one-tenth the power than operating on the appliance processor (AP). A key ingredient to the IPU’s effectivity is the tight coupling of hardware and software program—our software program controls many extra particulars of the hardware than in a typical processor. Handing extra management to the software program makes the hardware easier and extra environment friendly, but it surely additionally makes the IPU difficult to program utilizing conventional programming languages. To keep away from this, the IPU leverages domain-specific languages that ease the burden on each builders and the compiler: Halide for picture processing and TensorFlow for machine studying. A custom Google-made compiler optimizes the code for the underlying hardware.

In the approaching weeks, we’ll allow Pixel Visual Core as a developer choice within the developer preview of Android Oreo 8.1 (MR1). Later, we’ll allow it for all third-party apps utilizing the Android Camera API, giving them entry to the Pixel 2’s HDR+ expertise. We can’t wait to see the attractive HDR+ pictures that you simply already get by your Pixel 2 digital camera change into accessible in your favourite pictures apps.