The future of photography part4

HDR Stacking and Dimming

Whether the camera matrix can capture the entire range of brightness available to our eyes is an old hot topic for srachs. Some say no, because the eye is able to see up to 25 f-stops when even a top-up full-frame matrix can be pulled out to a maximum of 14. Others call the comparison incorrect, because the eye is helped by the brain, automatically adjusting the pupil and completing the image with its neural networks, and the instant dynamic range of the eye is really no more than 10-14 f-stops. Let’s leave these disputes to the best couch thinkers of the Internet.

The fact remains: when shooting friends against a bright sky without HDR on any mobile camera, you get either a normal sky and black faces of friends, or traced friends, but the sky is burned to hells.

The solution was invented long ago - to expand the range of brightness using HDR (High-dynamic-range). You need to take a few frames with different shutter speeds and glue them together. So that one was “normal”, the second is lighter, the third is darker. We take dark places from the light frame, fill the lights from the dark - profit. It remains only to solve the problem of automatic bracketing - how much to shift the exposure of each frame, so as not to overdo it, but the sophomore of a technical high school will now cope with determining the average brightness of the picture.

On the latest iPhone, Pixel and Galaxy, the HDR mode generally turns on automatically when a simple algorithm inside the camera determines that you are shooting something contrast on a sunny day. You can even notice how the phone switches the recording mode to the buffer in order to save the frames shifted in the exposure - the camera drops fps, and the picture itself becomes juicier. The moment of switching is clearly visible on my iPhone X, if you shoot on the street. Take a closer look at your smartphone next time too.

The minus of HDR with exposure bracketing is its impassable helplessness in poor lighting. Even with the light of a room lamp, the frames are so dark that the kumphuktir cannot align and glue them. To solve the problem with the light in 2013, Google showed a different approach to HDR in the then released Nexus smartphone. He used stacking by time.

Time Stacking
Long Exposure and Time Lapse Simulation

Time stacking allows you to get a long shutter speed using a series of short. The pioneers were lovers of shooting the traces of the stars in the night sky, which was inconvenient to open the shutter immediately for two hours. So it was hard to calculate all the settings in advance, and from the slightest shaking the whole frame went out of order. They decided to open the shutter only for a couple of minutes, but many times, and then went home and pasted the received frames in Photoshop.

Such drawings of stars are always glued from several photos. So it was easier to control the exposure
It turns out that the camera never actually took a long shutter speed, but we got the effect of imitating it, adding several shots taken in a row. For smartphones, there have long been a bunch of applications that use this trick, but they are all not needed since the feature was added to almost all standard cameras. Today, even an iPhone easily sticks to you a long excerpt from Live Photo.

Three clicks long shutter speed
Let’s go back to Google with its nightly HDR. It turned out that with the help of time bracketing you can realize a good HDR in the dark. The technology first appeared in the Nexus 5 and was called HDR +. The rest of the Android phones received it as a gift. The technology is still so popular that it is praised even in the presentation of the latest Pixel.

HDR + works quite simply: having determined that you are shooting in the dark, the camera unloads 8-15 recent RAW photos from the buffer to put them on top of each other. Thus, the algorithm collects more information about the dark areas of the frame to minimize noise - the pixels, where for some reason the camera could not collect all the information and turned on.

As if you didn’t know what a capybara looks like and ask five people to describe it - their stories would be about the same, but each would mention some unique detail. So you would collect more information than simply asking one. The same with pixels.

? HDR +: Low Light and High Dynamic Range photography in the Google Camera App

Adding shots taken from one point gives the same fake long exposure effect as with the stars above. The exposure of dozens of frames is summed up, errors on one are minimized on others. Imagine how much each time you had to click the shutter of the SLR to achieve this.

Pixel advertising glorifying HDR + and Night Sight
It only remained to solve the problem of automatic color correction - the shots taken in the dark usually turn out to be completely yellow or green, and we seem to want juicy daylight. In earlier versions of HDR +, this was solved by simply twisting the settings, as in filters a la to the insgrams. Then they drove the neural networks to help.

That is how Night Sight appeared - the technology of “night photography” in Pixel 2 and 3. In the description, they say: “machine learning techniques built on top of HDR +, that make Night Sight work” . In essence, this is an automation of the color correction stage. The car was trained on dataset photos “before” and “after” in order to make one beautiful one from any set of dark curve photos.

Dataset, by the way, laid out in open access. Maybe the guys from Apple will take it and finally teach their glass shovels to shoot normally in the dark.

In addition, in Night Sight, the calculation of the motion vector of objects in the frame is used to normalize the blurring, which will necessarily be obtained on a long exposure. So the smartphone can take clear parts from other frames and paste.

…