Denis Uses Neural Networks To Upscale A 1896 Film to 4K 60 FPS Quality

The Arrival of a Train at La Ciotat Station is one of the most famous silent films ever made. It was directed and produced by Auguste and Louis Lumière, it achieved an unprecedented level of quality for its time.

The 50-second silent film was first released in 1895. The was filmed in La Ciotat, Bouches-du-Rhône, France. It was filmed by means of the Cinématographe, an all-in-one camera, which also serves as a printer and film projector. As with all early Lumière movies, It was made in a 35 mm format with an aspect ratio of 1.33:1.

The film now widely-discredited legend about its first screening. Supposedly the moving image of a train headed right towards the camera was so overpowering when it was first shown to an audience that people ran screaming from the theater, fearful that the train was about to barrel through the screen and crush them.

Some people regard its commercial exhibition in 1896 as the birth of the film industry. An urban legend likely apocryphal says that viewers found the footage so realistic that they screamed and ran to the back of the room as the train approached. I’ve embedded a video of the original film above.

A man named Denis Shiryaev used modern machine-learning techniques to upscale the classic film to 21st-century video standards. Denis used neural networks to create a 4K 60 Fps version of The Arrival of a Train.

The results feel shockingly real, particularly the movements of the passengers in the foreground, who feel flesh and blood in a way they never did, it feels the world of our great-great-great-grandparents come to life.

Formerly murky details of the train, the clothing, and the faces of the passengers now stand out clearly.

How did Shiryaev do it? He says he used commercial image-editing software called Gigapixel AI. Created by Topaz Labs, the package allows customers to upscale images by up to 600 percent. Using sophisticated neural networks, Gigapixel AI adds realistic details into an image to avoid making it look blurry as it’s scaled up.

Neural networks are networks of artificial neurons mathematical functions that transform a set of input values into an output value. Neural Network can be trained, if you have a bunch of example inputs whose “correct” outputs are known, you can tune the parameters of the network to make it produce correct answers.

The hope is that this training will generalize, that once you’ve trained it to produce the right answer for inputs the network has seen before, it will also produce good answers for inputs it hasn’t seen, too.

To train a network, you need to have a database of examples where the right answer is already known. Sometimes AI researchers have to hire human beings to produce these right answers by hand.

But for an image upscaling, there’s a convenient shortcut: you start with high-resolution images and continue the process by downsampling them. The low-resolution images now become your inputs and the high-resolution originals serve as the “correct” answer the network is aiming to produce.

“A neural network analyzes thousands of photos pairs to learn how details usually get lost,” Topaz Labs explains on their product page for Gigapixel AI. “The algorithm learns to ‘fill in’ information in new images based on what it has learned, effectively adding new detail to your photo.”

Show the neural network a low-resolution image of a face and it will figure out that it’s a face and fill in the right details for the subject’s eyes, nose, and mouth. Show the neural network a low-resolution brick building and it will add a suitable brick pattern in the high-res version.

An obvious next step would be to colorize the video. Neural networks can also do that, too, using the same basic technique: the first step is to start with a bunch of color photos, after that convert them to black and white or say into grayscale images, and then train a neural network to reconstruct the color originals.