Gaining resolution from adjacent moments

I was listening to an edition of Creative Planet’s Digital Production Buzz podcast where they talked to some people from DTS about digital restoration work (32 min 39 sec in).

DTS have a system that has enough maths in it to calculate what was originally captured on the film negative. The work isn’t done by eye, by expert restorers. This system understands the maths of lenses, emulsions, stocks and film grain to calculate what detail has gone missing over the years.

This runs counter to the a blog post referred to me by Jean P. It covers what information about our current movies will make it into the future for scholars to research the 20th and 21st centuries.

The element of he DTS story that is relevant to today’s production is that they can use the same restoration technology to solve problems that are all too common right now. They talk about recovering footage from footage that is processed incorrectly and even dealing with footage that is shot out of focus! They also can increase resolution from SD to well past HD…

Their system is not real-time, but it can deal with digital files as well as celluloid. This isn’t a photochemical process. They apply mathematical equations to digital files. They say they are solving digital signal processing problems.

One of their secrets is using information from adjacent frames to gather information about the current frame. Even when a camera is locked off individual film grains or digital pixels receive slightly different light information from frame to frame. Very few shots have cameras that are locked off completely solidly. Cameras move. Subjects move. The differences between frames provide the extra resolution. They take this information to understand what light would have been captured between the film grains and digital pixels. They use that information to give more detail for every frame.

This means that your old Hi-8 or home DV footage may hold enough information to be scaled up to HD or better in the future. Now this software is only available from DTS. One day we’ll have the software and processing power to do this for our own footage.

At last we’ll have technology in the real world that will be able to do what they do in movies and TV shows like CSI and other police procedurals: select a small part of a video image and press the button marked ‘Enhance!’

My tip for security camera designers: improve the number of pictures you take a second at the current resolution for signal processing tools to be able to reveal details never seen before.