What's super resolution and why does it fail?

There are a lot of technique to enhance the quality of a picture. One of them is the super resolution. It can be done in multiple ways such as with machine learning combined with a single image where the machine will allucinate the missing details. This technique will go by improving contineously , but it's not actually showing you the reality.

If you want to improve your photo and they remain real, combining multiple picture might be a better approach. There are a tutorial on the web already for that, but not much information concerning at the limitation of these algorithms.

In order to increase the resolution of an image, we have to find more details. The highest the details level is, the highest the spatial frequency is requiered. By analysing the sub pixel information of multiple slightly offsetted picture it should be in theory possible to do so.

So imagine we take 3 picture. The oranges/red points shows the brightness level of the first ~10 pixels in the picture. With only one picture, we can try to guess the upsampled version by reading the value on the corresponding curve. (Known as Bi-Cubic Interpolation, actually only Cubic Interpolation in this example as we look at a single row of pixels, not a 2D map.) Some it fall pretty close, sometime it's pretty far away (it is a luck game).

However if we combine the pixels of all 3 picture, it reduces our possible to the blue curve which match the reality when enough point are present... Pretty cool isn't it?

But wait! Why are we still buying higher resolution camera when all we have to do is take multiple picture of the same thing? There are a few reasons...

1-It only works with static subjects.

Furthermore, you have to move the camera a little bit between each pictures which cause paralax error and might be difficult to reproduce or can contain duplicated frames. But that's not the main reason...

2-Anti-Aliasing filter!

Most camera are equiped with Anti-Aliasing filter. This avoid potential moiré pattern when recording lines, stripes and other repeating pattern that are near the size of 2 pixels long. With the anti-aliasing filter the result will be a smooth color as opposition to camera without anti-aliasing filter where the pattern will start to change shape. It is partically annoying in video when a shirt start wiggling in all directions!

The anti-aliasing filter basically filter any frequencies that are too high. So the information is lost unfortunately... These filters aren't perfect, so there might persist a little bit of information so it can't still be worth trying in some occasion but it really work best with high end camera without anti-aliasing filter.