I’ve never done this before but last week I actually ran into the trap of commenting a stupid review on amazon.
Someone named Denis was complaining that an old anime series “The Rose of Versailles” from 1979 was not available in Blue-ray quality and 16:9 format. He/She (most probably “he”) claims to be somewhat of an expert on the matter of video processing but can offer no further knowledge whatsoever to back up his expertise. He simply insists that nowadays videos are easy to transform and upscale because there is software for this. And that people should insist on Blue-ray quality for everything. And that everyone who is satisfied to view anything (even if it is a 35 years old anime) in DVD quality is somehow stupid.
It’s funny how some people think that software can magically conjure up quality or content that just does not exist. Or stretch a square into a rectangle without distorting it.
His ignorance made me kind of aggressive. In the end, though, (if it is yet over) I found it quite amusing — him angrily lashing about. But enough about people on the internet. Also some research quickly told me that this battle has probably been fought already a hundred times in countless forums all over the internet.
Anyhow… This discussion got me interested in doing some experiments myself.
I don’t have much experience on video processing. So I thought I’ll just employ common sense instead.
First of all, I find it quite evident that you have to be able to transform one single picture. Afterwards I am quite willing to accept that everything that can be automated on a single picture can be applied to a video (in a way to preserve something like “continuity” over time). Then I decided this was as good an opportunity as any to play around with Photoshop.
By the way, the anime (also known as “Lady Oscar”) featuring a noble tomboy named Oscar, who is good at fighting and enters the French Army at 14 shortly before the French Revolution, is actually great, no doubt here.
Upscaling and sharpening
Photoshop provides algorithms to “sharpen” images but they are just effects that make the images appear sharper. (Still I use them all the time.) They cannot create information though. E.g. a blurred photo of some text will not likely become legible by this. (And 4:3 DVD quality will never become crystal clear 16:9 Blue-ray quality.)
I actually tried to remove a Gaussian blur on text with Photoshop’s sharpen feature: (This is from my application flyer.)
I played with the settings to get it is sharp as possible and the result looks pretty crisp to me but is still completely illegible.
I tried the same with a picture of Lady Oscar:
It looks less blurred but it also takes away all softness.
4:3 vs 16:9
The initial and biggest point of disagreement in our discussion was different image formats. The anime was drawn in 4:3 format but Denis wanted to have it in 16:9 and is perpetually convinced that this can simply be “done”.
So here’s a mathematician’s view on different image formats:
Let’s start with a 4:3 image (a screen shot from the intro of “The Rose of Versailles”) and put it into a 16:9 box. We see that 16:9 is much wider. (There remain two black vertical bars at both sides.)
Now if we want the image to fill the entire screen, we have to make it wider, e.g. by stretching the image:
In fact, this doesn’t even look as bad as I’d expected (if you ignore the original). But I guess for photos instead of drawings this might look really bad.
One could also try to scale the image without changing its proportions. In this case the 4:3 image will be higher than the 16:9 and we have to crop it (and lose the strips above and below the center), a process also called letterboxing, e.g.:
Of course you could combine both effects and do a little bit of both to obtain quite acceptable results. But I wonder what we would really gain. I’d rather see the complete, unstretched picture and put up with the black bars.
The best way to widen the image would certainly be to fill the empty bars on the left and right with image material. Since it does not seem feasible to have some poor kid complement all those images in a video I thought of a fun test for my very favourite Photoshop feature:
Playing with content aware fill
Let Photoshop fill the bars with what it finds appropriate.
For this intro picture this worked extremely well. But let’s try some others:
These were all 4:3 images (not necessarily from TV) that have been extended to 16:9. As expected, Photoshop performs well on unstructured contents like huge crowds of people or nature. (In fact, I am thoroughly impressed by what it did with the lake and the mirroring.) Whenever there are any unique or recognizable features, however, (like protagonists or people we otherwise recognize) this is bound to yield duplicates or other strange effects.
But I guess it still could work for some videos — in a similar way that things like Ambilight enhance the spectator’s experience.