Mixed Aspect Ratio Mini-Guide

[Cannonaire]

Mixed Aspect Ratio Mini-Guide
*Note: These numbers only work for NTSC footage. Maybe some other time I can get some PAL footage and figure out the numbers for that.

The purpose of this mini-guide is to present a way to easily get the aspect ratio of your video correct. It is specifically written to help those mixing sources with different aspect ratios, but it should also be useful for people who have had difficulty with aspect ratios. This guide does all of the work for you and gives you Avisynth lines which you can just copy and paste for a correct result, provided you read which one is used for which case.

The problem:
When you are working with multiple sources with different aspect ratios, you can't simply combine them and expect a good result. You will have to resize and crop or add borders to at least one of the sources in order to get a correct result. Unfortunately, aspect ratios can be confusing, even for people who have been editing for a long time.

My solution:
Using Avisynth, you can resize source footage to a common PAR (Pixel Aspect Ratio) when you start you project, and you will not have to worry about it at any other point in your creation process. The lines presented here will guarantee you are working with a common PAR with all of your footage, and that everything will have correct geometry with square (1:1) pixels, which is how things are displayed on computer monitors.

(Yes, I know that if you are only working with a single source, it might be better to do a soft-resize at the end of your project when you mux, but this adds complexity which a lot of people do not want to deal with.)

*Note: The following lines will crop a bit off the edges. This is normal, and it's most likely junk anyway. Unless otherwise specified, the crop lines also assume that you want to keep everything centered. If needed you can crop more off one side than the other as long as the total number of pixels cropped is the same as is written. It is also best and sometimes (depending on colorspace) required to crop in 2 pixel increments, but that is a topic for another day.

Step 1: Finding your your source aspect ratio
When you index your ripped DVD footage, your indexer (DGIndex, DVD2AVI, etc.) should show you the aspect ratio in the box that pops up (hit f5 for preview in these two examples). This will pretty much either be 4:3 or 16:9, regardless of what your DVD case says. You need to know this.

Step 2: Resizing and Cropping with Avisynth

If you want your final video to be 4:3:
Final results will all be 640x480

4:3 Source:

Code: Select all

Spline36Resize(648, 480, src_left=4.575, src_top=0, src_width=710.85, src_height=480)
Crop(4, 0, -4, 0)

Adjust crop as necessary, but you probably won't want to change it from this.

16:9 Source:

Code: Select all

Spline36Resize(864, 480, src_left=4.575, src_top=0, src_width=710.85, src_height=480)
Crop(112, 0, -112, -0)

If you want your final video to be 4:3, you would crop to 640x480. That means removing a total of 224 pixels from the sides. In this case, you may want to adjust the cropping based on your scene in order to keep important things in your frame; just be sure to pay attention to the edges and crop off any junk or blank space. Something like Crop(56, 0, -168, -0) would also work for the second line.

If you want your final video to be widescreen:
Final results will all be 848x480

16:9 Source:

Code: Select all

Spline36Resize(864, 480, src_left=4.575, src_top=0, src_width=710.85, src_height=480)
Crop(8, 0, -8, 0)

You can adjust the crop if needed, so long as the first and third add up to 16 (well, actually one is negative, but speaking strictly the value without the -). Pay attention to any blank space or junk video on the edges (dot crawl, noise, etc.)

4:3 Source:**

Code: Select all

Spline36Resize(864, 640, src_left=4.575, src_top=0, src_width=710.85, src_height=480)
Crop(8, 80, -8, -80)

Again, you can adjust the crop if needed. Make sure the first and third add up to 16 and the second and fourth add up to 160
**If you go this route, make absolutely certain your footage is NOT interlaced, or it will result in a mess.

That's it!
Now when you mix different aspect ratios, your circles will be circles and your square will be squares! Hurray for correct image geometry!

All of my numbers are based on information I read at this rather useful page:
http://ps-auxw.de/cgi-bin/ar-calc.pl

[Scintilla]

16:9 Source:

Code: Select all

Spline36Resize(864, 480, src_left=4.575, src_top=0, src_width=710.85, src_height=480)
Crop(112, 0, -112, -0)

If you want your final video to be 4:3, you would crop to 640x480. That means removing a total of 224 pixels from the sides. In this case, you may want to adjust the cropping based on your scene in order to keep important things in your frame; just be sure to pay attention to the edges and crop off any junk or blank space. Something like Crop(56, 0, -168, -0) would also work for the second line.

Or just don't crop at all, allowing your 16:9 sources to be bigger than the project resolution (of 640x480), then move those clips left or right in your NLE for the scenes that require it. This will not work in Adobe Premiere 5.1 due to that program's habit of automatically resizing everything to fit the project dimensions, but it will work in Adobe Premiere Pro (I've used this pan-and-scan tactic in at least two of my own videos now).

I should point out that the small amounts you crop from the left and right will have to be tweaked for sources that have bigger black borders on those sides (and I was under the impression that that was most sources). Other than that, I see no big problems with this thread, and I know there's tons of different ways to do it.

[Cannonaire]

Thanks for the input, Scintilla. Princess Mononoke comes to mind as a release which needs a lot more cropping. Keep in mind that in the lines I posted, the video is cropped twice. The end result is close to what you would get if you first cropped to 702 before resizing, but more exact. The main focus is on the actual resize lines, which will give you a common PAR.

[Phantasmagoriat]

Incoming tl;dr

I usually suggest people work out the math themselves. It's not that hard :

AR = width/height
Fullscreen = 4:3 = 4/3 = 1.333333...
Widescreen = 16:9 = 16/9 = 1.7777777...
xWidescreen ~ 2.35:1 = 2.35/1 = 2.35
[That last one was easy, but seriously, it makes more sense to use decimals. Fractions are annoying ]

so,
any resolution where the height/width = 1.3333... is 4:3 and
any resolution where the height/width = 1.7777... is 16:9

examples:
640/480 = 1.33333... = 4:3
512/384 = 1.33333... = 4:3
768/432 = 1.77777... = 16:9
1280/720 = 1.7777... = 16:9
(I'll talk about 848/480 later*)

So what about DVDs?
Using NTSC DVDs as an example, you have 720/480 = 3:2 = 1.5
This is not a standard AR. Which is pretty much mathematical proof that NTSC DVDs *must* be stretched during playback to look correct. You should resize your footage for AMVs too. (This is where a lot of confusion comes from, especially when you could technically deal with AR after editing, but that usually just leads to more problems... so I always recommend resizing prior to editing...)

Once people understand this, everything else is just geometry...
There are really only 3 things you can do to your footage:

Code: Select all

spline36resize(width, height)        # stretch to any resolution that gives you the correct AR
crop(left, top, -right, -bottom)     # chop off parts that you don't want
addborders(left, top, right, bottom) # (if you want black borders)

And if you have a particular target resolution in mind, you just combine/rearrange them so you always get the same height & width in the end. (Like if you crop off 8px from the left/right, your resultant width is obviously 16px less than it first was. This needs to be accounted for by resizing or whatever...)
Yes, that last part requires a little bit of thinking, but there are only 3 things you can do, so it's not that bad...



_________________________
That being said, I realize there are people who are not very strong in geometry, which is why I created the function BestFit().
You just have to specify: 1.) a target resolution; 2.) method to get to said resolution and; 3.) the input AR of the DVD. example:

Code: Select all

BestFit(768, 432, "crop", AR_in=1.77777) # for 16:9 footage
BestFit(768, 432, "crop", AR_in=1.33333) # for 4:3 footage

-16:9 footage gets resized to become 768x432 (in this case, saying "crop" doesn't really do anything);
-4:3 footage gets resized and cropped to become 768x432.
-Works with any target resolution.
-"Little black garbage borders" should still be cropped beforehand.
-One day, when I figure out how to make BestFit() AR-aware, you won't even need to type AR_in=...
[see BestFit() Documentation for full details]



_________________________
Anyways, to respond to your actual post I suppose this would work, although I've never seen anyone use the src_left, src_top, src_width, src_height parameters before (even the documentation for spline36resize() doesn't seem to mention how they are used). I think I would just crop first. It makes the math significantly easier. For instance, if it looks like I have 8px of black on the left/right, I can simply go:

Code: Select all

crop(8,0,-8,0)          # removes junk borders
spline36resize(848,480) # "fixes" the AR
crop(104,0,-104,0)      # results in 640x480 to be mixed with other 4:3 sources

And if I have 12px on the left, I can easily change the 8 to a 12 and my intermediate will still be 848x480, so I never have to change my crop values later. Using the method you suggest, I would have to calculate a new intermediate resolution and crop values. ie.) spline36resize(868,480).crop(116,0,-112,0). It still works, it just takes extra calculations.


But like Scintilla said, there are other ways of dealing with mixed AR's. This is the crop method. Personally, I would add borders so all my footage is present, then combine the two in my editor by panning/zooming so I can choose which areas I want to keep. (actually you don't even need to add borders if you are able to tweak your project setting appropriately) But everyone does things differently I guess.


______________
*It should be noted that 848x480 isn't really 16:9 (848/480 = 1.766666..., where 16/9 = 1.7777...), it just happens to be the closest mod16 resolution**
**values divisible by 16, or higher powers of 2, can sometimes make encoding/decoding more efficient because of the way computing works...

[mirkosp]

So what about DVDs?
Using NTSC DVDs as an example, you have 720/480 = 3:2 = 1.5
This is not a standard AR. Which is pretty much mathematical proof that NTSC DVDs *must* be stretched during playback to look correct.

While I know what you mean, I find this to be a really confusing statement as-is...

You should resize your footage for AMVs too. (This is where a lot of confusion comes from, especially when you could technically deal with AR after editing, but that usually just leads to more problems... so I always recommend resizing prior to editing...)

Nah, the whole flagging thing is only confusing if you really have no clue as to what to do. It's actually very easy. Just crop the amount needed before editing and then when encoding set the Sample Aspect Ratio to the same as the DVD, it's as easy as that.

Once people understand this, everything else is just geometry...
There are really only 3 things you can do to your footage:

Code: Select all

spline36resize(width, height)        # stretch to any resolution that gives you the correct AR
crop(left, top, -right, -bottom)     # chop off parts that you don't want
addborders(left, top, right, bottom) # (if you want black borders)

And if you have a particular target resolution in mind, you just combine/rearrange them so you always get the same height & width in the end. (Like if you crop off 8px from the left/right, your resultant width is obviously 16px less than it first was. This needs to be accounted for by resizing or whatever...)
Yes, that last part requires a little bit of thinking, but there are only 3 things you can do, so it's not that bad...



_________________________
That being said, I realize there are people who are not very strong in geometry, which is why I created the function BestFit().
You just have to specify: 1.) a target resolution; 2.) method to get to said resolution and; 3.) the input AR of the DVD. example:

Code: Select all

BestFit(768, 432, "crop", AR_in=1.77777) # for 16:9 footage
BestFit(768, 432, "crop", AR_in=1.33333) # for 4:3 footage

-16:9 footage gets resized to become 768x432 (in this case, saying "crop" doesn't really do anything);
-4:3 footage gets resized and cropped to become 768x432.
-Works with any target resolution.
-"Little black garbage borders" should still be cropped beforehand.
-One day, when I figure out how to make BestFit() AR-aware, you won't even need to type AR_in=...
[see BestFit() Documentation for full details]

See, the whole issue with this, is that it totally ignores the active area of DVDs, as opposed to the lines that Cannonaire posted.
NTSC DVDs are actually wider than pure 4:3 or pure 16:9. Proof? A widescreen DVD is 1.85:1 and not 1.78:1 if taken as is, which is exactly why the guide suggests to crop(8,0,-8,0). Guess why? Yes, to get back to the active area of the picture that represents the actual 4:3 or 16:9. Which means that if the user doesn't know this, even by using BestFit they would technically get the AR wrong (although, I'm positive most people wouldn't even notice an AR error of this amount ─ hell, some can't even realize that they mixed up 4:3 with 16:9, mind you).

_________________________
Anyways, to respond to your actual post I suppose this would work, although I've never seen anyone use the src_left, src_top, src_width, src_height parameters before (even the documentation for spline36resize() doesn't seem to mention how they are used). I think I would just crop first. It makes the math significantly easier. For instance, if it looks like I have 8px of black on the left/right, I can simply go:

Code: Select all

crop(8,0,-8,0)          # removes junk borders
spline36resize(848,480) # "fixes" the AR
crop(104,0,-104,0)      # results in 640x480 to be mixed with other 4:3 sources

And if I have 12px on the left, I can easily change the 8 to a 12 and my intermediate will still be 848x480, so I never have to change my crop values later. Using the method you suggest, I would have to calculate a new intermediate resolution and crop values. ie.) spline36resize(868,480).crop(116,0,-112,0). It still works, it just takes extra calculations.

While nobody uses them, there is a specific reason why they are used in those lines... the reason is to be as precise as possible with the geometry. Your line is actually off in AR, if one was to be as pedantic as Cannonaire was. That is because, you cropped to the perfect 16:9 (or, well, a very close approximation), but then resized to a non perfect 16:9 area (848x480 is 1.76666...)... now, if you were to just edit at that, it's fine, 848x480 is a common res. However, since you use it to bring it back to 640x480, it would make more sense to resize to something like 854x480 (1.7791666...) which is closer to perfect 16:9 and then crop to 640x480 (the fact that 854 is just mod2 is no concern to us since we won't actually be using that).

*It should be noted that 848x480 isn't really 16:9 (848/480 = 1.766666..., where 16/9 = 1.7777...), it just happens to be the closest mod16 resolution**
**values divisible by 16, or higher powers of 2, can sometimes make encoding/decoding more efficient because of the way computing works...

These things are trivial when it really comes to it. They might factor, what... a 1fps difference in a 300fps decoding speed? And even that, I'm not even sure. Mod8 and mod4 are mighty fine for actual encoding with x264, and if you're going to just resize to crop again, using a mod2 resolution is not problematic. Hell, if you feel like it, you could converttoyv24() (assuming you're using avisynth 2.6), so you can crop odd amounts before resizing, and then just converttoyv12() again. An example of this would be a source where, say, you'd have to only crop(1,1,-1,-1), like some BDs which were 720p upscaled to 1080p, but really, one could also just do spline36resize(1280,720,1,1,-1,-1) in such a case... or perhaps spline36resize(1280,720,2,1,-2,-2) if you're pedantic about AR as Cannonaire is (well, this isn't perfect 16:9 either, but it's closer to it without overcropping too much or using decimal values).

[Cannonaire]

I am much more concerned with correct image geometry than the actual display aspect ratio. Really, you could take away all the crop lines in the guide, leaving the resizing, and the main goal of this guide is achieved. I'm just trying to help people who haven't taken the time to research things like active picture area and such. You can't tell the difference very easily in a lot of cases - especially with anime - but you can be sure that you are getting a correct result rather than an approximation. And this method has the advantage of being extremely easy.

As far as my obsession with this stuff... lol yes, I definitely spent at least two days obsessing over the minutia of all this. But now that I'm done, I have it boiled down to a few simple resize lines so I never have to worry about it again.

[Phantasmagoriat]

So what about DVDs?
Using NTSC DVDs as an example, you have 720/480 = 3:2 = 1.5
This is not a standard AR. Which is pretty much mathematical proof that NTSC DVDs *must* be stretched during playback to look correct.

While I know what you mean, I find this to be a really confusing statement as-is...

I'm just trying emphasize that you can't leave the footage at 720x480 unless you know how you are going to handle it later.

You should resize your footage for AMVs too. (This is where a lot of confusion comes from, especially when you could technically deal with AR after editing, but that usually just leads to more problems... so I always recommend resizing prior to editing...)

Nah, the whole flagging thing is only confusing if you really have no clue as to what to do. It's actually very easy. Just crop the amount needed before editing and then when encoding set the Sample Aspect Ratio to the same as the DVD, it's as easy as that.

Of course , but most people I've encountered don't know the difference between Aspect Ratio and Resolution, so that's why I think most people would get confused.

See, the whole issue with this, is that it totally ignores the active area of DVDs, as opposed to the lines that Cannonaire posted. NTSC DVDs are actually wider than pure 4:3 or pure 16:9. Proof? A widescreen DVD is 1.85:1 and not 1.78:1 if taken as is, which is exactly why the guide suggests to crop(8,0,-8,0). Guess why? Yes, to get back to the active area of the picture that represents the actual 4:3 or 16:9. Which means that if the user doesn't know this, even by using BestFit they would technically get the AR wrong (although, I'm positive most people wouldn't even notice an AR error of this amount ─ hell, some can't even realize that they mixed up 4:3 with 16:9, mind you).

Well yeah, you're right. BestFit() was never intended to account for the active area. I thought I was pretty clear when I said [ "Little black garbage borders" should still be cropped beforehand ]. I would rather leave the two processes separate. To me, it makes more sense to leave that up to Crop() since different DVD's have different amounts of black borders, because in the end, aren't you still just cropping junk pixels from the edges?

While nobody uses them, there is a specific reason why they are used in those lines... the reason is to be as precise as possible with the geometry. Your line is actually off in AR, if one was to be as pedantic as Cannonaire was. That is because, you cropped to the perfect 16:9 (or, well, a very close approximation), but then resized to a non perfect 16:9 area (848x480 is 1.76666...)... now, if you were to just edit at that, it's fine, 848x480 is a common res. However, since you use it to bring it back to 640x480, it would make more sense to resize to something like 854x480 (1.7791666...) which is closer to perfect 16:9 and then crop to 640x480 (the fact that 854 is just mod2 is no concern to us since we won't actually be using that).

I should have actually caught that myself, but still, what I basically want to say is: 1.)crop the black borders 2.)resize to the intended AR 3.) then crop to the target resolution
You'll get the same result, unless you want to split pixels

*It should be noted that 848x480 isn't really 16:9 (848/480 = 1.766666..., where 16/9 = 1.7777...), it just happens to be the closest mod16 resolution**
**values divisible by 16, or higher powers of 2, can sometimes make encoding/decoding more efficient because of the way computing works...

These things are trivial when it really comes to it. They might factor, what... a 1fps difference in a 300fps decoding speed? And even that, I'm not even sure. Mod8 and mod4 are mighty fine for actual encoding with x264, and if you're going to just resize to crop again, using a mod2 resolution is not problematic. Hell, if you feel like it, you could converttoyv24() (assuming you're using avisynth 2.6), so you can crop odd amounts before resizing, and then just converttoyv12() again. An example of this would be a source where, say, you'd have to only crop(1,1,-1,-1), like some BDs which were 720p upscaled to 1080p, but really, one could also just do spline36resize(1280,720,1,1,-1,-1) in such a case... or perhaps spline36resize(1280,720,2,1,-2,-2) if you're pedantic about AR as Cannonaire is (well, this isn't perfect 16:9 either, but it's closer to it without overcropping too much or using decimal values).

yes, pedantic. That's the perfect word to describe this. I would rather overcrop slightly to ensure I get rid of all the junk borders.
Apparently that's a crime But that's just me.

[Cannonaire]

Phantasmagoriat - Actually, the lines I posted do crop a bit into areas with picture on most DVD rips I've tested.

By active picture area, I don't mean area that has image on it as opposed to areas filled by the black borders. I mean the area which is part of the 4:3 image conforming to NTSC standards. I specifically mean the 710.85x486 pixels which make up the active picture area for correctly digitized NTSC frames.

I appreciate what you're saying, but you can't just crop off the black borders then resize to a 4:3 resolution like 640x480 and expect to get correct geometry. I learned this lesson the hard way.

A thoroughly pedantic explanation follows:

Spoiler :

NTSC DVDs are made to conform to the old analog, standard definition specifications. TVs displayed the analog equivalent of rectangular pixels (taller than they are wide), not square. I think this part is well known enough. But the fact is that this specific pixel aspect ratio was standard, and DVDs conform to it. Regardless of whether the image extends past the center-most 710.85x486 pixels (cropped to 480 to be mod16 and work with mpeg2), it is this area specifically that is refered to and intended to be 4:3 (or 16:9, depending on the content).

What my resize line does is isolate the 710.85 pixels and stretch them to a resolution which would be 4:3 if there were 486 pixels of vertical resolution still present (also 16:9 in that case). 648x486 is 4:3 and 864x486 is 16:9. The resize line starts 4.575 pixels into the image and works with the next 710.85 (this explains how the src_left and src_width parameters function). Math: 720-710.85=9.15, 9.15/2=4.575. No, I have no idea how precise the math is in spline36resize, but it does specifically accept float values for these, and I have verified that these numbers do affect the final image in both how it is scaled and how it is centered. This test is repeatable by anyone willing to try.

Finally, mpeg2 only has 4 possibilities for display aspect ratio flags:
"1:1" (You will not see a retail DVD mastered like this.)
"4:3" (~0.911 PAR, this is the one which conforms to the old analog NTSC spec.)
"16:9" (~1.215 PAR, anamorphic)
2.11:1 (lol what? I don't believe this is actually used on DVDs, but it is a valid flag for mpeg2 streams.)

This means that in the only two cases which matter, my resize lines will give you the centered, correct active picture area.
You can crop to whatever resolution you like after that; heck, you can crop the 864x480 image (in case of 16:9 content) to 300x200 if you want. The video's aspect ratio will change, but the geometry will remain correct as long as you don't resize again.

To summarize, if you use the resize lines I wrote in the first post, you will have a centered image with correct geometry.

[Phantasmagoriat]

Okay... that kind of makes sense. I still think it's a bit ridiculous. Like, is it because of some traditional analogue standard... I mean, how does 710.85 pixels even work? If we're talking about digital, a pixel is a pixel [unless you want to blow my mind right freaking now ]. So if you want the central 710.85 pixels, they would have to be rounded anyways even if you used floats. You would be doing the same thing as Crop(). Precision for the sake of precision?

[Cannonaire]

Well yeah. It doesn't matter a whole lot when you're close anyway , and I don't know how precise spline36resize is, but I can tell you that it does affect both the accuracy of the resizer and the way the image is centered. I did a test and there is a very small difference in image between using 710 pixels and 710.1 (I actually tested a lot more than just these two). You would never be able to tell without a comparison like opening both screenshots in tabs and switching between while staring at them, but it is kinda nice to know that the math is correct, if only for placebo/peace of mind.