Hatter vs. Tab: The Codec Grudge Match [SPLIT]

[Zarxrax]

Just because I like arguing,

Doesn't mean you're very good at it, keep that in mind.

I will tell you why you are wrong, hatter.

You mean you aren't just gonna sit there waitng in glorious anticipation of that magical blue fairy to come down, wave her magic want, and make all your dreams and arguments come true!? BLASPHEMY!

Because you are making the argument for your beloved OGM container based ONLY on usage by your warez folks. They are NOT the only people that need a container. There are legitimate needs and uses for every single feature that matroska supports. Just because YOU don't need it, doesn't mean its worthless.

And if we were arguing about it in the context you're on, heck you'd even be right! But the thing of it is, we're not. This whole argument is about the distribution of media, ie the warez folks. And I know a lot of you REALLY don't like this lil fact, but YOU ARE warez folks. Maybe not so much with anime since it's all chopped up, but uh, you are essentially sharing songs back and forth, up and down, all over the place. I'm surprised the RIAA hasn't shut this place down already...of course granted I am quite the active participant in this community and the RIAA generally doesn't like to get on my bad side.

I'm not pretending that I don't do any illegal activities. But you are STILL persisting that "distribution of media" = warez, which it DOES NOT. You're whole argument is that the additional features of matroska are not necessary for people, but I'm here telling you, as a prospective user of the matroska format, that its features are going to be EXTREMELY useful in the type of video that I encode. And as a user of the format that finds it useful, its existence and usefulness is justified, is it not?

[Onideus_Mad_Hatter]

Talking about ram, IRM's MRAM is very interesting and worth following.

I glanced over the site, they don't seem to be too keen on comparing it to other types of memory. Losing stuff isn't really a problem so long as you save every so often or have auotsaving turned on.

Still, it'd be nice to see how MRAM can hold up to RAMBUS, DDR and SLDRAM as far as speed. RAMBUS is pretty fuckin sweet, but tight controls and licensing have made buying RIMMs pretty freakin expensive.

[Tab.]

I'm starting to wonder if Matt purposely leaves things out and acts ignorant just because it makes these 'arguements' go around and around until the party who is making the sense gets too frustrated to continue. I'm beginning to think that's how all of his supposed online victories are won

I've been trying to express the viewpoint of the average encoder

That's not the only genre of average you've been portraying.

[Lyrs]

Talking about ram, IRM's MRAM is very interesting and worth following.

I glanced over the site, they don't seem to be too keen on comparing it to other types of memory.

Losing stuff isn't really a problem so long as you save every so often or have auotsaving turned on.

or so you say. woot, i can quotes as well. anyway, continue on with your discussion.

[Onideus_Mad_Hatter]

I'm not pretending that I don't do any illegal activities. But you are STILL persisting that "distribution of media" = warez, which it DOES NOT.

I hate to disillusion you, but outside of the DVD format, it generally is. Maybe not in ALL cases, but on the whole.

You're whole argument is that the additional features of matroska are not necessary for people, but I'm here telling you, as a prospective user of the matroska format, that its features are going to be EXTREMELY useful in the type of video that I encode. And as a user of the format that finds it useful, its existence and usefulness is justified, is it not?

Just as a stretch, why don't you tell me how it would be useful to you, as an encoder. And do keep in mind that anything you bring up, I'm going to be scrutinizing your purposed source footage. Because on the whole, a lot of it comes down to feature support that hardly any source footage even has at this point.

In the big picture, I could see it having some usefulness to people who do animation rendering, but as far as the distribution of their work, using mkv would only make sense as far as net.distribution and even then using the DVD format would be a much more logical choice since it could be distributed net wise and for desktop players. And the breaking point of course is security. What security features does mkv have? None? See, that's a big problem for the kinds of people you're trying to argue for. Essentially, what the peeps developing mkv are doing is trying to make a freeware, openware version of the DVD format. An interesting idea, however one that I can tell you is destined for failure. Simply because on side A, the warez side, it doesn't offer any new useful implemenations that aren't already widely avaiable and supported. And on side B, the developers side, it has no security features, which makes it a VERY poor choice of distribution. The developers, the people you're talking about, would much rather wait until the next DVD esque standard gets released. (ie a proffessionally done job, with very tight controls, licensing, etc, etc, etc).

[Onideus_Mad_Hatter]

On a further point, if there is a new DVD esque standard that becomes widely accepted and move and TV studios start using it, THEN MKV might become relevant as far as the warez community is concerned, but that's not likely to happen for at least another 5 to 10 years. And trust me, by then, there will be much better alternatives to MKV. MKV is gonna wind up being one of the black sheeps of video formats. Neither here nor there, just sorta inbetween.

[trythil]

I'm not pretending that I don't do any illegal activities. But you are STILL persisting that "distribution of media" = warez, which it DOES NOT.

I hate to disillusion you, but outside of the DVD format, it generally is. Maybe not in ALL cases, but on the whole.

You're whole argument is that the additional features of matroska are not necessary for people, but I'm here telling you, as a prospective user of the matroska format, that its features are going to be EXTREMELY useful in the type of video that I encode. And as a user of the format that finds it useful, its existence and usefulness is justified, is it not?

Just as a stretch, why don't you tell me how it would be useful to you, as an encoder. And do keep in mind that anything you bring up, I'm going to be scrutinizing your purposed source footage. Because on the whole, a lot of it comes down to feature support that hardly any source footage even has at this point.

In the big picture, I could see it having some usefulness to people who do animation rendering, but as far as the distribution of their work, using mkv would only make sense as far as net.distribution and even then using the DVD format would be a much more logical choice since it could be distributed net wise and for desktop players. And the breaking point of course is security. What security features does mkv have? None? See, that's a big problem for the kinds of people you're trying to argue for. Essentially, what the peeps developing mkv are doing is trying to make a freeware, openware version of the DVD format. An interesting idea, however one that I can tell you is destined for failure. Simply because on side A, the warez side, it doesn't offer any new useful implemenations that aren't already widely avaiable and supported. And on side B, the developers side, it has no security features, which makes it a VERY poor choice of distribution. The developers, the people you're talking about, would much rather wait until the next DVD esque standard gets released. (ie a proffessionally done job, with very tight controls, licensing, etc, etc, etc).

Actually, from a programmer's perspective, Matroska is very interesting. It is, as far as I know, the first media container based around the Extensible Binary Meta Language, aka EBML.

Think of XML, but for binary files.

The practical upshot of all this is that Matroska is very flexible -- moreso than any other container. It is possible to dramatically change the container specification and retain backwards compatibility with old specifications.

I don't know how much you know about file formats, but this is something that's very rarely achieved. Which makes it very, very cool.

High-level overview of the Matroska container:
http://cvs.corecodec.org/cgi-bin/viewcv ... index.html

Matroska specification:
http://cvs.corecodec.org/cgi-bin/viewcv ... index.html

Currently supported codecs:
http://cvs.corecodec.org/cgi-bin/viewcv ... index.html

If you have the ability and time, you should read over these documents carefully.

Matroska is really quite an interesting project; it's not just a VOB rehash.
There really exists no reason for media producers to NOT adapt it.

Your argument about security of the contained media is a very bad one. Security does not have to exist in the container; indeed, it is more often embedded into the video stream itself, usually in the form of encrypted content or watermarking. The video stream and the container are two separate (though interrelated) entities.

Security of attachments, on the other hand, is a problem;. This problem, however, is not something you should shun the Matroska container format for; it simply offers attachments as an option. The entire problem with attachment security is no longer a technical one; it mostly revolves around human issues. The technical issues with handling attachments safely have been worked out many times over; see the Java 2.0 security model for an extensive example.

It should also be noted that no player currently allows you to work with Matroska attachments, so this is one feature that I don't see being used very much. It might even be removed in a future Matroska specification.

Here, this is one example of a Matroska feature that no other container format has.

In Matroska there exists no such things as P- or B-frames. Instead, each block of data contains a ReferenceBlock field that describes which block a given block is based off of. You can, of course, have ReferenceBlocks reference other ReferenceBlocks, and they don't have to be sequential.

This is the inter-frame concept taken to an entirely new level. NOTHING else does this.* It has, among other things, the ability to dramatically improve compression ratios without further loss of quality. That is something that you would want to put on ANY medium of storage, since storage space is always finite, and everyone's always looking to pack more in less space.

* Ogg can CONCEIVABLY do this, but it won't be standardized by any means -- and that means fuck-all for widespread support. As I'm sure you know, the Ogg container is nothing but a stream of raw data with very basic error correction and seek support, with additional meaning taking shape only at the higher levels.

[Zarxrax]

Possible uses:

For amvs: I can attatch lyrics, production notes, etc to the file. I can include optional karaoke subtitles for the song.

For people that make anime tv captures: 95% of the raw encoders these days are encoding their anime at 120fps. Now aside from being a nasty hack, this greatly increases the amount of cpu required to playback the file, as well as adding a ton of overhead into the file. When fansubbers want to sub the show, they have to decimate it down to 23.976 fps in order to make it actually playable by most people. In the process, the smoothness of pans and other 29.97 fps scenes is lost. This can make said scenes look like SHIT at times. If there's a way to preserve it, then by all means, I'm certainly not going to stand in the way.

[Onideus_Mad_Hatter]

Actually, from a programmer's perspective, Matroska is very interesting. It is, as far as I know, the first media container based around the Extensible Binary Meta Language, aka EBML.

Think of XML, but for binary files.

The practical upshot of all this is that Matroska is very flexible -- moreso than any other container. It is possible to dramatically change the container specification and retain backwards compatibility with old specifications.

From a hackers perspective flexibility usually translates to easily exploitable. The perspective I'm going on about here in this thread is your average encoder/downloader. And if you think trying to explain your stance to me is hard, just imagine trying to convince the countless hordes out there who are gonna be practically at your throat when you try pushing a new format on them. That's the primary problem with most of the peeps around Doom9, they don't seem to bother giving any thought to the general social mechanics of net.society. Being a social engineer, amongst various other things, I do, and what I'm trying to get you and others to understand is that in all likelihood Matroska is gonna be rejected. Why? Well if you need a point of comparison, why don't you go subscribe to some binary newsgroups and check out some of the PAR vs PAR2 flame wars. Or even OGM itself. Yes, surprising as it may be to some, OGM hasn't even been entirely accepted. There are countless hordes of people who are fighting against it's use, whether out of ignorance, rejection of change, whatever, it doesn't matter. And now you have the brilliant idea of trying to push ANOTHER OGM incarnation onto the masses and in less than 2 years of each other?! That is just the absolute freakin HEIGHT of stupid.

I don't know how much you know about file formats, but this is something that's very rarely achieved. Which makes it very, very cool.

Not only is it most likely gonna be easily exploitable but since the amateur programmers at Doom9 are gonna be building, it's most likely gonna be massively unstable on top of it all. Tab himself has already gone on about how he thinks OGM is unstable, and it's workin off a Direct Show filter! o_O

High-level overview of the Matroska container:
http://cvs.corecodec.org/cgi-bin/viewcv ... index.html

Matroska specification:
http://cvs.corecodec.org/cgi-bin/viewcv ... index.html

Currently supported codecs:
http://cvs.corecodec.org/cgi-bin/viewcv ... index.html

If you have the ability and time, you should read over these documents carefully.

Oh I probably will...and if there's anything good in there I might just go ahead and repost the links for some of the peeps I know in the hacking community.

Matroska is really quite an interesting project; it's not just a VOB rehash.
There really exists no reason for media producers to NOT adapt it.

Your argument about security of the contained media is a very bad one. Security does not have to exist in the container; indeed, it is more often embedded into the video stream itself, usually in the form of encrypted content or watermarking. The video stream and the container are two separate (though interrelated) entities.

Given how well the "security" on the DVD format worked out, it's most likely that producers are gonna want as much security as they can get, encoded into the video stream itself, the container, the transport mechanism, etc, etc. And trust me on this one, they're NOT gonna want something put together by some kids on a group called *snicker* "Doom9" that's based on vastly unsupported and untried programming theory. Might be kind of a fun experiment at exploring a new area in programming, but to base a wannabe mainstream video container on it with wet dreams of replacing the DVD format or future DVD esque formats...yeah, no programmer/producer in their right mind would want that.

Security of attachments, on the other hand, is a problem;. This problem, however, is not something you should shun the Matroska container format for; it simply offers attachments as an option.

Actually at this point I'm more concerned about the spam artists than I am about the hackers. Usually the hackers have SOME ethics, the spam artists won't hesitate to exploit every single last hole they can find.

The entire problem with attachment security is no longer a technical one; it mostly revolves around human issues. The technical issues with handling attachments safely have been worked out many times over; see the Java 2.0 security model for an extensive example.

*snicker*

I think maybe I'll quote you and include some links for Sugien, I'm sure he'd love to take a crack at it. Java is about as secure as the ICQ protocol when you get right down to it and Sugien has found ways to do some rather malicious things with it in the past. ^_^

It should also be noted that no player currently allows you to work with Matroska attachments, so this is one feature that I don't see being used very much. It might even be removed in a future Matroska specification.

Boy that'd probably be a good idea. Integration does not always == smart.

Here, this is one example of a Matroska feature that no other container format has.

In Matroska there exists no such things as P- or B-frames. Instead, each block of data contains a ReferenceBlock field that describes which block a given block is based off of. You can, of course, have ReferenceBlocks reference other ReferenceBlocks, and they don't have to be sequential.

This is the inter-frame concept taken to an entirely new level. NOTHING else does this.*

Nothing huh? Heh, what I'm working on can, only it takes it a few steps beyond anything the kids at Doom9 have thought of. ^_^

You should also keep in mind that there are a variety of others like me who are also developing new codecs, formats, etc. The difference between people like us and the kids at Doom9? Most of us are working off private contracts for various corporate entities...in other words, we get paid.

It has, among other things, the ability to dramatically improve compression ratios without further loss of quality. That is something that you would want to put on ANY medium of storage, since storage space is always finite, and everyone's always looking to pack more in less space.

Well as far as that's concerned they're right on track, but they're still way behind most corporations R&D as far as that's concerned.

[trythil]

You seem to have missed my point entirely, which involved correcting several technical misconceptions about the Matroska container format.

But for the sake of argument, I'll play along.

Actually, from a programmer's perspective, Matroska is very interesting. It is, as far as I know, the first media container based around the Extensible Binary Meta Language, aka EBML.

Think of XML, but for binary files.

The practical upshot of all this is that Matroska is very flexible -- moreso than any other container. It is possible to dramatically change the container specification and retain backwards compatibility with old specifications.

From a hackers perspective flexibility usually translates to easily exploitable.

True, to a point. Flexibility in systems means higher complexity, which does translate to a higher number of exploitable points. It doesn't necessarily mean <i>easily</i> exploitable, though.

Semantics aside, first I'll provide a general counterexample of a flexible yet secure system and then move on to address the point.

Take a look at the design of the GNU HURD for a counterexample. It's an extremely flexible design that keeps the most critical parts of the system secure. Read the designer's paper for an architectural overview of the system.

The Matroska specification by itself is no more exploitable than the OGM specification or AVI specification.

Your arguments seem to center around the idea of file attachments in Matroska as making the container "exploitable". I already addressed this case in my last post. Your arguments are invalid for the rest of Matroska; I have not seen you provide evidence that libmatroska or the underlying EBML parser are inherently insecure.

And now you have the brilliant idea of trying to push ANOTHER OGM incarnation onto the masses and in less than 2 years of each other?! That is just the absolute freakin HEIGHT of stupid.

It's not another OGM implementation or even an incarnation thereof, it's an entirely new spec.

Does it add to the diversity out there? Certainly. May the best container win.

Not only is it most likely gonna be easily exploitable but since the amateur programmers at Doom9 are gonna be building, it's most likely gonna be massively unstable on top of it all. Tab himself has already gone on about how he thinks OGM is unstable, and it's workin off a Direct Show filter! o_O

libmatroska and libebml are very stable, even at this early stage. I encourage you to try them for yourself.

Given how well the "security" on the DVD format worked out, it's most likely that producers are gonna want as much security as they can get, encoded into the video stream itself, the container, the transport mechanism, etc, etc. And trust me on this one, they're NOT gonna want something put together by some kids on a group called *snicker* "Doom9" that's based on vastly unsupported and untried programming theory.

That's funny that you say that. CSS was an untried, home-brewed cipher; look what came out of it

Yes, EBML is an untested extension of XML. We don't really know what will come out of it just yet.

However I don't see how you can support OGM and blast everything else, esp. Matroska, when both are very much untested and produced by the same "amateur programmers".

*snicker*

I think maybe I'll quote you and include some links for Sugien, I'm sure he'd love to take a crack at it. Java is about as secure as the ICQ protocol when you get right down to it and Sugien has found ways to do some rather malicious things with it in the past. ^_^

Please elaborate. I've worked with Java for quite a while now and have been quite impressed at the safety features that the VM offers.

Nothing huh? Heh, what I'm working on can, only it takes it a few steps beyond anything the kids at Doom9 have thought of. ^_^

Please provide code and/or specifications. I am capable of reading both, and the chances of me stealing your ideas are nil, as I have no interest in doing so.

Want me to sign an NDA? Provide it. I will verify the company and your employment there. If you are not qualified to provide an NDA, please provide me with the contact information of someone who can, as well as your name so that I can provide some basis for my request.


You see, I demand evidence, not just words. I provide it for you, and I expect you to provide me the same courtesy.