(01/09/2010 03:31 PM)ZiNgA BuRgA Wrote:  The video probably won't compress well, however the file structure (container) probably has a lot of redundancy.  Open one in a hex editor and you'll probably find a fair amount of null (0x00) bytes.  Not exactly sure what the padding is for, perhaps there's something to do with attaining CBR.

126/189 = 0.6772486...

Overhead equal to about ~50% of data?  Don't remember program streams to have so much overhead, sounds like a goddamn transport stream.

(01/09/2010 03:31 PM)ZiNgA BuRgA Wrote:  It's lossless, and there's always a size premium for lossless encoding, whether it's video (x264 lossless vs crf20), audio (FLAC vs AAC) or image.  And for the images you're testing, JPEG can probably throw off a lot of redundancy.

Not only PNG, I dislike lossless on a whole.  Lossless pretty much screams waste, and the more complicated the medium, the more waste.  And no normal human can tell the difference between high bitrate lossy and lossless anyway, under normal circumstances.

(01/09/2010 03:31 PM)ZiNgA BuRgA Wrote:  Also you don't mention bit-depth.  I believe PNG has the ability to use 16-bits per channel, along with a full alpha channel, none of which JPEG can achieve.

Which is useless for everything besides for some very limited applications (and for those, you may as well use goddamn RGBA).  Really, have you ever even seen a 64bit image?  I mean 99% of the stuff wee get these days are 24bit or 32bit (dubbed "true color").  Someone needs to design something that supports 256bit, just so they can go "my fucking image is better than yours" and "I'm 200 years ahead of time".

Also, in the comparison, I certainly did not use 64bit, so the results aren't skewed.  (Unless PNG is retarded and automatically upsamples my stuff, which I doubt).

(This post was last modified: 01/09/2010 05:04 PM by Assassinator.)

(01/09/2010 04:54 PM)Assassinator Wrote:  Not only PNG, I dislike lossless on a whole.  Lossless pretty much screams waste, and the more complicated the medium, the more waste.  And no normal human can tell the difference between high bitrate lossy and lossless anyway, under normal circumstances.

That's just merely your opinion of things really.  Lossy is only as good as the algorithm, and I can certainly say that algorithms are nowhere near perfect and there are probably many examples of them totally failing (JPEG fails quite hard on simple or high detail *clean* CG content for example).
Besides, it has applications more than just for viewing.  Ever tried Photoshopping a JPEG?  Upscaling is also easier when you don't have to worry about all the noise of JPEG. (hq3x low res CG :P)  And people pay for perfection.

(01/09/2010 04:54 PM)Assassinator Wrote:  Which is useless for everything besides for some very limited applications (and for those, you may as well use goddamn RGBA).

Don't be silly.  A format gives capabilities.  It's up to the user and application to properly use them.

(01/09/2010 04:54 PM)Assassinator Wrote:  Really, have you ever even seen a 64bit image?  I mean 99% of the stuff wee get these days are 24bit or 32bit "true color".

The eye cannot distinguish more than 24bit colour (no alpha), however, as mentioned earlier, there may be more applications than for simply viewing.  Image manipulation may make use of it, and various other things (eg imaginary colours or colours which exceed defined limits, eg a whiter white).

(01/09/2010 04:54 PM)Assassinator Wrote:  Also, in the comparison, I certainly did not use 64bit, so the results aren't skewed.

You may have used 32bit, which means an unused alpha channel.
Are your BMPs 24bit?
JPEG uses something like YUV, which obviously can't be used for a (truly) lossless format.

(This post was last modified: 01/09/2010 05:18 PM by ZiNgA BuRgA.)

(01/09/2010 05:15 PM)ZiNgA BuRgA Wrote:  That's just merely your opinion of things really.  Lossy is only as good as the algorithm, and I can certainly say that algorithms are nowhere near perfect and there are probably many examples of them totally failing (JPEG fails quite hard on simple or high detail *clean* CG content for example).

It's more about the people using the it than lossless itself.  Having the capability to do lossless is, ofcourse, good.  Having the capability to do anything is always good.

But I do not want to download 3MB for people's shitty 1080p screencaps, I'm going to stare at for 3 seconds before moving on.  I also don't want to download 100+MBs of fucking FLAC in every one of my anime episodes.  And maybe one day in the future, someone's going to come up with the smart idea of releasing all their poo poo in x264 lossless...

Also, there's a general misconception about JPEG being more horseshit than it actually is.  It's just that some software use really low default settings, and people don't have enough brains to figure that out and/or change it.  If you use high settings, you can't really tell the difference most of the time, besides for some special types of content JPEG sucks at.

But PNG is only the least of the "waste of lossless" (also, JPEG does sort of suck, in terms of the technology behind it).  Lets not get into WAV/FLAC and god forbid, lossless video.

(01/09/2010 05:15 PM)ZiNgA BuRgA Wrote:  Besides, it has applications more than just for viewing.  Ever tried Photoshopping a JPEG?  Upscaling is also easier when you don't have to worry about all the noise of JPEG.  (hq3x low res CG :P)

Yeah, that's for internal stuff.  When you're putting stuff on the net, noone really cares.

As for 3x upsizing low res CG, it's going to look like a pile of poo poo even if you have 256bit color.  Welcome to the world of aliasing.

(01/09/2010 05:15 PM)ZiNgA BuRgA Wrote:  And people pay for perfection.

You mean people are making you pay for their perfection which you don't give a rats donkey about?

(01/09/2010 05:15 PM)ZiNgA BuRgA Wrote:  Don't be silly.  A format gives capabilities.  It's up to the user and application to properly use them.

And people tend to miss-use them.

Like x264 just newly got 10bit encoding.  But noone should ever use that, because unless you work at a filming studio, you probably can't get input material higher than 8bit.

(01/09/2010 05:15 PM)ZiNgA BuRgA Wrote:  You may have used 32bit, which means an unused alpha channel.
Are your BMPs 24bit?

Don't know, can't check right now, but probably 24bit.  But if my input images are in fact 24bit, and PNG "turns" them into 32bit by adding an empty alpha channel, that wouldn't be my fault.

(This post was last modified: 01/09/2010 06:11 PM by Assassinator.)

(01/09/2010 05:59 PM)Assassinator Wrote:  But I do not want to download 3MB for people's shitty 1080p screencaps, I'm going to stare at for 3 seconds before moving on.  I also don't want to download 100+MBs of fucking FLAC in every one of my anime episodes.  And maybe one day in the future, someone's going to come up with the smart idea of releasing all their poo poo in x264 lossless...

Most of these people probably live in the US/UK which have unmetered internet connections starting at 3Mbps, some going up to 100Mbps.  The difference between a 40MB album and a 100MB album is about 2.5 minutes of extra download on a "low end" 3Mbps connection, and those people on this connection probably don't mind the slight extra wait.  On a more reasonable 12Mbps connection, the 3MB lossless file takes like less than 2 seconds longer to load than a 300KB JPEG.  Honestly, no-one cares.
You, on a slow connection, are really just a very small fragment/niche of the internet really, and most people really don't care.

(01/09/2010 05:59 PM)Assassinator Wrote:  Also, there's a general misconception about JPEG being more horseshit than it actually is.  It's just that some software use really low default settings, and people don't have enough brains to figure that out and/or change it.  If you use high settings, you can't really tell the difference most of the time, besides for some special types of content JPEG sucks at.

Most applications don't offer the option.  MSPaint doesn't, the Office suite doesn't, many digital cameras don't.  And for apps that do, they don't always make it obvious.  But most aren't that low - typically around 70-85% from what I've seen.
And JPEG can be pretty poo poo, even on 100% quality setting.  JPEG is good for photos and similar content, but on simpler content etc, it makes very obvious defects in the image.

(01/09/2010 05:59 PM)Assassinator Wrote:  Yeah, that's for internal stuff.  When you're putting stuff on the net, noone really cares.

Games may have to do image filtering etc, which means that they may distribute files with extra info.

(01/09/2010 05:59 PM)Assassinator Wrote:  As for 3x upsizing low res CG, it's going to look like a pile of poo poo even if you have 256bit color.  Welcome to the world of aliasing.

I'm talking about hq3x.

(01/09/2010 05:59 PM)Assassinator Wrote:  And people tend to miss-use them.

Still not the format's fault.

(01/09/2010 05:59 PM)Assassinator Wrote:  Don't know, can't check right now, but probably 24bit.  But if my input images are in fact 24bit, and PNG "turns" them into 32bit by adding an empty alpha channel, that wouldn't be my fault.

It may be your fault if you "don't have enough brains to figure that out and/or change it", or the application's fault.

(01/09/2010 06:36 PM)ZiNgA BuRgA Wrote:  Most of these people probably live in the US/UK which have unmetered internet connections starting at 3Mbps, some going up to 100Mbps.  The difference between a 40MB album and a 100MB album is about 2.5 minutes of extra download on a "low end" 3Mbps connection, and those people on this connection probably don't mind the slight extra wait. On a more reasonable 12Mbps connection, the 3MB lossless file takes like less than 2 seconds longer to load than a 300KB JPEG.  Honestly, no-one cares.

Even if I had nice internet (will have something not excellent, but decent, soon), waste is always bad (as a matter of principle).  Still don't want to download 120MB of FLAC every 24 minute episode of anime (or even worse, 250MB of 5.1 FLAC).  Also a waste of HDD space.

Lol, since this thread is pretty much just between you and me anyway... Lol, my father still wants to go with Telstra, at least for the moment, until our current contract expires.  100GB for $88 including phone, he went and argued a bit with them, and they agreed to use our current contract (few months remaining) instead of starting a new one.  Will figure out whether to keep or change afterwards (I like that $40 60/160 on/off plan, $70 with phone).

(01/09/2010 05:59 PM)Assassinator Wrote:  And JPEG can be pretty poo poo, even on 100% quality setting.  JPEG is good for photos and similar content, but on simpler content etc, it makes very obvious defects in the image.

Make the choice based on the data then.

Heh, wish wee had some better image format than something from 20yrs ago.  Need something equivalent to lets say, x264's intra, then a lot of those problems would be solved.  Actually, there probably is something better, just no good support for it (and i don't mean JPEG2000, that's not really better... well, not strictly anyway).

(01/09/2010 06:36 PM)ZiNgA BuRgA Wrote:  I'm talking about hq3x.

Emulator resizing?  Well, most people don't have a need for using this.

(01/09/2010 06:36 PM)ZiNgA BuRgA Wrote:  Still not the format's fault.

Didn't say it was really the format's fault.  It's (always) the people's fault.

Only thing I said about the format was it not giving enough compression benefits (which does matter, but it's old, so meh), and being slow (which doesn't really matter...much).

(01/09/2010 06:36 PM)ZiNgA BuRgA Wrote:  

(01/09/2010 05:59 PM)Assassinator Wrote:  Don't know, can't check right now, but probably 24bit.  But if my input images are in fact 24bit, and PNG "turns" them into 32bit by adding an empty alpha channel, that wouldn't be my fault.

It may be your fault if you "don't have enough brains to figure that out and/or change it", or the application's fault.

If I feed it 24bit in, I expect 24bit out.  That should be natural, it's not something that should (need to) be changed.  If I need to actually change some setting in order for the compressor to return the same bit depth as I gave it, that's obviously the compressor's fault.  Assuming it supports that bit depth ofcourse, but anything that doesn't support 24bit (the most common of them all) is retarded.

As for the application's (Irfanview's) fault, no idea.

(This post was last modified: 01/09/2010 08:30 PM by Assassinator.)

(01/09/2010 07:23 PM)Assassinator Wrote:  Even if I had nice internet (will have something not excellent, but decent, soon), waste is always bad (as a matter of principle).  Still don't want to download 120MB of FLAC every 24 minute episode of anime (or even worse, 250MB of 5.1 FLAC).  Also a waste of HDD space.

I would - have heaps and heaps of HDD space that's going to be left there to waste if it isn't used.  Why bother yourself with something which may be substandard?  FLAC guarantees quality, whereas AAC could've been badly encoded (eg uguu batches of Air) and sound absolute poo poo.  Blame the group if you wish, but from the perspective of a user, there's less that could go wrong with FLAC.  (as far as the group is concerned, most people who download BD rips only do so for quality rather than filesize (or uncensoring maybe) so I think FLAC can be justified)
Plus, I can chop out the songs and have nice lossless quality.

(01/09/2010 07:23 PM)Assassinator Wrote:  Heh, wish wee had some better image format than something from 20yrs ago.  Need something equivalent to lets say, x264's intra, then a lot of those problems would be solved.  Actually, there probably is something better, just no good support for it (and i don't mean JPEG2000, that's not really better... well, not strictly anyway).

There are very few encoders for JPEG2000 (are there any free ones?) and support is very limited.
Ultimately, compatibility is king, and there never has really been a need for a better format, so it will be unlikely to change unless something forces it.  Wee still use x86 CPUs because of compatibility.  Wee use Windows because of compatibility.  Both do the job fine, so any reason to change?  Same applies to images.  They're so small and insignificant that no-one particularly cares.  Audio compression is already pretty much like that - MP3 dominates and there's not much significant development otherwise.  FLAC is small enough for many people that it can be considered accessible.
Video, on the other hand, may be considered significantly large to justify codec changes, though Xvid in AVI is still quite popular.

(01/09/2010 07:23 PM)Assassinator Wrote:  Emulator resizing?  Well, most people don't have a need for using this.

No, just a resampling algo.
Upscaling works better if the source is clean (ie not JPEG).

(01/09/2010 07:23 PM)Assassinator Wrote:  If I feed it 24bit in, I expect 24bit out.  That should be natural, it's not something that should (need to) be changed.  If I need to actually change some setting in order for the compressor to return the same bit depth as I gave it, that's obviously the compressor's fault.  Assuming it supports that bit depth ofcourse, but anything that doesn't support 24bit (the most common of them all) is retarded.

As for the application's (Irfanview's) fault, no idea.

I don't think IrfanView does 32bit actually, so it probably is 24bit.

At home now, looking at the subarasikihibi_op.mpg file you gave:

Some padding, but it does seem most is compressed.
subarasikihibi_op.mpg   42,657,796 bytes
subarasikihibi_op.7z (LZMA2)   39,567,054 bytes
subarasikihibi_op.rar   39,922,378 bytes

So LZMA2 wins here, but I don't get huge compression benefits.

Looking at the byte distribution (JPEG for your convenience):
subarasikihibi_op.mpg

As you can see, fairly uneven, though there aren't exactly a whole heap of 0x00 and 0xFF bytes.  Entropy coding has the effect of smoothing out the byte distribution (a perfect compressor probably should give a flat distribution).

subarasikihibi_op.7z

subarasikihibi_op.rar


So as you can see, LZMA2 achieves a better distribution here.

(03/09/2010 07:01 PM)ZiNgA BuRgA Wrote:  So LZMA2 wins here, but I don't get huge compression benefits.

If you're interested in looking at something that does give significant compression benefits, you can check out the Sharin no Kuni OP.  You can find Sharin no Kuni in the folder "VN Stash" where I have all my VN stuff.  Moved it to shared.

(03/09/2010 07:01 PM)ZiNgA BuRgA Wrote:  Entropy coding has the effect of smoothing out the byte distribution (a perfect compressor probably should give a flat distribution).

So as you can see, LZMA2 achieves a better distribution here.

Upon taking a closer look at the 2 thing for which RAR significantly beat 7z, I found that the exact location where RAR is getting it's advantage is in raw PCM data.  While on the newer games, the BGM, and more importantly, the voice, is usually compressed.

I actually unpacked a few games that are trivial to unpack (have existing tools), and found their audio to be in vorbis most of the time.  Image formats vary a lot.  Usually something like PNG, but I see JPG, BMP, and stuff like TLG (KiriKiri2's proprietary format?).

(This post was last modified: 04/09/2010 03:04 AM by Assassinator.)