Post Reply 
[useless thread] gzip compression
Author Message
ZiNgA BuRgA
Smart Alternative

Posts: 17,024.1882
Threads: 1,174
Joined: 19th Jan 2007
Reputation: -1.71391
E-Pigs: 446.1887
Offline
Post: #20
RE: [useless thread] gzip compression
(31/08/2010 11:06 PM)Assassinator Wrote:  
(31/08/2010 10:49 PM)ZiNgA BuRgA Wrote:  - even if media is compressed, most games group stuff into large files which hold this compressed media, which usually have a lot of redundancy in them (long bits of null padding, structure is almost never compressed)

Even so, I can't really image redundancy making up even close to the 20% or so of compression that usually occur.  Must be either the images or the WAVs (since the MPEG1 is not going to compress at all, neither is the voice).
Depends on the file structure and how much meta-info there is (plus hashtables for fast file access etc).  The PSP's RCO format, for example, can have lots of smaller files, and has a fair bit of meta/structural info, along with hashtables to speed up finding files, so compressing the header with deflate can reduce the file size a fair bit.
There's also the ISO file format:
- there are a fair bit of redundancies in a filesystem
- internal fragmentation if there are a lot of files
IMG+CCD / BIN+CUE include RAW data, which, unless the game uses it, is probably just ECC data, which I would assume has fairly high entropy, so won't compress.

Also remember that long sequences of null bytes get compressed to practically nothing after dictionary coding.



(01/09/2010 03:00 AM)Assassinator Wrote:  The images compressed the most, as expected.  Now that I think about it, they're probably in BMP format.
They almost always are, from what I've experienced.

(01/09/2010 03:00 AM)Assassinator Wrote:  What's surprising is that the goddamn video compressed.  I mean isn't pretty much any video codec supposed to do some form of entropy encoding at the end?  Then you shouldn't really be able to compress it more at all.   MPEG1, I are dissapoint!!!
The video probably won't compress well, however the file structure (container) probably has a lot of redundancy.  Open one in a hex editor and you'll probably find a fair amount of null (0x00) bytes.  Not exactly sure what the padding is for, perhaps there's something to do with attaining CBR.

(01/09/2010 03:00 AM)Assassinator Wrote:  PNG just sucks (never liked PNG much, this doesn't help).  It's also fucking slow to encode to (takes like 5x longer than the other 2 types, and that's not even using the slowest method).  Not to mention 55% gain over BMP is really quite dissapointing.
It's lossless, and there's always a size premium for lossless encoding, whether it's video (x264 lossless vs crf20), audio (FLAC vs AAC) or image.  And for the images you're testing, JPEG can probably throw off a lot of redundancy.
Also you don't mention bit-depth.  I believe PNG has the ability to use 16-bits per channel, along with a full alpha channel, none of which JPEG can achieve.
(This post was last modified: 01/09/2010 03:32 PM by ZiNgA BuRgA.)
01/09/2010 03:31 PM
Visit this user's website Find all posts by this user Quote this message in a reply
Post Reply 


Messages In This Thread
RE: [useless thread] gzip compression - ZiNgA BuRgA - 01/09/2010 03:31 PM

Forum Jump:


User(s) browsing this thread: 1 Guest(s)

 Quick Theme: