Thank you very much for PNGout. It's an excellent utility that I use quite frequently. What it does, it does very well. Unfortunately, I often find that it does far to good a job and consequently takes far too long to do so.
While I admire its ability to create the smallest possible file, I often find myself wishing that it would just "give up" on trying to save those last few bits. I would be interested in seeing a "quick mode" that would mearly create a decently compressed file instead of a perfectly compressed file. My processor is a long way from the cutting edge, and squeezing out those last 3 bytes usually ends up taking a full three extra minutes.
A perfect example of where this would be useful is creating screenshots to quickly send someone in an email or over an instant messager. The /i tag is obviously very useful for this, however the extra time invested making the compression perfect is hardly worth it for these single use .PNG's.
I realise some other utilities offer this ability and I could just as easily use them (except they can't load bitmaps straight from the clipboard), but I like the beauty of a simple all in one solution.
Awesoken at
I often find myself wishing that it would just "give up" on trying to save those last few bits.
I did a quick hack so if you press 'ESC' during compression, it will now quit as soon as it finishes the current pass. Thanks for the suggestion.
AcetoliNe at
Yeah, I've been wanting this feature too. Thanks for adding it, Ken.
Parkotron. at
Thanks for the quick response and adding the "ESC to quit after current pass" functionality.
However, ideally I would like a quickmode that would run unsupervised. For example, when using a batch compress. I've thought over how best to implement this, and I think that a "number of passes" argument would work very well.
It could have the form /p#, where # is the number of passes to perform before quitting. /p0 would the default and take as many passes as possible. /p1 would be a "quick and dirty" compression, while /p3 would be a decent, but somewhat speedy compression.
Anyway, just a suggestion. By the way, the changes made to pallets and bitdepths are excellent. Two colour PNGs just got a whole lot smaller!
Awesoken at
There's already a way to speed up PNGOUT without user intervention - an undocumented feature actually. If you use negative values for /b#, it will disable the pattern compression, and rely solely on Huffman encoding for compression. (The blocks are still split according to the absolute value of the /b# value) Sometimes, this gives smaller files, but it's pretty rare. So far, I've only found it helpful for photographic images that have a lot of randomness in the lower bits of significance (usually caused by noise from an old scanner). The other side effect of using /b# with negative values is it will cause PNGOUT to always quit after the 2nd pass - which is exactly what you want.
BarrenSoul at
yay ken :D upgrades and flexability always make for a good program even though it was already an awsome prog!
Awesoken at
I've been cleaning up my PNGOUT code recently and released a new version a few days ago. I moved around a lot of code, so hopefully I didn't break anything. : ) The new features are:
* PNGOUT now replaces PNG files - and only if the result is smaller. My code no longer generates weird filenames with underscores (i.e. mypic_.PNG) or .PN2 extension.
* Huffman-only compression is now MUCH faster. This is an undocumented feature that's been around for a while. To use it, specify negative values for the /b# option. The "Block split threshold" will then use the absolute value of this number. For example: "pngout /i mypic /b-256" will write mypic.png from the windows clipboard fairly quickly. Very few images compress well in Huffman-only mode, so I don't recommend it for general use - use it only when speed is a major concern.
counting_pine at
Nice, now I can run PNGOut without the hassle of having to check whether it's made the file bigger. Is there an option to turn it off though? Sometimes PNGOut is useful for converting images to a different colour type.
Have you though about adding in code to try a few tricks on the original image? For example, removing unnecessary chunks, combining multiple IDATs, removing unused colors off the end of the PLTE.
Awesoken at
Is there an option to turn it off though?
There is now : ) I thought about doing this, but wasn't sure if anyone was actually using PNGOUT in this way. Now that you have brought it to my attention, I have fixed it. Use "pngout /force ..." to force it to write the output file even if it's bigger.
Have you though about adding in code to try a few tricks on the original image? For example, removing unnecessary chunks, combining multiple IDATs, removing unused colors off the end of the PLTE.
These sound like the tricks of PNGREWRITE (by Jason Summers). In fact, PNGOUT uses all these tricks. By default, PNGOUT keeps only the necessary chunks. You can use the /v option to see exactly which chunks are being kept and which ones are being removed. PNGOUT always writes exactly 1 IDAT. My code always remove unused palette entries - and moves any translucent colors to the beginning of the palette. I go one step further than most programs by moving around palette indices if they allow the Huffman table to be stored more compactly. This sometimes saves a byte or 2 : ) There are other tricks I know of that could improve the compression, but none of them are practical. I don't want to make PNGOUT any slower than it already is : )
counting_pine at
Awesoken said
PNGOUT uses all these tricks.
Only on the images it generates itself. If the original image is smaller, then PNGOut won't perform any of them, just copy the original byte for byte, even with /k0 on the command line.
It probably doesn't help on most images, but checking for multiple IDATs and seeing if the PLTE/tRNS can be reduced (by seeing if there are redundant entries at the end) are compuationally cheap, compared to recompressing the IDAT. And they're guaranteed to not increase the image size.
(By the way, I'm impressed how quickly you update your software:)
BarrenSoul at
so you have things that can make it smaller? why not implement them under a solo feature called High compression? so if the user dosn't want to wait they can just leave it out? some people obssess over size and some over speed :)
Gast at
Filter trials
Generally, PNGOut gives better results than OptiPNG for me: As the compression algorithm works great, there is no need for trying different methods. Still, I wonder how PNGOut selects its filter if /f# is not specified: Does it use any heuristics? And would it be worthwhile to try out all possible filter options?
dx at
PNGout: is a quick mode a possibility?
Indeed, if you know of other methods to make the files smaller, don't worry about speed (or if you are worried, make the slower methods optional). :) I generally use pngout as a "last step" before publishing websites, and with the new feature where it keeps the smaller file if the file it produces is larger I can easily automate it via batch-files and let it run for however many hours it needs to process all the files.
Another thing you might consider is something of a brute-force mode (ala pngcrush) which would automate trying different "/b#" values, then giving you the smallest file generated (or the original if the new file is larger).
Thanks for this cool utility Ken!
Awesoken at
First, more PNGOUT news:
* New version with option to preserve palette order: /kp
* Updated PNGOUT tutorial * A PNGOUT plug-in for Irfanview is on the way ... and it will feature ... (drumroll) ... a dialog box! : )
Now to answer your questions:
even with /k0 on the command line
Putting /k0 on the command line does absolutely nothing. It is the default.
checking for multiple IDATs and seeing if the PLTE/tRNS can be reduced (by seeing if there are redundant entries at the end) are compuationally cheap
Are you asking me to replace PNGREWRI? Yes, it would be cheap, but also not very effective - unless the program that generated your PNG file is really stupid : P I don't intend to replace PNGREWRI.
so you have things that can make it smaller? why not implement them under a solo feature called High compression?
You should know that it's slower than turtles crawling backwards in a pool of molassis. It just isn't worth the 0.1% (if you're lucky) decrease in file size. Even so, I'm thinking about releasing this super slow mode in the form of an undocumented option.
I wonder how PNGOut selects its filter if /f# is not specified: Does it use any heuristics? And would it be worthwhile to try out all possible filter options?
Here's a quote from my updated PNGOUT tutorial:
"The default value is /f0 for palette images (/c3), and /f5 for everything else."
Another thing you might consider is something of a brute-force mode (ala pngcrush) which would automate trying different "/b#" values, then giving you the smallest file generated (or the original if the new file is larger).
Yeah, trials is on my todo list. I plan to support something like: /f1-3,5 which would try /f1, /f2, /f3, /f5. I think running through a list of 1000 trials is a bad idea for PNGOUT because it is so much slower than pngcrush/optipng. Also, my block split threshold (/b#) is a floating point value, so that makes it impossible to try all the cases. The best solution would be to add some kind of intelligence to which trials to attempt.
My initial attempt at intelligent trials made some assumptions like: If /f3 is smaller than /f4 in one mode, then it must be better in all modes. Obviously, that's an approximation, but as it turns out, it's a pretty bad one. Things as simple as changing the color mode or the bit depth can unlock the potential of a certain mode. What I'm saying is: if the results are close, you really shouldn't be eliminating a whole branch of cases. Ideally, I would want my trial algorithm to work similar to the way I do it by hand. BTW, this could all be done with an external script. Is anybody interested in writing an intelligent script - which I can later incorporate into my own code? : )
counting_pine at
Awesoken said
Are you asking me to replace PNGREWRI? Yes, it would be cheap, but also not very effective - unless the program that generated your PNG file is really stupid : P I don't intend to replace PNGREWRI.
PNGREWRITE would take your super-optimised file - so optimised that not even PNGOUT could beat it - recompress the IDAT using a different method that probably leaves it taking up more space, then split it up into 8192 byte chunks. No, I don't want PNGOUT to replicate this functionality.
These are all the things I can think of that might shrink an image without having to redeflate the IDAT:
Combine all the IDAT chunks (OptiPNG tries this)
Remove anciliary chunks
Truncate the PLTE and/or tRNS if the end colours aren't used in the image
Truncate the tRNS if the last entries are all FF.
I don't know any program that tries to do all these.
Awesoken said
You should know that it's slower than turtles crawling backwards in a pool of molassis
LOL
Got any numbers? How long would it take to do unionjack.png?
Awesoken said
A PNGOUT plug-in for Irfanview is on the way ... and it will feature ... (drumroll) ... a dialog box! : )
Nice! Especially since Irfanview is guilty of being "really stupid" when it comes to saving PNGs.
Awesoken at
These are all the things I can think of that might shrink an image without having to redeflate...
I didn't realize that PNGREWRI re-deflated the file. I looked at the code very quickly and then let my imagination take over on how it was implemented. Somehow, I overlooked the author's note about requiring the LibPng & Zlib libraries (and all the external function calls it used).
I see your point now. Yes, PNGOUT does all the non-deflate related things you mentioned. Unfortunately, there's no easy way to disable the re-deflate easily. It wouldn't be much easier than just writing a whole new utility from scratch. I know of another program called ADVDEF which does the re-deflate and none of the other stuff ... which is sort of the opposite of what you want :)
Got any numbers? How long would it take to do unionjack.png?
The 2 tricks I know of to improve compression are:
* Full brute-force deflate
* Randomizing the initial Huffman tables slightly and then doing lots of trials
Here's a comparison:
Public PNGOUT version:
3028 bytes, 3.197 sec
With slow deflate (unreleased):
3015 bytes, 95.286 sec
(A savings of 0.43% at a cost of 30 times longer than the public version)
With slow deflate, randomized initial Huffman tables, and taking the best of 100 or so trials (also unreleased):
3013 bytes, I assume it was about 95 seconds per trial
(A savings of 0.50% at a cost of about 3000 times longer than the public version)
BTW, during my trials, I got a distribution something like this:
mean=3015 bytes, standard deviation:1 byte
I got a 3013 byte file once every 20 or so trials.
The point is, you really aren't missing much.
BarrenSoul at
yeah that might not seem like alot ken but don't forget how many visitors some sites get! over 10k a day = 130kb (per image and some have more than one) which is 3.9 megs per month on the site, it all adds up, although not to many do use png (some images might get even better returns then the one you tested)
try the supper slow one on the large test image (the 16mil.png that I used) and see what happens.
does pnout use octree by default? this one image I had worked better with median cut on paint shop pro and then was 9% smaller after pngout, how do I specify median cut? (although median cut mage the image darker... anyone tell me why?)
MxxCon at
Ken, thank you for such an excellent util.
a while ago i found an interesting post here http://www.numbera.com/forums/viewtopic.php?p=49#49
do you think dynamic size detection could be added to pngout?
also regarding 'insane-slow' compression mode
would cpu-specific(p4, athlon, etc) optimizations/compiles help?
Awesoken at
do you think dynamic size detection could be added to pngout?
ShadowCaster proposes an interesting algorithm for searching for the optimal value of /b#. This could be incorporated into my intelligent options finder (that is, if I ever get around to implementing that).
A much better way to decide on whether it's worth it to split a block would be to generate the actual Huffman tables and compare the sizes directly.
regarding 'insane-slow' compression mode would cpu-specific(p4, athlon, etc) optimizations/compiles help?
The bottleneck of my "slow deflate" is the pattern matching and comparing all the cases. Unfortunately, my code is currently full of branches so I don't see where I could use SIMD to accelerate it. There may exist a much better algorithm for what I'm doing ... but I haven't been thinking about it much lately.
counting_pine at
BarrenSoul said
does pnout use octree by default? this one image I had worked better with median cut ...
(A couple of quick Googles later...)
Aren't octree and median cut both lossy algorithms to convert a full-colour image to 256 colours? If they are, then PNGOUT wouldn't be using either, since it's a lossless converter. If you only want to "losslessly" (if that is a word) generate a palette for an image with 256 colours or fewer, I expect there are other algorithms. I'm not an expert, but I'd guess that they'd be a lot simpler than lossy colour-reduction ones.
markcramer at
I had a look at the post referenced above, and decided to play with the /b parameter on a few images, and it appears that "it might not be that simple" to find the optimum value for /b.
The size curve isn't a simple U shape, it starts and finishes at the same point (ie for b=0 and b sufficiently large). The size increases from b=0 to b~40-80 then drops again, sometimes lower than the b=0 value, sometimes not, but the curve isn't smooth after the peak, there can be multiple minima, before the curve returns to the original value at large b. On every image I've tried, the first minima after the peak is the deepest, though not necessarily smaller than b=0
Here are the size values for a range of b's for one image: The minimum is at b=383, but there is another minima at b=388-395 and another around 960-1024.
Reading through your forums I have come to understand that I am not alone in running PNGOUT multiple times with varying parameters...in order to get the smallest PNG's possible. I personally have a script that runs pehaps 100+ iterations...selecting the smallest afterwards. As I can imagine, initial image reading, decoding and final writting to disk of the image are duplicate efforts (among possibly other tasks perfomed by PNGOUT). Having PNGOUT smartly do these iterations (saving only the smallest output to disk) would save me and others an enourmous amount of time and efforts (and hard disk wear and tear). I understand that you are working on optimizing both "fast" and "slow" modes of PNGOUT, but could you release the "slow'" mode anyway? Its not uncommon to optimize both best and worst cases of an algorithm after release. I do not think anyone would think any less of the product, but you would save me and others alot of time doing meta optimizations work "by hand".
Thank You for your consideration and great job on your software.
Awesoken at
PNGout: is a quick mode a possibility?
Yes, I can enable the super slow mode for the next version - whenever that is. I'll probably choose some very long name for it - and not advertise it heavily : )
Anonymous at
As long as you advertise it here.... :-)
Please.
Mark
KillerQ13 at
I like how the discussion went from a quick mode to a super slow mode.
BarrenSoul at
I'll graph the values latter (will need more samples if anyone cares to contribute) also I will make a few of my own. from what I can see it dosn't follow a sinusoidal pattern, it looks more like the pattern prime numbers show, every once in a while a strange number just pops up in the pattern :) and the only thing that seems to (although dosn't) follow that pattern is the riemann zeta function http://www.claymath.org/millennium/Riemann_Hypothesis/
counting_pine at
markcramer, I'm assuming this is the image you used.
I can't reproduce your results exactly, so I guess you started with a different palette order.
I suggest anyone who wants easily reproducible results on the same image does something like:
for /l %i in (0,1,1024) do pngout /b%i /c3 half_res_rgb.gif half_res_rgb_%i.png
(I think you need an NT-based Windows to get the advanced 'for' command to work)
Awesoken at
Wow, after all these years, I never thought to look at the /l option of the "for" command. Thanks for showing that, counting_pine! This is great because it means I can tell people how to run many trials without having to change my PNGOUT syntax : P I'll probably add it at some point anyway just to clean things up. Did you know you could nest for /l commands together? : ) Check it out:
for /l %i in (0,16,1024) do for /l %j in (0,1,5) do pngout /b%i /f%j mypic.png
This particular example runs through 65*6 = 390 trials (/b0, b16, /b32, /b48, ... /b1024 and /f0, /f1, ... /f5)
Happy batch file making! : P
counting_pine at
Oh, that reminds me: If you're working in a batch file, you have to use two %'s instead of one.
eg.
REM Tries all filters on mypic.png
for /l %%i in (0,1,5) do pngout /f%%i mypic.png
markcramer at
CountingPine, yes that was the image I was using, though it had already been converted to a png, which may explain the different palette.
Here is my current batch file, it currently tries /b 0,32,64,96,128,192,256,384,512,768,1024,1536,2048,3072,4096,6144,8192,... etc out to 65536 and stops if it hits a file the same size as b=0. The /s parameter lets me set whether I want to try f=0, f=5, f=0&5 or f=0 to 5 the /r parameter lets you add an extra range of b values, /r start step end. This got a lot more complex at one stage and could probably do with being simplified :-)
I can't believe how much time I spend trying to recompress png's, now if only I had something faster than a P3 750 to work with.
BTW Ken, do you use the standard heuristic for picking the filters in /f5 mode, I would have thought that /f5 should ALWAYS beat the other /f values, but sometimes one of the others will be smaller, have you though about making Pngout even slower by trying a couple of other algo's to vary the filters chosen in /f5 mode to see if that produces smaller files :-)
@Echo Off
Echo PngOutMx [/s 0(0,1,0)/5(5,1,5)/a(0,1,5)/b(0,5,5)] [/r Start Step End] File
If "%~1" Equ "/s" (
If "%~2" Equ "0" Echo 0,1,0 >C:\Temp\PngoutMx.Par
If "%~2" Equ "5" Echo 5,1,5 >C:\Temp\PngoutMx.Par
If "%~2" Equ "b" Echo 0,5,5 >C:\Temp\PngoutMx.Par
If "%~2" Equ "a" Echo 0,1,5 >C:\Temp\PngoutMx.Par
Shift&Shift
)
Set Rng=32,8,32
If "%~1" Equ "/r" Set Rng=%2,%3,%4& Shift& Shift& Shift& Shift& Echo !Rng!
Set Par=0,1,5
Set Sz0=0
Set Sz=1
If Exist C:\Temp\PngoutMx.Par For /f "delims=
Awesoken at
Nice batch file ... a little too complex for me to spend time figuring it out though : )
...do you use the standard heuristic for picking the filters in /f5 mode ... have you though about making Pngout even slower by trying a couple of other algo's to vary the filters chosen in /f5 mode ...
The standard mixed filter algorithm compares the 5 filters by adding the distance of the bytes from 0 (absolute values of signed chars) and simply picks the filter with the lowest sum. My mixed filter algorithm is different. I calculate the entropy (log(f)*f) of each line and choose the filter with the lowest entropy. It's difficult to say which algorithm is actually better because there are a lot of factors at play.
BarrenSoul at
-_-' give me about an hour if I knew what the syntax was for the batch file scripting langauge and I might be able to figure it out... or just get one hell of a bad ass headache
markcramer at
Wow, is it really that bad?
I've been writing semi-complex and complex batch files like this since Dos 3.1 days (yeah I'm getting old) I like the extensions that got shoved into the Win2K command interpreter, pity they were only half done.
To make understanding it a bit easier for the subroutine at the end %1 is the file name, %2 the /f mode and %3 the /b parameter, there are two ways to refer to env variables in Win2k+ %var% and !var!, you need the second (immediate) form inside a for loop if the variable is getting updated in the loop
:Sub
Left pad the /b parameter with 5 zeroes
Set Ext=00000%3
and just take the right hand five chars, so 0 becomes 00000, 16 becomes 00016
Set Ext=!Ext:~-5!
%~d1 is just the drive letter of %1, p is the path and n the name, %~dpn1 is the full name without extension, %~f1 is the filename with full path spec.
So if %1 is C:\Temp\Fred.Png /b is 16 and /f is 5, "%~dpn1.%2.!Ext!.Png" is C:\Temp\Fred.5.00016.Png
The 'If' checks if the file already exists, "Start /low /b /wait" executes Pngout at idle priority
runs Pngout on the input "%~f1" produces the output with appropriate parameters using %PngoutMx% if it has been set
If Not Exist "%~dpn1.%2.!Ext!.Png" Start /low /b /wait Pngout "%~f1" "%~dpn1.%2.!Ext!.Png" /b%3 /f%2 /k1 /force %PngOutMx%
Set the variable Sz to the size of the output file %~z1 is the size of file %1, we need this as we can stop processing, I think, it the size at large b has returned to the same size as /b=0
For %%d in ("%~dpn1.%2.!Ext!.Png") Do Set Sz=%%~zd
If %3 Equ 0 Set Sz0=!Sz!&Set Sz=0
There, that's not that bad really ? :shock: Is it.
The rest of the batch file is left as an exercise for the reader, (I used to hate it when my teachers/lecturers used to say that) :D
BarrenSoul at
oooo it looks pretty :) so many squigly symboles :D
counting_pine at
If you want to learn about how to do advanced stuff in batch files, one of the best places to start is to use the HELP command in a cmd window. "HELP FOR" and "HELP SET" are particularly useful. (Note: FOR/? and SET/? are equivalent)
I think the advanced stuff is in all NT-based Windows versions, e.g. NT4, 2000, XP. It's probably one of the main reasons I don't use Windows 98 much anymore.
This link might also be useful.
There's a lot of info to take in. I only use the stuff I can remember, which isn't much. Just some of the FOR stuff and some of the %~ extensions.
I don't know how you do it markcramer, you must have a lot of patience.
Cryoburner at
I just wanted to note that I tried Pngout tonight, and have found it quite useful. The default settings seemed to work well enough when just accessing the executable via a file's right-click menu. An Irfanview plugin sounds like a good idea, since it's already the program that I use for much of my general image viewing.
Great work on this nice and compact utility. It puts all that processing power of my new computer to good use. : )
markcramer at
Hi counting_pine,
It's more a case of years of experience and the fact that I'm rubbish programmer. :-)
There was a big enough improvement in the batch language in Win2K that a lot of my old Turbobasic programs could be reproduced in bat files and made LFN compliant, so it was worth playing around to learn more. I've been telling my self since the 80's that I have to learn to code in C. Guess it's probably a bit late now. ;-)
Cryoburner, playing with the /b parameter can gain you quite a few percent in file size, I've been playing with it and it's a bit time intensive to find the optimum /b value. I think there is a relation between image size and optimum /b value,. I have some images where the optimum /b's are 0, 383, 384 and 6144.
I'll post my results when I've had more time to check.
Mark
counting_pine at
I've done some more tests on half_res_rgb.gif (sorry if I'm turning it into the definitive test image!), concentrating my attention on the number of passes that the pngout algorithm takes.
I haven't quoted my results since they take up 96 lines. I can post/e-mail them if anyone wants them.
Generally, it seems larger values (or 0) for b give fewer passes. In half_res_rgb.gif, b=65536 gives 6 passes, while b=32 results in 17 passes (and produces one of the biggest files).
I was hoping that the best value for b would produce the smallest size on the first pass, but sadly it didn't. The smallest file size was with b=384 (takes 12 passes) but b=384 doesn't give the smallest file size until pass 8.
The best results for pass 1 are at high values of b (or b=0), but converge downward quickly. At pass 2, b=528 gets the smallest file size, and b=400 gets the smallest size for passes 3-7.
BarrenSoul at
using the http://www.libpng.org/pub/png/img_png/16million-pschmidt.png is also good comparison since it has 16millionish colours :) but it seems to compress realy well so it would be nice to have a randomly mixed up one just for testing purposes :)
counting_pine at
I'm not sure I even want to know how long PNGOut would take on that image
[EDIT: About 10 minutes on a P4 - more bearable than I thought:)]
It does have a lot of different colours, but it's made up of a lot of even, gradual colour changes.
It compresses really well because it takes advantage of the alternative row filter types available in PNG. In fact, it's probably about the best example there is of how different filter methods can affect the file size. To illustrate, here's some results from PNGOut. As you can see, the difference is pretty dramatic:
f = 0 Size = 38941130
f = 1 Size = 137056
f = 2 Size = 83617
f = 3 Size = 131744
f = 4 Size = 57608
f = 5 Size = 57608
Filter methods:
0 - none
1 - sub (dx)
2 - up (dy)
3 - average (dx&y)
4 - paeth
5 - mixed
Advpng uses a different "extreme" deflate implementation, based upon the "7-zip" compression engine. Just for kicks I downloaded the latest version (1.13) from sourceforge.net and ran it against a directory of already "pngout" ed files. The command line was:
advpng -z -4 *.png
Suprisingly about 10% of the files were shrunk by a few bytes. Advpng will not expand any files, and the utility was very fast, so there was not much to lose.
At this point, I will run advpng against everything in the future....but what I really want is the horribly slow but excellently compressing pngout...any word yet?
counting_pine at
PNGout: is a quick mode a possibility?
Try AdvDef as well. Sometimes it can beat AdvPNG.
As I understand it, AdvDef simply recompresses, but AdvPNG refilters and compresses. Sometimes refiltering the image makes it compress better, but sometimes it makes it compress worse.
(Note that if you run AdvDef after AdvPNG it won't have any effect, because AdvPNG has already compressed it with the 7zip strategy, so try running it after PNGOut, but before AdvPNG)
Dioxaz at
I also tried AdvPNG and AdvDef too... and I was quite surprised by their performances.
On my K6-2, what makes 5 ou more hours to re-compress with PNGOut, it only takes a few minutes with AdvDef, and achieving almost the same result (PNGOut only beating AdvComp by a few bytes). And, sometimes, AdvDef can beat PNGOut (I've got proofs).
Some of my wallpapers were re-compressed with AdvDef, if you want to take a look and make some tests.
By what I've tried so far, it seems that AdvDef is more appropriate with complex images (all types of PNGs) and AdvPNG for simple Truecolor, Truecolor+alpha and paletted images (correspond to a "filter 0" re-compression).
For me, it seems that 7-Z makes all the difference...
counting_pine at
On my K6-2, what makes 5 ou more hours to re-compress with PNGOut, it only takes a few minutes with AdvDef
That sounds a bit long. I don't know if PNGOut works at different speeds on different processor types, but I have a Pentium-II 400MHz computer that managed wallpaper_sonic_and_knuckles_1024.png in about 25 minutes.
Dioxaz at
Errm sorry :P. I wasn't speaking of my wallpapers when I said "5 hours", I was speaking of CG artworks like this one (which is an official Sonic artwork from Sega, that I converted to PNG). My apologizes.
The PNGout recompress lasted half an a day (I still remember: I let the PC switched on during an entire morning) and gave a file size of 405 KB (the version I linked to).
While AdvDef managed to achieve a size of 409 KB only in one minute. This is frightening. :roll:
I also tried AdvPNG for the same image but it's completely off course: this proves that I was right about AdvPNG being better for simple color images (line art, CG art with solid colors and some linear gradients, etc.) and AdvDef better for all types of more complex PNG images (more photo-like, with many effects).
BTW, the K6-2 processor is different from the Pentium II. It can be a lot less powerful for certain tasks and lacks some instructions from the original Pentium II, too. (it also runs on Socket 7 mainboards, which makes all the difference)
For the wallpaper you mentionned, I don't remember exactly how it (the PNGOut recompress) lasted, it was about an hour I think.
(Ken, if you read this, look into your PMs pleaaaase ^^; )
edit: I Infact, I should have noticed this earlier:
Awsoken said
PNGOUT now replaces PNG files - and only if the result is smaller. My code no longer generates weird filenames with underscores (i.e. mypic_.PNG) or .PN2 extension.
I'd like to congratulate Ken for that feature I had waited for so long, I no longer have to play with incomplete long filename support in Windows98. lol, I always ended up with things like 'WALLP~78.PN2' that I had to rename manually (for each file... hah... hah... ^^; ) To my mind, only this, is a great advencement for PNGOut ;).
counting_pine at
"Dioxaz" said
The PNGout recompress lasted half an a day (I still remember: I let the PC switched on during an entire morning) and gave a file size of 405 KB (the version I linked to). While AdvDef managed to achieve a size of 409 KB only in one minute. This is frightening. :roll:
Well, the truth is, you often have to expend a lot more computing time for just a small file-size decrease. Hence PNGOut, being a lot more thorough, usually beats AdvDef, but not by much. I think the ZLib method used in OptiPNG is even quicker, although maybe not as effective, but OptiPNG's power comes from running the compressor many times, each time with slightly different settings. "Dioxaz" said
and AdvDef better for all types of more complex PNG images (more photo-like, with many effects).
AdvDef's performance is very much dependent on the program that made the original PNG File. For example, if you run it straight after AdvPNG, it won't be able to compress it any further, while running it after PNGOut often produces better results.
AdvPNG uses filter type 0, which happens to (usually) be the best for images with few colours and solid areas. On images with a lot of colours and smooth colour changes, like photographs, a mixed filter type is best. In those cases, you might gain from trying OptiPNG and/or PNGOut before AdvDef. As seen earlier in this topic, PNGOut has a quick mode which could be used to "pre-process" the file for AdvDef. Here's a couple of examples:
pngout /b-256 /f5 file.png advdef -z -4 file.png
optipng -f0-5 file.png advdef -z -4 file.png
Edited by JonoF at
Dioxaz at
lol, I also forgot one thing that have been evocated earlier into that thread in addition. AdvPNG really recompress using a filter 0 while AdvDef just "re-organizes" the data to be more compressible.
For example, an image previously saved with filter 0 will recompress the same way with AdvDef and AdvPNG.
counting_pine said
AdvDef's performance is very much dependent on the program that made the original PNG File. For example, if you run it straight after AdvPNG, it won't be able to compress it any further, while running it after PNGOut often produces better results.
You're right. Running AdvDef after an AdvPNG recompress is a bad idea anyway ('cause you'll get the same result :P).
But I managed to make AdvDef and PNGOut compete by giving them the same image to recompress (coming straight from OptiPNG, and compressed with filter 0).
Result was: AdvDef and PNGOut compressed nearly the same way (144 KB for both, 1 KB bigger for AdvDef)
Then, I gave the PNGOut output to AdvDef, just to see and...
I got exactly the same file size as when I made it perform on the original OptiPNG recompress! (errm... hope you caught what I'm trying to say ^^; -- I'm French, you know)
So, the most important is not really the program used to generate the PNG, but the methods used to compress it ;).
I may make more tries on one of my wallpaper for example. (especially that Shadow one which is an interesting specimen)
counting_pine at
Dioxaz said
I got exactly the same file size as when I made it perform on the original OptiPNG recompress! (errm... hope you caught what I'm trying to say ^^; -- I'm French, you know)
Your English is very good. I'd assumed it was your first language.
Sounds like OptiPNG and PNGOut both gave you the IDAT filtered with type 0.
AdvDef doesn't refilter or rearrange the IDAT in any way, it just recompresses it. This is the reason for its variable results. It depends on how the IDAT was filtered and whether or not it was interlaced (non-interlaced usually compressed better).
PNGOut and OptiPNG use a different algorithm for choosing row types in filter method 5 (Mixed). So if you want to compress a file using PNGOut's mixed filter and AdvDef's speed, I suggest using PNGOut's quick mode.
By the way, I managed to beat the compression on tails-sonicadvance2.png I converted to a palette image and experimented a bit with the block split threshold
pngout /b128 /c3 tails-sonicadvance2.png should do it.
PS. Small warning: If you use AdvPNG on a type 2 (RGB) image with a transparent colour (like tails-sonicadvance2.png), it seems to ignore the transparency chunk and make it opaque.
This might give you the illusion of compressing better than it does, because the transparency chunk in that image will have taken up something like 18 bytes.
none at
New PNGOUT dated Feb 11, 2005
What does this new release contain?
Super slow, but super optimized output
by any chance?
Awesoken at
PNGout: is a quick mode a possibility?
The latest version features a bug fix in handling the tRNS chunk in color modes /c0 and /c2. I haven't made any changes to the compression algorithm in a long time.
Dioxaz at
Oh no, I'm really sorry for my late answer. I was ill and busy. Which explains.
counting_pine said
Your English is very good. I'd assumed it was your first language.
Oh, thank you. I'm feeling delighted now ^^.
counting_pine said
AdvDef doesn't refilter or rearrange the IDAT in any way, it just recompresses it.
Absolutely :P. That's what I completely forgot to mention or wasn't able to explain.
counting_pine said
By the way, I managed to beat the compression on tails-sonicadvance2.png I converted to a palette image and experimented a bit with the block split threshold
pngout /b128 /c3 tails-sonicadvance2.png should do it.
I intentionally let the image as Truecolor. I know that it couls be converted to Palette (thanks for the tip anyway).
counting_pine said
PS. Small warning: If you use AdvPNG on a type 2 (RGB) image with a transparent colour (like tails-sonicadvance2.png)
I noticed that too (yes and on the same image we're talking about ^^. That's why I no longer use AdvPNG on transparent (via tRNS chunk) images.
Now coming to what I planned to do with this image (a nice Shadow the hedgehog wallpaper I made with help of an official artwork).
It comes straight from AdvPNG (or AdvDef, don't remember) and is 433 KB huge.
Now, let's assume that we use any conventional software to save it.
Let's re-compress it with AdvPNG... and guess what we get? The same size as the one mentonned above. (431 KB if you use the "-4" commutator instead of "-3")
And the result is the same if we re-compress it with OptiPNG using filter 0 and use AdvDef after. This confirms that filters play a big role in PNG compression (and bigger than the software itself used to save the PNG ^^).
Some people will notice that this partcular image compressed better with filter 0 than filter 5 (surprising, isn't it?).
So I'm sure that PNGOut will compress it even better with filter 0 (and not 5 like I tried first). I didn't try it yet.
By: Nobody/Anonymous - nobody
RE: Possible ADVPNG bug :type 2 RGB image w/trans
2005-02-20 06:27
Please try the BETA version 1.14. It should fix the problem.
Available at:
http://advancemame.sourceforge.net/beta
counting_pine at
PNGout: is a quick mode a possibility?
Thanks, None! I'll give the new version a try.
The (uncompressed) IDAT of a PNG image is affected by four things: The colour type; the bit depth; whether or not it's interlaced; the filter types on each row.
Probably most image editors will save it with roughly the same IDAT, but PNG optimisers are capable of a few extra tricks. If you've just saved an image from IrfanView or something, the chances are AdvDef won't work as well as it would if you've run it through a PNG optimiser first, because they might try different things on the uncompressed IDAT.
The most important thing isn't what made the PNG but how it made it. For example, a common example seems to be an image compressed with all row filters on type 0. Different applications may compress the IDAT differently, and get different file sizes, but AdvDef will always recompress them the same way.
Re tails-sonicadvance2.png:
I managed to get the image down to 110783 bytes. I posted the PNGOUT options I used, because I figured your computer was too slow to bother with trial and error.
Re wallpaper_shadow_v04_1024.png:
I am surprised that the image compresses better with filter type 0. I tried various programs at filter 5 to check, but I couldn't beat it. I guess it compresses well on type 0 because there are large areas of one colour.
btw, if you compress it with PNGOut, try /b1280 /f0 to get it down to 440375 bytes.
By: Nobody/Anonymous - nobody
RE: Possible ADVPNG bug :type 2 RGB image w/trans
2005-02-20 06:27
Please try the BETA version 1.14. It should fix the problem.
Available at:
http://advancemame.sourceforge.net/beta
Argh, my (our?) version(s) was slightly outdated. Thank you for poiting that ;).
I've tried AdvPNG 1.14 and yes, it no longer removes tRNS chunks when converting less-than-256-color Truecolor images to palette. Thanks to the AdvanceComp Team for having fixed that.
counting_pine said
The most important thing isn't what made the PNG but how it made it
How good it has been said ^^. This is true, we don't know exactly how ordinary software play with zlib stratgy and zlib memory (Paint Shop Pro seems to apply a zlib memory of 6 on palette images, which explains why the old PNGCrush couldn't compress some of them better).
BTW, do PNGOut and AdvComp take those things in paramater? Or are those two factors exclusive to libpng-dependant software?
Now, about tails-sonicadvance2.png:
- Keeping the c2 (Truecolor) mode, it leads to a size of 145,807 bytes, which is already smaller than the equivalent GIF version anyway, with the PNG being Truecolor! I updated the image with this version (and thanks again to Ken for fixing the tRNS problem).
- Converting it to c3 (Palette) effectively gives a file size of 110,783 bytes using your options (but as I said, I intentionally let it as c2... for Paint Shop Pro users, as I'm doing with all the sprite sheets hosted on my site)
About wallpaper_shadow_v04_1024.png...
Oh... oh... oh... "/b1280" you said... and you're right! I'm updating with that new re-compressed version (lol, for 3 poor little KiloBytes in less ^^; ). Proof that PNG and PNGOut didn't stop amazing us (thinking about the fact that the image was near or beyond 500 KB out of a traditional tool). Thank you for the tip.
I may want to try tral and error more often in the future.
counting_pine said
I guess it compresses well on type 0 because there are large areas of one colour
Possibly. Even if I admit that it's not that obvious, mainly because of the semi-tone effect giving the illusion of a complex image. You know, I even have more cases like this one, with completely linear gradients and that still compress better with no filtering:
http://membres.lycos.fr/pnggallery/artworks/sue386.png (a little draw I made under Paintbrush and the old Paint Shop Pro 3.12)
And, of course, i didn't try trial and error yet on it :P.
(thanks to awesoken for having "fixed" the double-post I made)
NerdIII at
PNGout: is a quick mode a possibility?
You talked about the block-sizes as a constant. What if experimentation showed that a variable block size would achieve about 10% svaings in file size?
Could there be an algorithm that finds out the best block-sizes for a file and does not take ages to complete?
Awesoken at
PNGOUT already supports variable block size. My algorithm detects block boundaries by looking for where the byte distribution changes most rapidly. You can view these block boundaries by running pngout with the /v option.
Yes, it's possible to find good block boundaries without taking ages to complete. The bottleneck in PNGOUT is in my pattern matcher. These 2 algorithms are completely independent of each other. It's certainly possible to plug in 7-Zip's pattern matching algorithm with PNGOUT's block boundary finder. I'm just not interested in taking other people's code - that would be too easy : P I'd rather satisfy my own curiosity by seeing if I can optimize my existing algorithm.
Anonymous at
Is there any chance the 'super-slow' mode has been included in the latest release? You hinted earlier in the thread that you might give some indication to the magic switch.
Cheers.
Awesoken at
No, sorry. The latest release only included a change of exit code: 0 for good, and 1 (instead of -1) for error.
Anonymous at
Thanks Ken (/"Awesoken")
For the quick reply and wonderful software!
NerdIII at
ahh very well. I'd not have copied the code either. Good luck with the algorithm! Your program helps me propagating png files where adobe or corel are no help: "Look it's bigger than a GIF!" "Look it's a crappy exporter, try dropping it onto PNGOUT."
counting_pine at
Do you plan to include the super-slow deflate method in PNGOUT?
mongid at
Re: New PNGOUT dated Feb 11, 2005
none said
What does this new release contain?
Super slow, but super optimized output
by any chance?