![]() ![]() db_dump.sqlĪs a bottomline I would not recommend the zopfli algorithm since the compression took tremendous amount of time for a not-that-significant amount of disk space spared. Pigz + zopfli algorithm time pigz -11kMN. The file being compressed is 9.25 GiB SQL dump.GNU/Linux distribution: Xubuntu 17.10 (artful).Hardware used: Intel(R) Core(TM) i7-7700HQ CPU 2.80GHz (4c/8t) + Nvme SSD.As pigz is the multithreaded evolution I personally have chosen to use from now on. ![]() It is not really an answer, but I think it is relevant enough to share my benchmarks comparing speed of gzip and pigz on a real HW in a real life scenario. A choice of either size or speed optimizationsĪllows for either better compression than even lzma can provide, orīetter speed than gzip, but with bzip2 sized compression levels.Ī small Compression Benchmark (Using the test Oli created):Ī small Compression Benchmark (Using a Text file): It is designed to scale with increases with RAM size, improvingĬompression further. It uses the combinedĬompression algorithms of zpaq and lzma for maximum compression, lzoįor maximum speed, and the long range redundancy reduction of rzip. This work explores the possibility of using dedicated hardware to accelerate the same general-purpose compression algorithm normally run at the warehouse-scale computer level. Ratios and speed when used with large files. In the exa-scale age of big data, le size reduction via compression is ever more impor-tant. LRZIP - A multithreaded compression program that can achieve very high compression ![]() V1.0.2 (ie: anything compressed with pbzip2 can be decompressed with The output of this version is fully compatible with bzip2 PBZIP2 - pbzip2 is a parallel implementation of the bzip2 block-sorting fileĬompressor that uses pthreads and achieves near-linear speedup on SMP Replacement for gzip that takes advantage of multiple processors and multiple PIGZ - pigz, which stands for Parallel Implementation of GZip, is a fully functional Of big software files and large scale data archiving. Multiprocessor machines, which makes it specially well suited for distribution Plzip is intended for faster compression/decompression of big files on Plzip is a massively parallel (multi-threaded) version of lzip using the lzipįile format the files produced by plzip are fully compatible with lzip. Which makes it well suited for software distribution and data archiving. Lzip decompresses almost as fast as gzip and compresses better than bzip2, Integrity checking and a user interface similar to the one of gzip or bzip2. PLZIP - Lzip is a lossless data compressor based on the LZMA algorithm, with very safe Speed up compression time with minimal possible influence on compression Its primary goal is to utilize all resources to PXZ - Parallel XZ is a compression utility that takes advantage of running LZMAĬompression of different parts of an input file on multiple cores and After looking for all compression tools that were also parallel I found the following: ![]()
0 Comments
Leave a Reply. |