Compressed File (*.z8) Specification

Modified on 2011/08/15 01:09 by Administrator — Categorized as: Specs, Tools

Compressed File (*.z8) Specification

The *.z8 files use industry-standard stream compression/decompression library 'ZLIB'. Please refer to the original vendor's web site for specification of the compression/decompression interfaces in greater detail, as well as for latest updates and versions.

The ZLIB interface allows you to quickly compress and decompress streaming data and achieve outstanding compression ratios. All this provides significant benefits when dealing with large amounts of data associated with order flow. On multi-core systems, performance can be almost doubled by unpacking the stream in one thread, and performing actual processing in another thread.

The *.z8 file format is very simple.

Offset Length Type Description
0 8 bytes Signed 64-bit The size of the unpacked stream. It can be useful when reading the decompressed stream via a decompression library that does not provide an End-Of-File flag for input.
8 Till EOF Binary data The compressed stream itself.

You can also follow these algorithms to read the data from *.z8 files:

Algorithm 1:

  1. Open the source *.z8 file
  2. Skip the first 8 bytes
  3. Read a portion of compressed data into the source buffer
  4. Uncompress data from the source buffer into the destination buffer using ZLIB 'inflate' interface
  5. Repeat step 3 until the end of the source file
  6. Flush the destination buffer and close the source file

Algorithm 2:

  1. Open the source *.z8 file as a File stream
  2. Read the destination file size into a 64-bit signed integer N
  3. Create a Decompression stream and assign the File stream as its input
  4. Read N bytes from Decompression stream
  5. Close Decompression and File streams

You can also use the tools for handling the *.z8 files.