Zlibdev

Proposal

A vfile device driver, zlibdev, will provide device nodes for inflation (uncompress) and deflation (compress) of files as a streaming service.

This will be useful, for example, in file transfers. In these example, only the compressed file is transferred over the network.

• A large file, compressed on the host, put on the target using the inflation streamer.
• A large file on the target, being retrieved by the host using the deflation streamer.

Potential first usage:

DVMS RAM MAL is a (for now) 32MB platform resource-backed "RAM disk". For DVMS debugging it is often necessary or useful to retrieve the 32MB "file" using /proc/res for host-offline analysis.

Quite a bit of the 32MB space is "unused" and likely would deflate dramatically. A benchmark showed 98% file compression results when retrieving a 32MB RAM disk image that had been used to run the dvmsbasher-with-exfat example. The time to FTP 32MB on the target was 3:52. The time to FTP the equivalent compressed file size (~580K) was 4 seconds!

Productivity win.

Usage

To use these devices, their unparsed remainder will be the "target-based source or destination file". For example:

• Use FTP to transfer and uncompress partition.gz to /dev/sda1:

ftp> put partition.gz /dev/inflate/dev/sda1

• Use FTP to compress and retrieve /dev/sda1 to host partition.gz:

ftp> get /dev/deflate/dev/sda1 partition.gz

Details

zlibdev vfile device driver provides:

/dev/deflate
/dev/inflate

These each invoke, hopefully obviously, the stream-based zlib deflate or inflate functionality. libdeos-zlib.so will be updated to expose this stream-based interface as API:

inflateInit
inflate
inflateEnd
deflateInit
deflate
deflateEnd

Device operations needed:

1. open
   • Prepend / on the unparsed remainder and attempt to open the resultant absolute path.
   • Call inflateInit or deflateInit depending on which device node was accessed.

1. close
   • Call inflateEnd or deflateEnd depending on which device node was accessed.
   • Flush any remaining inflate/deflate buffer to the file.
   • Close the opened file.

1. read
   • For deflate: Read uncompressed data from the source file into the input side of the stream as needed. Call deflate to push the data through the stream. If there is compressed file data in the output side of the stream, provide it to the caller.
   • For inflate: Read compressed data from the source file into the input side of the stream as needed. Call inflate to push the data through the stream. If there is uncompressed file data in the output side of the stream, provide it to the caller.

1. write
   • For deflate: Read uncompressed data from the caller's buffer into the input side of the stream as needed. Call deflate to push the data through the stream. If there is compressed file data in the output side of the stream, write it to the destination file.
   • For inflate: Read compressed data from the caller's buffer file into the input side of the stream as needed. Call inflate to push the data through the stream. If there is uncompressed file data in the output side of the stream, write it to the destination file.

1. lseek
   • Not absolutely necessary, but seeking in a stream is possible.