Every entry
point.

fnadler32

pub fn adler32(BytesView) -> UInt32

Adler-32 (RFC 1950): two running sums modulo the largest prime below 65536. Cheaper than a CRC and weaker, which is the trade zlib made. The folding is mooncrypt/sum/adler32, reached from here for the same reason crc32 is.

returns UInt32

fncrc32

pub fn crc32(BytesView) -> UInt32

CRC-32 as gzip, ZIP and PNG use it (RFC 1952): the reflected form of the IEEE 802.3 polynomial, starting and ending complemented.

The folding is mooncrypt/sum/crc32; it is reached from here so the formats that specify it — gzip in its trailer, zip in every entry and again in the central directory — find it in one place instead of each importing the other.

returns UInt32

fncrc32c

pub fn crc32c(BytesView) -> UInt32

CRC-32C, the Castagnoli polynomial: the checksum that has hardware support on every x86 since Nehalem and every ARM since v8, which is why Snappy and iSCSI chose it over the older CRC-32 that gzip and ZIP use.

It lives here beside the other two for the same reason they are here: it is a checksum the formats specify, not something a format invents, and a caller comparing two of them should find them in one place.

returns UInt32

fnmask_crc32c

pub fn mask_crc32c(UInt32) -> UInt32

The mask Snappy applies to a CRC-32C before putting it in a frame, so that a checksum never looks like the length in front of it — and so that a reader that echoes the bytes back cannot be fooled into accepting them.

returns UInt32

fndeflate

pub fn deflate(BytesView, level? : Int32, window? : Int32, chain? : Int32, blocks? : Blocks) -> Bytes

Compress into a DEFLATE stream, one block over the whole input.

LZ77 finds back-references with a hash chain over the window, capped so the search stays linear in the input. blocks decides how the result is coded.

level runs 0 to 9 and means what it means in the format's reference: 0 stores the input without compressing it, 9 searches hardest, and 6 is the default every mainstream encoder uses. It sets how far back the match finder looks, with the same numbers the reference tables give it.

window is how far a back-reference may reach — 32 KiB is the most DEFLATE allows and the only value a reader can be assumed to handle. chain sets the search depth directly for a caller who would rather not go through a level.

level=0 writes stored blocks, which is what "no compression" means everywhere; blocks then has nothing to choose between.

returns Bytes

fninflate

pub fn inflate(BytesView, limit? : Int32) -> Bytes raise @moonzip.Refused

Decompress a raw DEFLATE stream.

All three block types: stored, fixed Huffman, and the dynamic Huffman blocks that carry their own code tables — which is what zlib, gzip and every browser actually emit, so a reader without them reads almost nothing.

limit bounds the output. A compressed stream says nothing about how much it expands to, and a small one can expand to more memory than the machine has; a reader handling input from elsewhere should say how much it is prepared to hold. Unbounded by default, which is what every library does.

returns Bytes · raises @moonzip.Refused

fncrc32

pub fn crc32(BytesView) -> UInt32

CRC-32, where the container that specifies it names it.

The implementation is @moonzip.crc32: zip writes the same checksum, and neither container should have to import the other to reach it.

returns UInt32

fngunzip

pub fn gunzip(BytesView, checksum? : Bool, length? : Bool, header_crc? : Bool, limit? : Int32) -> Bytes raise @moonzip.Refused

Read a gzip container and decompress what is in it.

The trailing CRC-32 and length are checked, because a container that carries a checksum and does not look at it is not a container. Each check is a parameter for the cases where a stream is known to be truncated and the part that arrived is still wanted.

limit bounds the output, for a stream that came from elsewhere.

returns Bytes · raises @moonzip.Refused

fngzip

pub fn gzip(BytesView, level? : Int32, window? : Int32, chain? : Int32, mtime? : UInt32, name? : String, comment? : String, os? : Int32) -> Bytes

Wrap DEFLATE output in a gzip container.

The modification time is zero and the name is absent unless asked for, so the same input gives the same bytes. A compressor that stamped the clock would make a build unreproducible and a cache entry never match — which is why Go's gzip.Writer also writes zero unless told otherwise. Set mtime when the output is a file that should carry one.

level, window and chain go straight through to the compressor.

returns Bytes

fnadler32

pub fn adler32(BytesView) -> UInt32

Adler-32, where the container that specifies it names it.

The implementation is @moonzip.adler32; keeping it here as well means a caller who has the container in hand does not have to reach for the root package to checksum what it read.

returns UInt32

fnunzlib

pub fn unzlib(BytesView, checksum? : Bool, limit? : Int32) -> Bytes raise @moonzip.Refused

Read a zlib container and decompress what is in it.

checksum is what makes it a container rather than a wrapper; limit bounds the output, for a stream that came from elsewhere.

returns Bytes · raises @moonzip.Refused

fnzlib

pub fn zlib(BytesView, level? : Int32, window? : Int32, chain? : Int32) -> Bytes

Wrap DEFLATE output in a zlib container.

This, not raw DEFLATE, is what HTTP's Content-Encoding: deflate was specified to mean — a detail that half the servers on the web got wrong, which is why gzip won. level, window and chain go straight through to the compressor; the header says which window was used, so a reader knows how much history it must keep.

returns Bytes

fnaes_decrypt

pub fn aes_decrypt(BytesView, String, Int32) -> Bytes raise @moonzip.Refused

Decrypt what aes_encrypt wrote.

The salt names the key, the verification value rejects a wrong password before any ciphertext is touched, and the authentication tag is checked before the bytes are handed back — a truncated HMAC is a message authentication code, and skipping it would turn a tampered archive into a silent decompression error three layers down.

returns Bytes · raises @moonzip.Refused

fnaes_encrypt

pub fn aes_encrypt(BytesView, String, String, Int64, Int64, strength? : Int32) -> Bytes

Encrypt data under password with WinZip AES.

strength is 128, 192 or 256. Returns the bytes that go where the payload used to: salt, verification value, ciphertext, authentication tag. The AE-1 versus AE-2 choice belongs to the writer, because it is a choice about the CRC in the header rather than about the ciphertext.

returns Bytes

fnentry

pub fn entry(BytesView, String, limit? : Int32, codecs? : ArrayView[&Codec], password? : String) -> Entry? raise @moonzip.Refused

One entry by name, expanding only that one.

None for a name the archive does not hold — asking whether something is in an archive is a question, not a failure. A name that is there but whose content is damaged still raises: that is a broken archive, not an absent member.

returns Entry? · raises @moonzip.Refused

fnjoin

pub fn join(ArrayView[Bytes]) -> Bytes

Put the volumes back together, in the order they were split.

The result is the archive the volumes were cut from, which is what unzip and names expect.

returns Bytes

valuelevel

pub let level : Int32

fnnames

pub fn names(BytesView) -> Array[String] raise @moonzip.Refused

Every entry's name, without decompressing anything.

The central directory is a table at the end of the archive that names every member; reading it costs nothing next to expanding them. This is what a caller listing an archive wants, and what zipfile.namelist() is.

returns Array[String] · raises @moonzip.Refused

fnsplit

pub fn split(BytesView, Int32) -> Array[Bytes]

Cut a finished archive into volumes of at most size bytes, in order.

Every volume but the last is exactly size bytes; the last holds the rest, which may be less. An archive that fits in one volume comes back as a single volume, and an empty one as an empty array. size must be positive: a volume that holds nothing would be a loop that never advances.

returns Array[Bytes]

fnunzip

pub fn unzip(BytesView, limit? : Int32, codecs? : ArrayView[&Codec], password? : String) -> Array[Entry] raise @moonzip.Refused

Every entry, expanded.

limit bounds what one entry may expand to. An archive is worth bounding even more than a single stream is: a few kilobytes of archive can declare a hundred members that each claim to be a gigabyte.

returns Array[Entry] · raises @moonzip.Refused

fnunzip_spanned

pub fn unzip_spanned(ArrayView[Bytes], limit? : Int32, codecs? : ArrayView[&Codec], password? : String) -> Array[Entry] raise @moonzip.Refused

Read an archive that arrived as spanned volumes.

The volumes are joined, each directory record's disk and relative offset are translated back to one coordinate space, and the result is read exactly as unzip reads a single buffer. Volumes must be in disk order.

returns Array[Entry] · raises @moonzip.Refused

fnunzip_split

pub fn unzip_split(ArrayView[Bytes], limit? : Int32, codecs? : ArrayView[&Codec], password? : String) -> Array[Entry] raise @moonzip.Refused

Read an archive that arrived as several volumes.

The volumes are joined first — a split archive is the same bytes cut at boundaries, so the pieces in order are the archive — and then read exactly as unzip reads one buffer.

returns Array[Entry] · raises @moonzip.Refused

fnvolume_names

pub fn volume_names(String, Int32) -> Array[String]

The file names a split archive conventionally uses.

For an archive called base, the first count - 1 volumes are base.z01, base.z02, … and the last is base.zip — the extension the tools look for, which is why the final volume carries the one that says "zip" rather than another number. A single volume is just base.zip.

returns Array[String]

fnzip

pub fn zip(ArrayView[Entry], level? : Int32, codecs? : ArrayView[&Codec], zip64? : Bool, descriptor? : Bool, password? : String, aes? : Int32, ae? : Int32) -> Bytes

Write an archive.

Entries are written in the order they are given, each with a local header, then the central directory, then the end record — the layout APPNOTE.TXT describes and the one every reader expects.

level reaches the compressor for the deflated entries; Stored ones carry their bytes unchanged. An entry that deflates to no less than it started as is stored instead, which is what zip itself does: a member should never grow by being compressed.

zip64, descriptor, password, aes and ae are the Writer's knobs, and they mean the same thing here. aes is 0 for no AES (ZipCrypto if a password is given), or 128, 192 or 256 for WinZip AES at that strength. This function is that writer with every entry in hand at once.

returns Bytes

fnzip_spanned

pub fn zip_spanned(ArrayView[Entry], Int32, level? : Int32, codecs? : ArrayView[&Codec], descriptor? : Bool, password? : String, aes? : Int32, ae? : Int32) -> Array[Bytes]

Write an archive as spanned volumes of at most size bytes each.

Each member's central-directory record is rewritten with the disk it starts on and its offset from that disk's first byte, and the end record is rewritten with the last disk's number and the directory's disk-relative offset. The arguments after size are zip's.

A caller with a file system writes volume i under the name volume_names gives it. The result differs from a split only in those header fields: the bytes of the members are the same, so a mode that has no need of disk numbers should use the simpler one.

returns Array[Bytes]

fnzip_split

pub fn zip_split(ArrayView[Entry], Int32, level? : Int32, codecs? : ArrayView[&Codec], zip64? : Bool, descriptor? : Bool, password? : String, aes? : Int32, ae? : Int32) -> Array[Bytes]

Write an archive and split it in one step.

The arguments are zip's, plus the volume size. This is the call a caller with a large archive and a size limit wants: it never builds the whole stream twice, and the volumes are the ones unzip_split reads back.

returns Array[Bytes]

fnzipcrypt_decrypt

pub fn zipcrypt_decrypt(BytesView, String, Byte) -> Bytes raise @moonzip.Refused

Decrypt what zipcrypt_encrypt wrote.

The check byte is compared before any of the body is handed back, so a wrong password says so instead of turning into a decompressor error three layers further in — which is the difference between "you typed the password wrong" and "this archive is damaged".

returns Bytes · raises @moonzip.Refused

fnzipcrypt_encrypt

pub fn zipcrypt_encrypt(BytesView, String, String, Int64, Byte) -> Bytes

Encrypt data under password.

check is the byte a reader compares after decrypting the header: the high byte of the CRC-32, or — when the entry's sizes are in a data descriptor rather than its local header — the high byte of the MS-DOS time. The caller knows which, because the caller owns the flags.

returns Bytes

fnnames

pub fn names(BytesView) -> Array[String] raise @moonzip.Refused

Every entry's name, which still has to walk the archive: tar has no index.

returns Array[String] · raises @moonzip.Refused

fntar

pub fn tar(ArrayView[Entry]) -> Bytes

Write an archive.

Members go out in the order they are given, each as a header, its bytes padded to a block boundary, then two empty blocks to say the archive is over.

A name longer than a hundred bytes is carried in a pax extended header, which is a member of its own whose content is a list of keyword=value records. That is the POSIX answer, and it is what GNU tar and bsdtar both read; the older GNU L record is read but not written, because one way to say a thing is enough.

The archive is reproducible: the modification time comes from the entry and the owner is nobody, so the same entries always give the same bytes. Set mtime and uid on an entry when it should carry something else.

returns Bytes

fntargz

pub fn targz(ArrayView[Entry], level? : Int32) -> Bytes

A tarball: the archive gzipped, which is what .tar.gz and .tgz are.

The two formats stack cleanly — gzip compresses the bytes, and the bytes are a tar archive — so this is three lines rather than a format of its own. The compression is of the whole archive, which is why a tarball compresses far better than the same files compressed one at a time: gzip sees every member's redundancy, not just each member's own.

returns Bytes

fnuntar

pub fn untar(BytesView, limit? : Int32) -> Array[Entry] raise @moonzip.Refused

Read an archive.

Both ustar and pax come out, and the GNU long-name records that older archives use are understood on the way. limit bounds what one member may expand to, and tar needs it more than most: an archive is a header claiming a length, and a header can claim anything.

returns Array[Entry] · raises @moonzip.Refused

fnuntargz

pub fn untargz(BytesView, limit? : Int32) -> Array[Entry] raise @moonzip.Refused

Read a tarball.

returns Array[Entry] · raises @moonzip.Refused

fnframe

pub fn frame(BytesView, level? : Int32) -> Bytes

Wrap LZ4 blocks in a frame: a header, the blocks, an end mark and a content checksum.

Blocks are independent, so a decoder can start at any of them; the content checksum is over the original bytes rather than the compressed ones, so it catches a decoder that produced the wrong thing rather than only a link that flipped a bit.

A block that does not compress is stored as it is with the high bit of its length set, which is how the format says "this one is verbatim".

returns Bytes

fnlz4

pub fn lz4(BytesView, level? : Int32) -> Bytes

Compress into the LZ4 block format.

A block is a run of sequences. Each is a token whose high nibble is how many literals follow and whose low nibble is the match length minus four, then the literals, then a two-byte back-reference, then the match length's overflow bytes. The last sequence carries literals only, and the format reserves the last five bytes of the input for them — a decoder relies on both, which is why match_start stops twelve bytes short of the end.

level picks the search depth: 0 uses one candidate per position, which is what LZ4's own "fast" mode does; higher values walk a chain. The ratio gain above the default is small and the cost is real — this is a format chosen for speed.

returns Bytes

fnunframe

pub fn unframe(BytesView, checksum? : Bool, limit? : Int32) -> Bytes raise @moonzip.Refused

Read a frame back.

checksum is what turns "some bytes came out" into "the right bytes came out"; a caller recovering what it can from a damaged transfer turns it off. A skippable frame — the format's way of saying "not mine, step over it" — is stepped over rather than refused.

returns Bytes · raises @moonzip.Refused

fnunlz4

pub fn unlz4(BytesView, limit? : Int32) -> Bytes raise @moonzip.Refused

Decompress an LZ4 block.

There is no length in the format, so limit is the only thing standing between a hostile block and the heap.

returns Bytes · raises @moonzip.Refused

fnxxh32

pub fn xxh32(BytesView, seed? : UInt32) -> UInt32

The hash itself.

returns UInt32

fnframe

pub fn frame(BytesView) -> Bytes

Wrap Snappy blocks in the framing format.

The stream identifier comes first, then a chunk per sixty-four kilobytes — compressed with a masked CRC-32C when compressing helped, verbatim with one when it did not. A chunk that grew would be a chunk that cost time to decode and bytes to store, so the format has a way to say "verbatim" and this uses it.

returns Bytes

fnsnappy

pub fn snappy(BytesView) -> Bytes

Compress into the Snappy block format.

A block starts with the uncompressed length as a varint — the one thing a decoder is told up front, and the reason Snappy needs no bound of its own to be safe — and then a run of tags. A tag is a literal or a copy, and its two low bits say which: 00 a literal, 01 a copy with an eleven-bit offset, 10 one with a sixteen-bit offset, 11 one with a thirty-two-bit offset. The three copy forms exist so that the common cases cost two bytes.

The compressor is greedy: one candidate per position from a hash of four bytes, extended as far as it runs. Snappy's own encoder is not much cleverer, because the format's whole point is that decompression be quick.

returns Bytes

fnunframe

pub fn unframe(BytesView, checksum? : Bool, limit? : Int32) -> Bytes raise @moonzip.Refused

Read a framed stream back.

checksum is what makes the CRC-32C in each chunk mean something; a caller recovering what arrived from a broken transfer turns it off. A skippable chunk — the format's way of letting a caller put its own data in the middle — is stepped over, and an unskippable one is refused, because that is the difference the two ranges of type numbers exist to make.

returns Bytes · raises @moonzip.Refused

fnunsnappy

pub fn unsnappy(BytesView, limit? : Int32) -> Bytes raise @moonzip.Refused

Decompress a Snappy block.

The length in front is a claim, not a fact, so limit still applies: a sixteen-byte block can claim four gigabytes. The claim is also what lets this reader refuse a stream that ends early rather than return what it got.

returns Bytes · raises @moonzip.Refused

valuechunk_size

pub let chunk_size : Int32

fndecoded_dict

pub fn decoded_dict(Int32) -> Int32

returns Int32

fndict_code

pub fn dict_code(Int32) -> Int32

The LZMA2 dictionary-size code: the smallest property whose decoded size is at least what was asked for, because a decoder has to know how much memory to reserve before it starts.

returns Int32

fndictionary_for

pub fn dictionary_for(Int32) -> Int32

The dictionary size a level asks for. It bounds how far back a match may reach, and therefore how much a decoder must hold; 8 MiB at the top is what xz -9 uses.

returns Int32

fnencode

pub fn encode(BytesView, Int32, Int32, level? : Int32) -> Bytes

Compress with LZMA, using the given properties and dictionary size.

The match finder is a hash chain over three bytes, walked greedily: the first match of three or more bytes wins, and the repeat distance is tried before the chain because it costs a handful of bits instead of a distance. That is a long way from an optimal parse, and it is the same order of effort a mainstream level-six encoder makes — the point here is a stream that real decoders read, not the last three percent.

returns Bytes

fnlzma

pub fn lzma(BytesView, level? : Int32) -> Bytes

Compress into the .lzma "alone" container: thirteen bytes of header, then the range-coded stream.

The header says the properties (how many context bits the literals use), the dictionary size, and the uncompressed size — which is written, so the stream does not need an end marker. level maps to the same presets xz uses, and the dictionary is capped at what a decoder can be asked to hold.

returns Bytes

fnlzma2

pub fn lzma2(BytesView, Int32, Int32) -> Bytes

LZMA2: LZMA in chunks, each with its own range coder, and a control byte that says how much state to throw away first.

Every chunk resets the state and the properties, which costs a few hundred forgettable decisions per megabyte and makes each chunk independent; the dictionary is the only thing carried across. The first chunk must reset the dictionary too — a decoder has none to start from.

returns Bytes

fnproperties_of

pub fn properties_of(Int32, Int32, Int32) -> Int32

The properties byte for a set of parameters.

returns Int32

fnunlzma

pub fn unlzma(BytesView, limit? : Int32) -> Bytes raise @moonzip.Refused

Read a .lzma "alone" container.

A stream may leave its length unknown and end with a marker instead, which is what every encoder did before sizes could be trusted; both are read here.

returns Bytes · raises @moonzip.Refused

fnunlzma2

pub fn unlzma2(BytesView, Int32, Decoder) -> (Bytes, Int32) raise @moonzip.Refused

Decode the LZMA2 chunks starting at at, and say where they ended.

returns (Bytes, Int32) · raises @moonzip.Refused

fnunxz

pub fn unxz(BytesView, limit? : Int32) -> Bytes raise @moonzip.Refused

Read an xz stream.

The header, the block header, the index and the footer are each checked against their own CRC, and the block against the check the header named. A stream that disagrees with itself is refused rather than decoded into something plausible.

returns Bytes · raises @moonzip.Refused

fnxz

pub fn xz(BytesView, level? : Int32, check? : Check) -> Bytes

Compress into an xz stream.

One block holds everything; a stream of several blocks is what a parallel compressor produces and a reader does not care either way. The block names LZMA2 with a stated dictionary, the index follows it, and the footer closes it — all three of which a reader checks before trusting a byte.

The block's data is padded to a four-byte boundary *before* its check, not after: that is where the specification puts it, and a stream that pads afterwards reads its own check three bytes early and is rejected by every other implementation.

returns Bytes

fnbunzip2

pub fn bunzip2(BytesView, limit? : Int32) -> Bytes raise @moonzip.Refused

Decompress a bzip2 stream.

limit is how much output the caller is prepared to hold; a compressed stream says nothing about how far it expands, so a reader of untrusted input should say. Past it, TooBig rather than an allocation.

 moonbit nochecklet plain = @bzip2.bunzip2(packed[:])

returns Bytes · raises @moonzip.Refused

fnbzip2

pub fn bzip2(BytesView, level? : Int32) -> Bytes

Compress into bzip2.

level is one to nine and means what it means everywhere else: how much memory the compressor may use on a block, and so how far back a match can reach. Nine is the default because the format's own default is nine, and a caller who cares has an opinion.

 moonbit nochecklet packed = @bzip2.bzip2(body[:], level=9)

returns Bytes

fnnames

pub fn names(BytesView) -> Array[String] raise @moonzip.Refused

The names in an archive, without decompressing anything.

Reading a directory should not cost the directory's contents, which for a solid archive would be the whole thing.

returns Array[String] · raises @moonzip.Refused

fnsevenz

pub fn sevenz(ArrayView[Entry], level? : Int32, how? : Method, solid? : Bool) -> Bytes

Build a 7z archive.

 moonbit nochecklet archive = @sevenz.sevenz(members[:], level=6)

solid is 7-Zip's own default and the format's reason for existing: one folder holds every file, so the compressor sees them all at once and a repeated phrase across two files is paid for once. The cost is that reading the last file means decompressing the first ones, which is why it is an argument.

returns Bytes

fnun7z

pub fn un7z(BytesView, limit? : Int32) -> Array[Entry] raise @moonzip.Refused

Read a 7z archive.

 moonbit nochecklet members = @sevenz.un7z(archive[:])

limit is how much output the caller is prepared to hold. An archive declares its unpacked sizes, so this is checked before the allocation rather than after.

returns Array[Entry] · raises @moonzip.Refused

fnunzstd

pub fn unzstd(BytesView, limit? : Int32) -> Bytes raise @moonzip.Refused

Decompress a zstd frame.

 moonbit nochecklet plain = @zstd.unzstd(packed[:])

returns Bytes · raises @moonzip.Refused

fnzstd

pub fn zstd(BytesView, level? : Int32) -> Bytes

Compress into a zstd frame.

 moonbit nochecklet packed = @zstd.zstd(body[:])

The block is an LZ77 parse coded with the *predefined* FSE tables, and the literals are sent raw. That is a legal frame that any decoder reads, and it compresses: what it gives up against zstd -19 is the per-block table selection and the Huffman coding of literals, which are the two parts of the reference encoder that are about ratio rather than about the format.

returns Bytes

fnbrotli

pub fn brotli(BytesView, window? : Int32, quality? : Int32) -> Bytes

Brotli, RFC 7932.

The writer does all of what earns the format its ratio: literal context modelling, block types for the literal and command streams, the static dictionary, and short distance codes. At quality 9 — the default — it is level with the reference encoder on this repository's own documentation, and somewhat ahead of it below that.

window is the log of the window size, 10 to 24: how far back a copy may reach. Larger windows find more matches and cost nothing but memory while writing, since the reader keeps whatever it needs.

quality is 0 to 11 and is how hard the writer looks, not which features it uses. Up to 8 it sets the number of hash-chain candidates walked and whether a lazy step is taken, which is the whole of a greedy parse. From 10 the parse is chosen rather than taken: an optimal walk over the block keeps the cheapest price of reaching every position, prices come from the codes the previous round produced, and each round's result is encoded to see which is genuinely smaller. That is seven times the work of quality 9 on a 35 KB body — 193 ms against 28 — and four and a half per cent off the output, because the parse is where the last of the ratio lives. Ten and eleven differ only in how many candidates the search walks, which on text changes almost nothing once the parse is chosen.

A higher quality never produces more bytes and always takes longer.

returns Bytes

fnunbrotli

pub fn unbrotli(BytesView, limit? : Int32) -> Bytes raise @moonzip.Refused

Un-brotli a complete stream.

returns Bytes · raises @moonzip.Refused