lrzip-next-0.14x file format
Peter Hyman
February 2025
Update encryption salt bytes
Magic Header length and contents unchanged from v0.13x.

Magic Data
----------
6-13	Source File Decompressed Size
or	If encrypted, Encryption Cost Factor 2s exponent + Salt (1 byte + 7 bytes)

Cost factor byte value must be between 10 and 40 or 2^10 (1KB) to 2^40 (1TB).

Encrypted salt (bytes 6->13 in magic if encrypted):
0	2s exponent of Cost factor. 2^N = costfactor
1->7	Random data
(RCD0 is set to 8 bytes always on encrypted files)

lrzip-next-0.13x file format
Peter Hyman
June 2024
Adds RISC-V BCJ filter. Change to filter coding method at Byte 0x16
The rest of magic header remains the same as 0.12x see below.

Magic Data
----------
16	Filtering. 0=none, x86, ARM, ARMT, ARM64, PPC, SPARC, IA64, RISC-V (1..8)
	-- OR --
	Bit 7 will be set for Delta
	Delta Offset, 1..32, stored as 0..31
	stored as normal without byte shift used earlier.
	Filtering is for all compression methods.

lrzip-next-0.12x file format
Peter Hyman
July 2023
Adds ARM64 BCJ filter. Changes filter coding values

Byte	Content
0-20	Magic
21-84	Comment (if byte 20 > 0)
----
21+	RZip Chunk Data (RCD)
RCD+	Compressed Data Blocks
end-##	Selected hash (see CURRENT_HASHES)

Magic Data
----------
0-3	LRZI
4	LRZIP-NEXT Major Version
5	LRZIP-NEXT Minor Version
6-13	Source File Decompressed Size
or	If encrypted, Encryption Hash Loops + Salt (2 bytes + 6 bytes)
14	>=1 = md5sum hash is stored at end of the archive, otherwise CRC
	(Hashes defined in CURRENT_HASHES file)
15	>=1 = archive is encrypted. 1=AES128, 2=AES256
16	Filtering. 0=none, x86, ARM, ARMT, ARM64, PPC, SPARC, IA64 (1..7) Delta
	-- OR --
	high order 5 bits will contain delta offset, 0..31 if any
	Delta filter bit no longer stored
	Filtering is for all compression methods.
17	CTYPE: 0=NONE/OTHER, 1:LZMA, 2:ZPAQ, 3:BZIP3, 4:ZSTD.
	If ZSTD, high 4 bits will contain ZSTD Strategy (1-9).
18	LZMA Properties using LZMA2 encoding for dictionary size only
or
18	ZPAQ Properties. 0b0CCCBBBB where CCC is compression level, BBBB is
	block size. (high bit set is no longer needed)
or
18	BZIP3 Properties. 0b0000BBBB where BBBB is 0-8. Allowable BZIP3 block
	sizes.
or
18	ZSTD Properties. 0b000CCCCC where CCCCC is 1-22. ZSTD Strategy will
	be stored in high order bits of CTYPE (1-9).
19	lrzip-next compression levels, 0bRRRRLLLL
	high order bits Rzip compression level (magic >> 4)
	low order bits Lrzip compression level (magic & 15)
20	Comment length. 0 means no comment.
21-84	Comment 64 chars max

Encrypted salt (bytes 6->13 in magic if encrypted):
0->1	Encoded number of loops to hash password
2->7	Random data
(RCD0 is set to 8 bytes always on encrypted files)

Rzip Chunk Data:
0	Data offsets byte width (meaning length is < (2 * 8)^RCD0)
1	Flag that there is no chunk beyond this
(RCD0 bytes)	Chunk decompressed size (not stored in encrypted file)
(Header data is also encrypted in encrypted files)
XX	Stream 0 header data
XX	Stream 1 header data

Stream Header Data:
Byte:
0	Compressed data type
(RCD0 bytes)	Compressed data length
(RCD0 bytes)	Uncompressed data length
(RCD0 bytes)	Next block head

Data blocks:
0->(end-2) data
(end-1)->end crc data
(end - hash size) for md5

lrzip-next-0.11x file format
Peter Hyman
April 2023
Adds CTYPE byte for methods that have stored values.
Will store compression info for BZIP3, LZMA, ZPAQ, ZSTD.

Byte	Content
0-20	Magic
21-84	Comment (if byte 20 > 0)
----
21+	RZip Chunk Data (RCD)
RCD+	Compressed Data Blocks
end-##	Selected hash (see CURRENT_HASHES)

Magic Data
----------
0-3	LRZI
4	LRZIP-NEXT Major Version
5	LRZIP-NEXT Minor Version
6-13	Source File Decompressed Size
or	If encrypted, Encryption Hash Loops + Salt (2 bytes + 6 bytes)
14	>=1 = md5sum hash is stored at end of the archive, otherwise CRC
	(Hashes defined in CURRENT_HASHES file)
15	>=1 = archive is encrypted. 1=AES128, 2=AES256
16	Filtering. 0=none, x86, ARM, ARMT, PPC, SPARC, IA64, DELTA (1..7)
	high order 5 bits will contain delta offset, 0..31
	Filtering is for all compression methods.
17	CTYPE: 0=NONE/OTHER, 1:LZMA, 2:ZPAQ, 3:BZIP3, 4:ZSTD.
	If ZSTD, high 4 bits will contain ZSTD Strategy (1-9).
18	LZMA Properties using LZMA2 encoding for dictionary size only
or
18	ZPAQ Properties. 0b0CCCBBBB where CCC is compression level, BBBB is
	block size. (high bit set is no longer needed)
or
18	BZIP3 Properties. 0b0000BBBB where BBBB is 0-8. Allowable BZIP3 block
	sizes.
or
18	ZSTD Properties. 0b000CCCCC where CCCCC is 1-22. ZSTD Strategy will
	be stored in high order bits of CTYPE (1-9).
19	lrzip-next compression levels, 0bRRRRLLLL
	high order bits Rzip compression level (magic >> 4)
	low order bits Lrzip compression level (magic & 15)
20	Comment length. 0 means no comment.
21-84	Comment 64 chars max

Encrypted salt (bytes 6->13 in magic if encrypted):
0->1	Encoded number of loops to hash password
2->7	Random data
(RCD0 is set to 8 bytes always on encrypted files)

Rzip Chunk Data:
0	Data offsets byte width (meaning length is < (2 * 8)^RCD0)
1	Flag that there is no chunk beyond this
(RCD0 bytes)	Chunk decompressed size (not stored in encrypted file)
(Header data is also encrypted in encrypted files)
XX	Stream 0 header data
XX	Stream 1 header data

Stream Header Data:
Byte:
0	Compressed data type
(RCD0 bytes)	Compressed data length
(RCD0 bytes)	Uncompressed data length
(RCD0 bytes)	Next block head

Data blocks:
0->(end-2) data
(end-1)->end crc data
(end - hash size) for md5

lrzip-next-0.10x file format
Peter Hyman
January 2023

Byte 17 retasked to include BZIP3 block size
17	LZMA Properties using LZMA2 encoding for dictionary size only
or
17	ZPAQ Properties. 0b1CCCBBBB where CCC is compression level, BBBB is
	block size. CCC can be 3-5 only. High order bit always set.
	ZPAQ Compression level 3 = 0b1011BBBB
	ZPAQ Compression level 4 = 0b1100BBBB
	ZPAQ Compression level 5 = 0b1101BBBB
or
17	BZIP3 Properties. 0b1111BBBB where BBBB is 0-8. Allowable BZIP3 block
	sizes. High order bits of 0b1111 distinguish BZIP3 block size from
	ZPAQ compression level which can only be 3-5. When all high order bits
	are set, it cannot be ZPAQ. 0b1111BBBB implies a ZPAQ level of 7 which
	is out of range.

lrzip-next-0.9x file format
Peter Hyman
July 2022

Magic extended 2 bytes.

18	lrzip-next compression levels, 0bRRRRLLLL
	high order bits Rzip compression level (magic >> 4)
	low order bits Lrzip compression level (magic  &15)
19	Comment length. 0 means no comment.
20-83	Comment 64 chars max

Byte			Content
0-19			Magic
----
20 - 83			Comment (if any)
20+Comment length	RZip Chunk Data (RCD)
RCD+			Compressed Data Blocks
end-16 or 2		hash or CRC

lrzip-next-0.8x file format
Peter Hyman
July 2021

get rid of dead space
encryption is only bytes 6-13 since encryption loops are first
two bytes of salt.
Use lzma2 byte encoding for dictionary size 0-40
Dictionary size will be decoded to fit into the old LZMA Properties
lc, lp, pb will be constants and not needed to be stored. 0x5D, 93
lc = 3, lp = 0, pb = 2
control field and encoded prior to writing the Magic Header.
zpaq level and block size will be encoded in byte 17 also
high order bit will be set (128, 0b10000000) to indicate zpaq encoded
zpaq block size will be lower 4 bits (magic & 15)
zpaq level will be bits 5,6, and 7 (only levels 3, 4, and 5 used)
(magic & (01110000)) >> 4 (divide by 16)
move up fields for md5sum and encoding.

Byte	Content
0-17	Magic
----
18+	RZip Chunk Data (RCD)
RCD+	Compressed Data Blocks
end-16 or 2	MD5 hash or CRC

Magic Data
----------
0-3	LRZI
4	LRZIP-NEXT Major Version
5	LRZIP-NEXT Minor Version
6-13	Source File Decompressed Size
or	If encrypted, Encryption Hash Loops + Salt (2 bytes + 6 bytes)
14	>=1 = md5sum hash is stored at end of the archive, otherwise CRC
	(Hashes defined in CURRENT_HASHES file)
15	>=1 = archive is encrypted (only 1 used now, but for possible future
	expansion)
16	Filtering. 0=none, x86, ARM, ARMT, PPC, SPARC, IA64, DELTA (1..7)
	high order 5 bits will contain delta offset, 0..31
	Filtering is for all compression methods.
17	LZMA Properties using LZMA2 encoding for dictionary size only
or
17	ZPAQ Properties. 0b1CCCBBBB where CCC is compression level, BBBB is
	block size.

Encrypted salt (bytes 6->13 in magic if encrypted):
0->1	Encoded number of loops to hash password
2->7	Random data
(RCD0 is set to 8 bytes always on encrypted files)

Rzip Chunk Data:
0	Data offsets byte width (meaning length is < (2 * 8)^RCD0)
1	Flag that there is no chunk beyond this
(RCD0 bytes)	Chunk decompressed size (not stored in encrypted file)
(Header data is also encrypted in encrypted files)
XX	Stream 0 header data
XX	Stream 1 header data

Stream Header Data:
Byte:
0	Compressed data type
(RCD0 bytes)	Compressed data length
(RCD0 bytes)	Uncompressed data length
(RCD0 bytes)	Next block head

Data blocks:
0->(end-2) data
(end-1)->end crc data
(end - hash size) for md5

lrzip-0.7x file format
December 2019
Peter Hyman
Filtering included, for all compression types

Byte	Content
0-23	Magic
---
24+	Rzip Chunk Data (RCD)
RCD+	Data blocks
--- repeat
(end-MD5_DIGEST_SIZE)->(end) md5 hash

Magic data:
0->3	LRZI
4	LRZIP Major Version Number
5	LRZIP Minor Version Number
6->13	Source File Size or 0 if unknown
6->13	if Encrypted, bytes 6 and 7 are hash loops, 6-13 are salt
16	Filtering. 0=none, x86, ARM, ARMT, PPC, SPARC, IA64, DELTA (1..7)
	high order 5 bits will contain delta offset, 0..31
17->21	LZMA Properties Encoded (lc,lp,pb and dictionary size)
22	1 = md5sum hash is stored at the end of the archive
23	1 = data is encrypted with sha512/aes128

lrzip-0.6x file format
March 2011
Con Kolivas

Byte	Content
0-23	Magic
---
24+	Rzip Chunk Data (RCD)
RCD+	Data blocks
--- repeat
(end-MD5_DIGEST_SIZE)->(end) md5 hash

Magic data:
0->3	LRZI
4	LRZIP Major Version Number
5	LRZIP Minor Version Number
6->14	Source File Size or 0 if unknown, or salt in encrypted file
16->20	LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)
21	1 = md5sum hash is stored at the end of the archive
22	1 = data is encrypted with sha512/aes128
23	Unused

Encrypted salt (bytes 6->14 in magic if encrypted):
0->1	Encoded number of loops to hash password
2->7	Random data
(RCD0 is set to 8 bytes always on encrypted files)

Rzip Chunk Data:
0	Data offsets byte width (meaning length is < (2 * 8)^RCD0)
1	Flag that there is no chunk beyond this
(RCD0 bytes)	Chunk decompressed size (not stored in encrypted file)
XX	Stream 0 header data
XX	Stream 1 header data

Stream Header Data:
Byte:
0	Compressed data type
(RCD0 bytes)	Compressed data length
(RCD0 bytes)	Uncompressed data length
(RCD0 bytes)	Next block head

Data blocks:
0->(end-2) data
(end-1)->end crc data


lrzip-0.5x file format
March 2011
Con Kolivas

Byte	Content
0->23	Magic
--
24->74	Rzip chunk data
75+	Data blocks
-- repeat
(end-MD5_DIGEST_SIZE)->(end) md5 hash

Magic data:
0->3	LRZI
4	LRZIP Major Version Number
5	LRZIP Minor Version Number
6->14	Source File Size
16->20	LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)
21	Flag that md5sum hash is stored at the end of the archive
22-23	not used

Rzip chunk data:
0	Data offsets byte width
1-25	Stream 0 header data
26-50	Stream 1 header data

Stream Header Data:
Byte:
0	Compressed data type
1-8	Compressed data length
9-16	Uncompressed data length
17-24	Next block head

Data blocks:
0->(end-2) data
(end-1)->end crc data


lrzip-0.40+ file header format
November 2009
Con Kolivas

Byte	Content
0-3	LRZI
4	LRZIP Major Version Number
5	LRZIP Minor Version Number
6-14	Source File Size
16-20	LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)
21-24	not used
24-48	Stream 1 header data
49-74	Stream 2 header data

Block Data:
Byte:
0	Compressed data type
1-8	Compressed data length
9-16	Uncompressed data length
17-24	Next block head
25+	Data

End:
0-1	crc data


lrzip-0.24+ file header format
January 2009
Peter Hyman, pete@peterhyman.com

Byte	Content
0-3	LRZI
4	LRZIP Major Version Number
5	LRZIP Minor Version Number
6-9	Source File Size (no HAVE_LARGE_FILES)
6-14	Source File Size
16-20	LZMA Properties Encoded (lc,lp,pb,fb, and dictionary size)
21-23	not used
24-36	Stream 1 header data
37-50	Stream 2 header data
51	Compressed data type
