NKit/NKitFormat - Revision history

Nanook at 10:53, 20 September 2019

2019-09-20T10:53:14Z

← Older revision		Revision as of 10:53, 20 September 2019
Line 13:		Line 13:
	\| GameCube \|\| nkit.gcz \|\| No \|\| Yes \|\| Yes \|\| GCZ is Dolphin's own block seekable compression format		\| GameCube \|\| nkit.gcz \|\| No \|\| Yes \|\| Yes \|\| GCZ is Dolphin's own block seekable compression format
	\|-		\|-
	\| Wii \|\| nkit.iso \|\| No ~~(Yes with Dev Kit?)~~ \|\| Yes \|\| Yes \|\| RVT-H format only playable in Dolphin ~~or a Dev Kit (Untested)~~		\| Wii \|\| nkit.iso \|\| No \|\| Yes \|\| Yes \|\| RVT-H format only playable in Dolphin
	\|-		\|-
	\| Wii \|\| nkit.gcz \|\| No \|\| Yes \|\| Yes \|\| RVT-H in GCZ playable in Dolphin only		\| Wii \|\| nkit.gcz \|\| No \|\| Yes \|\| Yes \|\| RVT-H in GCZ playable in Dolphin only

Nanook at 11:47, 12 September 2019

2019-09-12T11:47:23Z

← Older revision		Revision as of 11:47, 12 September 2019
Line 18:		Line 18:
	\|}		\|}

	NKit Format images contain the bare minimum of data. All junk and scrubbing is removed. Non-uniform (NonJunk) data is preserved in 256 byte blocks with a 4 byte header. Wii encryption and hashes are fully recreatable and removed. Meaning any remaining data is as compressible possible.		NKit Format images contain the bare minimum of data. All junk and scrubbing is removed. Non-uniform (NonJunk) data is preserved in 256 byte blocks with a 4 byte header. Wii encryption and hashes are fully recreatable and removed. Meaning any remaining data is as compressible as possible.

	== Header ==		== Header ==

Nanook: /* FileSystem */

2019-09-10T14:26:55Z

FileSystem

← Older revision		Revision as of 14:26, 10 September 2019
Line 106:		Line 106:
	\| 0x0 \|\| 0x00000003 \|\| 0x0:AllJunk, 0x1:All 0x00 Scrubbed, 0x2:Mixed, 0x3:JunkFile		\| 0x0 \|\| 0x00000003 \|\| 0x0:AllJunk, 0x1:All 0x00 Scrubbed, 0x2:Mixed, 0x3:JunkFile
	\|-		\|-
	\| 0x4 \|\| 0xFFFFFFFF \|\| Optional: Caters for Gaps larger than 32 bit (Only seen in one image). Present if the value for the above mask is 0xFFFFFFFC. Add this value to the 0xFFFFFFFC for the full gap size.		\| 0x4 \|\| 0xFFFFFFFF \|\| Optional: Caters for Gaps larger than 32 bit (Only seen in one image). Present if the value for the above mask is 0xFFFFFFFC (all bits on). Add this value to the 0xFFFFFFFC for the full gap size.
	\|}		\|}

Nanook: /* FileSystem */

2019-09-10T10:34:18Z

FileSystem

← Older revision		Revision as of 10:34, 10 September 2019
Line 100:		Line 100:
	{\| class="wikitable"		{\| class="wikitable"
	\|-		\|-
	! Mask !! Value		! Offset !! Mask !! Value
	\|-		\|-
	\| 0xFFFFFFFC \|\| Size of gap without the last 2 bits set. Gaps are always to 4 byte boundaries so the last 2 bits will be 0 anyway		\| 0x0 \|\| 0xFFFFFFFC \|\| Size of gap without the last 2 bits set. Gaps are always to 4 byte boundaries so the last 2 bits will be 0 anyway
	\|-		\|-
	\| ~~0xFFFFFFFF~~ \|\| ~~Optional~~: ~~Caters for Gaps larger than 32 bit (Only seen in one image). Present if the value for the above mask is 0xFFFFFFFC. Add this value to the 0xFFFFFFFC for the full gap size.~~		\| 0x0 \|\| 0x00000003 \|\| 0x0:AllJunk, 0x1:All 0x00 Scrubbed, 0x2:Mixed, 0x3:JunkFile
	\|-		\|-
	\| ~~0x00000003~~ \|\| ~~0x0:AllJunk, 0x1:All 0x00 Scrubbed, 0x2:Mixed, 0x3~~:~~JunkFile~~		\| 0x4 \|\| 0xFFFFFFFF \|\| Optional: Caters for Gaps larger than 32 bit (Only seen in one image). Present if the value for the above mask is 0xFFFFFFFC. Add this value to the 0xFFFFFFFC for the full gap size.
	\|}		\|}

Nanook at 10:31, 10 September 2019

2019-09-10T10:31:57Z

← Older revision		Revision as of 10:31, 10 September 2019
Line 83:		Line 83:
	* Junk following the partition is encoded and stored in the NKit Format partition		* Junk following the partition is encoded and stored in the NKit Format partition
	* Partitions are then padded to 32KiB with 0x00 where the next partition follows (if any)		* Partitions are then padded to 32KiB with 0x00 where the next partition follows (if any)

			A lot of the complexity in this format is to cater for non-Redump images (scrubbed, hacks etc). Redump images don't require hash preservation, scrubbing preservation or invalid fst.bin checks. This format caters for them anyway.

	== FileSystem ==		== FileSystem ==

Nanook at 10:24, 10 September 2019

2019-09-10T10:24:05Z

← Older revision		Revision as of 10:24, 10 September 2019
Line 1:		Line 1:
			This is the technical detail of the NKit Format. It's concise and intended to be used along side the code.

	The NKit Format is a non-lossy format for shrinking and restoring Wii and GameCube images. It supports clean / Redump images as well as scrubbed and hacked images. Some corrupt and bad images are supported too, although these can error due to invalid fst.bin modifications.		The NKit Format is a non-lossy format for shrinking and restoring Wii and GameCube images. It supports clean / Redump images as well as scrubbed and hacked images. Some corrupt and bad images are supported too, although these can error due to invalid fst.bin modifications.

Nanook: /* Partition Processing */

2019-09-10T10:20:25Z

Partition Processing

← Older revision		Revision as of 10:20, 10 September 2019
Line 76:		Line 76:
	* System items are copied as is up to the fst.bin. The fst.bin will be modified to reflect new file offsets		* System items are copied as is up to the fst.bin. The fst.bin will be modified to reflect new file offsets
	* Immediately after the fst.bin follows a bit mask of flags indicating preserved hashes that didn't verify correctly. The length of this mask is 1 byte for every 8 groups in the partition. Add 1 byte if the number of groups isn't divisible by 8		* Immediately after the fst.bin follows a bit mask of flags indicating preserved hashes that didn't verify correctly. The length of this mask is 1 byte for every 8 groups in the partition. Add 1 byte if the number of groups isn't divisible by 8
	* The filesystem is compacted with the NKit Format		* The filesystem is compacted with the NKit Format, by encoding the gaps between files
	* If the fst.bin is invalid then the whole filesystem (post fst) is encoded as a gap. This is known as a Bad Conversion. Typically for corrupt images. Some customs may have this issue. Bad Conversions often error too as not all possibilities are convertable		* If the fst.bin is invalid then the whole filesystem (post fst) is encoded as a gap. This is known as a Bad Conversion. Typically for corrupt images. Some customs may have this issue. Bad Conversions often error too as not all possibilities are convertable
	* Hashes that failed to verify are stored at the end of the NKit Format partition data. The length can be calculated from the hash mask. Care must be taken to calculate the last group. Only read as many hashes are there are blocks		* Hashes that failed to verify are stored at the end of the NKit Format partition data. The length can be calculated from the hash mask. Care must be taken to calculate the last group. Only read as many hashes are there are blocks

Nanook: /* Removed Update Partition */

2019-09-10T10:18:36Z

Removed Update Partition

← Older revision		Revision as of 10:18, 10 September 2019
Line 57:		Line 57:

	==== Removed Update Partition ====		==== Removed Update Partition ====
	The update partition ~~may have been~~ removed in order to save space. In this case offset 0x218 in the disc header will ~~be the~~ CRC32 of the removed partition. The removed partition will have been backed up in the 'Redump' recovery folder or in the 'NkitExtracted' recovery folder if it's not a known partition.		The update partition can be removed in order to save space. In this case offset 0x218 in the disc header will hold CRC32 of the removed partition. The removed partition will have been backed up in the 'Redump' recovery folder or in the 'NkitExtracted' recovery folder if it's not a known partition. The file name will end with this CRC

	The original partition table is backed up at 0x50000. The next partition (Data or Channel) is located at 0x58000		The original partition table is backed up at 0x50000. The next partition (Data or Channel) is located at 0x58000

Nanook at 10:15, 10 September 2019

2019-09-10T10:15:44Z

← Older revision		Revision as of 10:15, 10 September 2019
Line 17:		Line 17:
	\|}		\|}

	NKit Format images contain the bare minimum of data. All junk and scrubbing is removed. Non-uniform data is preserved in 256 byte blocks with a 4 byte header. Wii encryption and hashes are fully recreatable and removed. Meaning any remaining data is as compressible possible.		NKit Format images contain the bare minimum of data. All junk and scrubbing is removed. Non-uniform (NonJunk) data is preserved in 256 byte blocks with a 4 byte header. Wii encryption and hashes are fully recreatable and removed. Meaning any remaining data is as compressible possible.

	== Header ==		== Header ==

Nanook at 10:15, 10 September 2019

2019-09-10T10:15:01Z

← Older revision		Revision as of 10:15, 10 September 2019
Line 12:		Line 12:
	\| GameCube \|\| nkit.gcz \|\| No \|\| Yes \|\| Yes \|\| GCZ is Dolphin's own block seekable compression format		\| GameCube \|\| nkit.gcz \|\| No \|\| Yes \|\| Yes \|\| GCZ is Dolphin's own block seekable compression format
	\|-		\|-
	\| Wii \|\| nkit.iso \|\| No (Dev Kit?) \|\| Yes \|\| Yes \|\| RVT-H format only playable in Dolphin or a Dev Kit (Untested)		\| Wii \|\| nkit.iso \|\| No (Yes with Dev Kit?) \|\| Yes \|\| Yes \|\| RVT-H format only playable in Dolphin or a Dev Kit (Untested)
	\|-		\|-
	\| Wii \|\| nkit.gcz \|\| No \|\| Yes \|\| Yes \|\| RVT-H in GCZ playable in Dolphin only		\| Wii \|\| nkit.gcz \|\| No \|\| Yes \|\| Yes \|\| RVT-H in GCZ playable in Dolphin only