Difference between revisions of "NKit/NKitFormat"
(→Partition Processing) |
(→Partition Processing) |
||
| Line 76: | Line 76: | ||
* The filesystem is compacted with the NKit Format | * The filesystem is compacted with the NKit Format | ||
* if the fst.bin is invalid then the whole partition is encoded as a gap. This is known as a Bad Conversion. Typically for corrupt images. Some customs may have this issue. Bad Conversions often error too as not all possibilities are convertable | * if the fst.bin is invalid then the whole partition is encoded as a gap. This is known as a Bad Conversion. Typically for corrupt images. Some customs may have this issue. Bad Conversions often error too as not all possibilities are convertable | ||
| − | * Hashes that failed to verify are stored at the end of the NKit Format partition data. The length can be calculated from the hash mask. Care must be taken to calculate the last group. Only read as many hashes are there are blocks. | + | * Hashes that failed to verify are stored at the end of the NKit Format partition data. The length can be calculated from the hash mask. Care must be taken to calculate the last group. Only read as many hashes are there are blocks |
| + | * If the partition data ID is 4 null bytes then 0x440 bytes for the header is written followed by 4 bytes indicating the original partition length. The remainder of the partition is encoded as a gap. The header is not modified. This is to cater for 2 system discs that have missing update partition data that's filled with sequential values. Decryption and hashes are valid and removed | ||
== FileSystem == | == FileSystem == | ||
Revision as of 10:36, 9 September 2019
The NKit Format is a non-lossy format for shrinking and restoring Wii and GameCube images. It supports clean / Redump images as well as scrubbed and hacked images. Some corrupt and bad images are supported too, although these quite often error.
There are 2 NKit output formats, raw (iso) and compressed (gcz). Both were designed with 1:1 preservation, size and playability in mind. Hardware support for Wii was sacrificed for smaller images. WBFS is currently the best available and Recoverable to Redump where not hacked or corrupt.
| System | Format | Hardware Supported | Dolphin Supported | Restorable 1:1 | Notes |
|---|---|---|---|---|---|
| GameCube | nkit.iso | Yes | Yes | Yes | Same as compacted GameCube iso |
| GameCube | nkit.gcz | No | Yes | Yes | GCZ is Dolphin's own block seekable compression format |
| Wii | nkit.iso | No (Dev Kit?) | Yes | Yes | RVT-H format only playable in Dolphin or a Dev Kit (Untested) |
| Wii | nkit.gcz | No | Yes | Yes | RVT-H in GCZ playable in Dolphin only |
NKit Format images contain the bare minimum of data. All junk and scrubbing is removed. Non-uniform data is preserved in 256 byte blocks with a 4 byte header. Wii encryption and hashes are fully recreatable and removed. Meaning any remaining data is as compressible possible.
Contents
Header
NKit places its header at 0x200 in the Wii and GameCube disc headers.
| Offset | Length | Name |
|---|---|---|
| 0x200 | 0x4 | NKit Header 'NKIT' |
| 0x204 | 0x4 | NKit Version ' v01' |
| 0x208 | 0x4 | Source image original CRC32 |
| 0x20C | 0x4 | NKit CRC - makes the NKit file CRC32 equal the source CRC at 0x208 (at 0x4 in GCZ) |
| 0x210 | 0x4 | Source image length |
| 0x214 | 0x4 | Forced Junk ID (When Disc ID differs - rare - GameCube only) |
| 0x218 | 0x4 | Wii Update partition CRC32 if removed when converting |
NKit also modifies the Wii header to set the bytes at 0x60 and 0x61 to 0x1. This indicates to a Dev Kit and Dolphin that the encryption and hashes aren't present.
GameCube
The disc header (Boot.bin) length is 0x440. It's modified as detailed above. The other items remain untouched.
The filesystem and fst.bin are modified when converting to NKit.
Wii
The disc header length is 0x50000. The partition tables are modified to point to new partition offsets.
Removed Update Partition
The update partition may have been removed in order to save space. In this case offset 0x218 in the disc header will be the CRC32 of the removed partition. The removed partition will have been backed up in the 'Redump' recovery folder or in the 'NkitExtracted' recovery folder if it's not a known partition.
The original partition table is backed up at 0x50000. The next partition (Data or Channel) is located at 0x58000
Partitions
The partition header length is always 0x20000. The NKit Format only modifies 1 4 byte value - the partition length (offset 0x2bc). The length is the new size of the compacted partition data and is used to restore the partition.
The partition data is unencrypted and hashes have been removed.
The new partition data length is calculated without the hashes, whereas the original includes them. The original value is preserved in the partition data at 0x210. The real RVT-H format stores 0 for the length in the header. It's unconfirmed if a Dev Kit uses this value
Partition Processing
NKit partition data information:
- Decrypted groups
- Hashes are verified and removed - if any hash sector fails in a group then all the group's hashes are cached and stored at the end of the partition
- System items are copied as is up to the fst.bin. The fst.bin will be modified to reflect new file offsets
- Immediately after the fst.bin follows a bit mask of flags indicating preserved hashes that didn't verify correctly. The length of this mask is 1 byte for every 8 groups in the partition. Add 1 byte if the number of groups isn't divisible by 8
- The filesystem is compacted with the NKit Format
- if the fst.bin is invalid then the whole partition is encoded as a gap. This is known as a Bad Conversion. Typically for corrupt images. Some customs may have this issue. Bad Conversions often error too as not all possibilities are convertable
- Hashes that failed to verify are stored at the end of the NKit Format partition data. The length can be calculated from the hash mask. Care must be taken to calculate the last group. Only read as many hashes are there are blocks
- If the partition data ID is 4 null bytes then 0x440 bytes for the header is written followed by 4 bytes indicating the original partition length. The remainder of the partition is encoded as a gap. The header is not modified. This is to cater for 2 system discs that have missing update partition data that's filled with sequential values. Decryption and hashes are valid and removed
FileSystem
Both the GameCube and Wii use the same code to encode the filesystem. The only real difference is that Wii FST file offsets and lengths must be multiplied by 4 when mapping to files. The NKit Format uses a custom Run Length Encoding (RLE) to remove the gaps between files and shrink gaps to 4 bytes in most cases.
<te be added>
