Difference between revisions of "NKit/Discs"
(→Fst.bin) |
|||
| (23 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | NKit supports GameCube and Wii disc images. They are named .iso, but are Nintendo's own formats. This documentation will describe known layouts and | + | NKit supports GameCube and Wii disc images. They are named .iso, but are Nintendo's own formats. This documentation will describe known layouts and focuses on key items that NKit uses or information that is not well documented elsewhere. Read the [[NKit/Glossary|Glossary]] for terminology. |
| + | |||
| + | == Disc Information == | ||
Here we will take a look at the parts of the Disc NKit is interested in. Wii will be the main focus as it's the most technical with more complexity. The GameCube disc is covered in the Partition Data section of this page as the contained data is basically a GameCube image. | Here we will take a look at the parts of the Disc NKit is interested in. Wii will be the main focus as it's the most technical with more complexity. The GameCube disc is covered in the Partition Data section of this page as the contained data is basically a GameCube image. | ||
| − | Note: Offset Addresses and lengths are denoted with the 0x prefix to indicate hexadecimal. | + | Note: Offset Addresses and lengths are denoted with the 0x prefix to indicate hexadecimal. |
Knowledge for the overall layouts was gained from code, Dolphin contributors and the docs below. This document will refer to them as Wii Docs and GameCube Docs. | Knowledge for the overall layouts was gained from code, Dolphin contributors and the docs below. This document will refer to them as Wii Docs and GameCube Docs. | ||
| − | + | Wii Docs: https://wiibrew.org/wiki/Wii_Disc | |
| − | + | GameCube Docs: http://hitmen.c02.at/files/yagcd/yagcd/chap13.html | |
| − | Since the above documents were written the full catalogue of games has been released. Scans have been performed. This document will aim to point out confirmed facts where possible. | + | Since the above documents were written the full catalogue of games has been released. Scans have been performed. This document will aim to point out confirmed facts where possible. |
| − | == Disc | + | == Disc Layout == |
The table below shows a Wii disc with an extra channel partition. A typical Wii disc is items 1 to 7 only. Each partition has a header and a data section. 0x-------- denotes that the offset follows the previous item. | The table below shows a Wii disc with an extra channel partition. A typical Wii disc is items 1 to 7 only. Each partition has a header and a data section. 0x-------- denotes that the offset follows the previous item. | ||
| Line 46: | Line 48: | ||
* One series of games has a _WIN partition that contains the bulk of the data. It would be in place of the Channel in this above example | * One series of games has a _WIN partition that contains the bulk of the data. It would be in place of the Channel in this above example | ||
| − | |||
| − | Redump image scans show that unless they have been modified, official releases never deviate from the update partition being at 0x50000 and the first channel or data partition is at 0xF800000. There are a couple of service discs that have invalid update partitions. In this case they have an entry in the partition table, have a valid partition header and the partition data is encrypted with valid hashes. The data is just incrementing 32 bit values starting at | + | The above table is an example the offsets of the various items specified in the [https://wiibrew.org/wiki/Wii_Disc#Partitions_information partition table]. The partition table is located in the disc header and should be used when parsing discs. |
| + | |||
| + | |||
| + | Redump image scans show that unless they have been modified, official releases never deviate from the update partition being at 0x50000 and the first channel or data partition is at 0xF800000. There are a couple of service discs that have invalid update partitions. In this case they have an entry in the partition table, have a valid partition header and the partition data is encrypted with valid hashes. The data is just incrementing 32 bit values starting at 0x00000000. | ||
=== Disc Header === | === Disc Header === | ||
| Line 72: | Line 76: | ||
|} | |} | ||
| − | NKit adds | + | The NKit Format adds it's own [[NKit/NKitFormat#Header|items]] to the header in an area that is always empty – 0x200. |
=== Partition Header === | === Partition Header === | ||
| − | Most of the info in the partition header are ignored by NKit. The following are used: | + | Most of the info in the [https://wiibrew.org/wiki/Wii_Disc#Partition partition header] are ignored by NKit. The following are used: |
{| class="wikitable" | {| class="wikitable" | ||
| Line 87: | Line 91: | ||
|- | |- | ||
| 3 || 0x2bc || 0x4 || Partition Size | | 3 || 0x2bc || 0x4 || Partition Size | ||
| + | |- | ||
| + | | 4 || || || H3 Table (pointed to by 0x2b4) | ||
|} | |} | ||
| − | The Partition Size is the size of the data partition including the hash sectors. | + | The Partition Size is the size of the data partition including the hash sectors. The NKit format writes it as the size without the hashes and preserves the original size in the partition data at 0x210 (data address after dehashing). |
| − | + | ||
| − | The NKit format | + | |
=== Partition Data === | === Partition Data === | ||
| Line 99: | Line 103: | ||
==== Encryption ==== | ==== Encryption ==== | ||
| − | The encryption covers the partition data only. NKit does not do anything special with it. The NKit format removes the encryption, when restoring the encryption is reapplied. When recovering an image it encrypts any repaired data groups the standard way. Scrubbing wipes the encryption with a value – usually 00 or FF. NKit can preserve scrubbing when converting. | + | The encryption covers the partition data only. NKit does not do anything special with it. The NKit format removes the encryption, when restoring, the encryption is reapplied. When recovering an image it encrypts any repaired data groups the standard way. Scrubbing wipes the encryption with a value – usually 00 or FF. NKit can preserve scrubbing when converting. |
==== Data (Hashes, Groups, Blocks and Sectors) ==== | ==== Data (Hashes, Groups, Blocks and Sectors) ==== | ||
| Line 107: | Line 111: | ||
* Each group is 64 32KiB blocks | * Each group is 64 32KiB blocks | ||
* Each block is 32 1KiB sectors | * Each block is 32 1KiB sectors | ||
| − | * The first sector is the hash sector – H0, H1, H2 tables | + | * The first sector is the hash sector – H0, H1, H2 tables ([https://wiibrew.org/wiki/Wii_Disc#Encrypted Wii Docs]) |
* Sectors 2 to 32 are data | * Sectors 2 to 32 are data | ||
* Offsets for files and junk generation offsets ignore the hash sectors (group 2 data start offset would be 0x7c00 = 31KiB) | * Offsets for files and junk generation offsets ignore the hash sectors (group 2 data start offset would be 0x7c00 = 31KiB) | ||
* A group can be less than 2 MiB and is always split on a block boundary (32KiB) | * A group can be less than 2 MiB and is always split on a block boundary (32KiB) | ||
| + | |||
The hash tables are linked, they hash the data and each other. This is described in the Wii Docs. NKit obeys those rules. | The hash tables are linked, they hash the data and each other. This is described in the Wii Docs. NKit obeys those rules. | ||
| Line 116: | Line 121: | ||
Hashes are checked and removed when converting to NKit, hashes that don't conform are preserved so they can be restored accurately. This includes hashes that not complete due to part group scrubbing are also preserved. Redump images have 100% recreatable hashes. | Hashes are checked and removed when converting to NKit, hashes that don't conform are preserved so they can be restored accurately. This includes hashes that not complete due to part group scrubbing are also preserved. Redump images have 100% recreatable hashes. | ||
| − | NKit removes all hashes from the data partition. Non recreatable hashes found in scrubbed, hacked and corrupt images are stored at the end of the NKit partition data. | + | |
| + | NKit removes all hashes from the data partition. Non recreatable hashes found in scrubbed, hacked and corrupt images are stored at the end of the NKit partition data. | ||
==== GameCube Disc / Wii Raw Partition Data ==== | ==== GameCube Disc / Wii Raw Partition Data ==== | ||
| − | The Wii Raw Partition Data (no encryption or hashes) is almost exactly the same as a GameCube disc. Offsets used in the data need to be multiplied by 4 for Wii whereas GameCube don't. There are other minor differences for example bi2.bin not being in a Wii Partition etc but they are not important for this document. | + | The Wii Raw Partition Data (no encryption or hashes) is almost exactly the same as a [http://hitmen.c02.at/files/yagcd/yagcd/chap13.html GameCube disc]. Offsets used in the data need to be multiplied by 4 for Wii whereas GameCube don't. There are other minor differences for example bi2.bin not being in a Wii Partition etc, but they are not important for this document. |
| + | |||
A raw data partition contains the following items | A raw data partition contains the following items | ||
| Line 166: | Line 173: | ||
Located at the position pointed to by 0x420 in the boot.bin | Located at the position pointed to by 0x420 in the boot.bin | ||
| − | The length is calculated by looping 18 | + | The length is calculated by looping 18 dol sections and lengths. Section offsets are based at 0x0, section lengths are based at 0x90. Where the section offset value is not 0x0, add the corresponding section length to it. Store the largest value over the 18 items. This is effectively finding the '''last''' dol offset and adding the length. This is the only reliable method proven to work on all images as it caters for the one image has the second value being the largest. Normally it's the last populated. |
===== Fst.bin ===== | ===== Fst.bin ===== | ||
| Line 183: | Line 190: | ||
===== Files ===== | ===== Files ===== | ||
| − | This is documented | + | This is documented under [[NKit/DiscFileSystem|DiscFileSystem]] |
Latest revision as of 18:13, 25 December 2019
NKit supports GameCube and Wii disc images. They are named .iso, but are Nintendo's own formats. This documentation will describe known layouts and focuses on key items that NKit uses or information that is not well documented elsewhere. Read the Glossary for terminology.
Contents
Disc Information
Here we will take a look at the parts of the Disc NKit is interested in. Wii will be the main focus as it's the most technical with more complexity. The GameCube disc is covered in the Partition Data section of this page as the contained data is basically a GameCube image.
Note: Offset Addresses and lengths are denoted with the 0x prefix to indicate hexadecimal.
Knowledge for the overall layouts was gained from code, Dolphin contributors and the docs below. This document will refer to them as Wii Docs and GameCube Docs.
Wii Docs: https://wiibrew.org/wiki/Wii_Disc GameCube Docs: http://hitmen.c02.at/files/yagcd/yagcd/chap13.html
Since the above documents were written the full catalogue of games has been released. Scans have been performed. This document will aim to point out confirmed facts where possible.
Disc Layout
The table below shows a Wii disc with an extra channel partition. A typical Wii disc is items 1 to 7 only. Each partition has a header and a data section. 0x-------- denotes that the offset follows the previous item.
| Item No. | Offset | Length | ID | Name | Wii Docs |
|---|---|---|---|---|---|
| 1 | 0x0 | 0x50000 | Game ID | Disc Header | https://wiibrew.org/wiki/Wii_Disc#Header |
| 2 | 0x50000 | 0x20000 | Update [Header] | https://wiibrew.org/wiki/Wii_Disc#Partition | |
| 3 | 0x70000 | RELS | Update [Data] | https://wiibrew.org/wiki/Wii_Disc#Partition_Data | |
| 4 | 0x-------- | Gap | |||
| 5 | 0xF800000 | 0x20000 | Data [Header] | https://wiibrew.org/wiki/Wii_Disc#Partition | |
| 6 | 0xF820000 | [Game ID] | Data [Data] | https://wiibrew.org/wiki/Wii_Disc#Partition_Data | |
| 7 | 0x-------- | Gap Junk | |||
| 8 | 0x-------- | Channel [Header] | |||
| 9 | 0x-------- | _INS | Channel [Data] | ||
| 10 | 0x-------- | Gap Junk |
There are variants of this arrangement:-
- Channel then Data partitions (reverse order)
- Multiple channels (Virtual Console) + gap junk (items 8 to 10 would be repeated as 11+)
- One series of games has a _WIN partition that contains the bulk of the data. It would be in place of the Channel in this above example
The above table is an example the offsets of the various items specified in the partition table. The partition table is located in the disc header and should be used when parsing discs.
Redump image scans show that unless they have been modified, official releases never deviate from the update partition being at 0x50000 and the first channel or data partition is at 0xF800000. There are a couple of service discs that have invalid update partitions. In this case they have an entry in the partition table, have a valid partition header and the partition data is encrypted with valid hashes. The data is just incrementing 32 bit values starting at 0x00000000.
Disc Header
The main items of interest in the disc header are:
| Item No. | Offset | Length | Name |
|---|---|---|---|
| 1 | 0x0 | 0x4 | ID |
| 2 | 0x4 | 0x2 | Maker Code |
| 3 | 0x5 | 0x1 | Disc No |
| 4 | 0x6 | 0x1 | Disc Version |
| 5 | 0x20 | 0x40 | Title |
| 6 | 0x40000 | 0x120 ? | Partition Info - https://wiibrew.org/wiki/Wii_Disc#Partitions_information |
| 7 | 0x4E000 | 0x20 | https://wiibrew.org/wiki/Wii_Disc#Region_setting |
The NKit Format adds it's own items to the header in an area that is always empty – 0x200.
Partition Header
Most of the info in the partition header are ignored by NKit. The following are used:
| Item No. | Offset | Length | Name |
|---|---|---|---|
| 1 | 0x2b4 | 0x4 | H3 Table Offset |
| 2 | 0x2b8 | 0x4 | Partition Data Offset |
| 3 | 0x2bc | 0x4 | Partition Size |
| 4 | H3 Table (pointed to by 0x2b4) |
The Partition Size is the size of the data partition including the hash sectors. The NKit format writes it as the size without the hashes and preserves the original size in the partition data at 0x210 (data address after dehashing).
Partition Data
The partition data is encrypted and split into blocks containing hash data. This section will cover the encryption, hashing then file system.
Encryption
The encryption covers the partition data only. NKit does not do anything special with it. The NKit format removes the encryption, when restoring, the encryption is reapplied. When recovering an image it encrypts any repaired data groups the standard way. Scrubbing wipes the encryption with a value – usually 00 or FF. NKit can preserve scrubbing when converting.
Data (Hashes, Groups, Blocks and Sectors)
This requires a high level overview of how the partition data is set out for clarity.
- The partition data is split in to 2MiB groups
- Each group is 64 32KiB blocks
- Each block is 32 1KiB sectors
- The first sector is the hash sector – H0, H1, H2 tables (Wii Docs)
- Sectors 2 to 32 are data
- Offsets for files and junk generation offsets ignore the hash sectors (group 2 data start offset would be 0x7c00 = 31KiB)
- A group can be less than 2 MiB and is always split on a block boundary (32KiB)
The hash tables are linked, they hash the data and each other. This is described in the Wii Docs. NKit obeys those rules.
Hashes are checked and removed when converting to NKit, hashes that don't conform are preserved so they can be restored accurately. This includes hashes that not complete due to part group scrubbing are also preserved. Redump images have 100% recreatable hashes.
NKit removes all hashes from the data partition. Non recreatable hashes found in scrubbed, hacked and corrupt images are stored at the end of the NKit partition data.
GameCube Disc / Wii Raw Partition Data
The Wii Raw Partition Data (no encryption or hashes) is almost exactly the same as a GameCube disc. Offsets used in the data need to be multiplied by 4 for Wii whereas GameCube don't. There are other minor differences for example bi2.bin not being in a Wii Partition etc, but they are not important for this document.
A raw data partition contains the following items
Boot.bin (Data Header)
| Item No. | Offset | Length | Name |
|---|---|---|---|
| 1 | 0x0 | 0x4 | ID |
| 2 | 0x4 | 0x2 | Maker Code |
| 3 | 0x5 | 0x1 | Disc No |
| 4 | 0x6 | 0x1 | Disc Version |
| 5 | 0x20 | 0x40 | Title |
| 6 | 0x420 | 0x4 | Main.dol location |
| 7 | 0x424 | 0x4 | Fst.bin location |
| 8 | 0x428 | 0x4 | Fst.bin size |
| 9 | 0x42C | 0x4 | Max Fst.bin size |
It's worth noting that the Max Fst.bin size is the same as the Fst.bin size in most cases. It can be different for 2 disc games though where the other disc has a larger fst.bin. Fst.bin size is always the size of the fst.bin on the same disc.
Bi2.bin
A GameCube file containing region info, this is not populated on the Wii. It's listed here for completeness
Appldr.bin
Extracted from GameCube images to form the Recover Files.
Located at 0x2440
The length of this file is calculated by adding the value 32 to the 4 byte values stored at 0x14 and 0x18 of the appldr.bin. 32 bytes being the length of the appldr header.
Main.dol
Not used by NKit. GameCube Recovery will scrub after this file before the fst.bin when attempting to brute force a dat CRC match.
Located at the position pointed to by 0x420 in the boot.bin
The length is calculated by looping 18 dol sections and lengths. Section offsets are based at 0x0, section lengths are based at 0x90. Where the section offset value is not 0x0, add the corresponding section length to it. Store the largest value over the 18 items. This is effectively finding the last dol offset and adding the length. This is the only reliable method proven to work on all images as it caters for the one image has the second value being the largest. Normally it's the last populated.
Fst.bin
Located at the position pointed to by 0x424 in the boot.bin
The length is located at the position pointed to by 0x428 in the boot.bin
The Fst.bin contains all the information about the GameCube disc / Wii Partition Data files positions and sizes.
NKit modifies this file when compacting or restoring images for Wii and GameCube.
Wii: Only hacks and customs have modified file positions that are not recoverable due to the file content being modified. NKit recovery does not support moved files as there have been 0 images found that require it. Wii Recovery works at group and block level and only looks at the file system to recover gaps (junk and trailing nulls).
GameCube: There are many GameCube scrubber / compacter apps that move files. NKit can recover these using the correct recovery files.
Files
This is documented under DiscFileSystem
