From WikiTemp, the GBAtemp wiki
 
(24 intermediate revisions by the same user not shown)
Line 1: Line 1:
NKit supports GameCube and Wii disc images. They are named .iso, but are Nintendo's own formats. This documentation will describe known layouts and focus on key items that NKit uses or information that is not well documented elsewhere. Read the Glossary for terminology.  
+
NKit supports GameCube and Wii disc images. They are named .iso, but are Nintendo's own formats. This documentation will describe known layouts and focuses on key items that NKit uses or information that is not well documented elsewhere. Read the [[NKit/Glossary|Glossary]] for terminology.  
 +
 
 +
== Disc Information ==
  
 
Here we will take a look at the parts of the Disc NKit is interested in. Wii will be the main focus as it's the most technical with more complexity. The GameCube disc is covered in the Partition Data section of this page as the contained data is basically a GameCube image.  
 
Here we will take a look at the parts of the Disc NKit is interested in. Wii will be the main focus as it's the most technical with more complexity. The GameCube disc is covered in the Partition Data section of this page as the contained data is basically a GameCube image.  
  
Note: Offset Addresses and lengths are denoted with the 0x prefix to indicate hexadecimal.  
+
Note: Offset Addresses and lengths are denoted with the 0x prefix to indicate hexadecimal.  
  
 
Knowledge for the overall layouts was gained from code, Dolphin contributors and the docs below. This document will refer to them as Wii Docs and GameCube Docs.  
 
Knowledge for the overall layouts was gained from code, Dolphin contributors and the docs below. This document will refer to them as Wii Docs and GameCube Docs.  
  
    Wii Docs: https://wiibrew.org/wiki/Wii_Disc  
+
Wii Docs: https://wiibrew.org/wiki/Wii_Disc  
    GameCube Docs: http://hitmen.c02.at/files/yagcd/yagcd/chap13.html  
+
GameCube Docs: http://hitmen.c02.at/files/yagcd/yagcd/chap13.html  
  
Since the above documents were written the full catalogue of games has been released. Scans have been performed. This document will aim to point out confirmed facts where possible.  
+
Since the above documents were written the full catalogue of games has been released. Scans have been performed. This document will aim to point out confirmed facts where possible.
  
== Disc Information ==
+
== Disc Layout ==
  
 
The table below shows a Wii disc with an extra channel partition. A typical Wii disc is items 1 to 7 only. Each partition has a header and a data section. 0x-------- denotes that the offset follows the previous item.  
 
The table below shows a Wii disc with an extra channel partition. A typical Wii disc is items 1 to 7 only. Each partition has a header and a data section. 0x-------- denotes that the offset follows the previous item.  
Line 46: Line 48:
 
* One series of games has a _WIN partition that contains the bulk of the data. It would be in place of the Channel in this above example  
 
* One series of games has a _WIN partition that contains the bulk of the data. It would be in place of the Channel in this above example  
  
The above table is an example the offsets of the various items specified in the partition table. The partition table is located in the disc header and should be used when parsing discs.
 
  
Redump image scans show that unless they have been modified, official releases never deviate from the update partition being at 0x50000 and the first channel or data partition is at 0xF800000. There are a couple of service discs that have invalid update partitions. In this case they have an entry in the partition table, have a valid partition header and the partition data is encrypted with valid hashes. The data is just incrementing 32 bit values starting at 0 and wrapping at 0xFFFFFFFF.  
+
The above table is an example the offsets of the various items specified in the [https://wiibrew.org/wiki/Wii_Disc#Partitions_information partition table]. The partition table is located in the disc header and should be used when parsing discs.
 +
 
 +
 
 +
Redump image scans show that unless they have been modified, official releases never deviate from the update partition being at 0x50000 and the first channel or data partition is at 0xF800000. There are a couple of service discs that have invalid update partitions. In this case they have an entry in the partition table, have a valid partition header and the partition data is encrypted with valid hashes. The data is just incrementing 32 bit values starting at 0x00000000.  
  
 
=== Disc Header ===
 
=== Disc Header ===
Line 72: Line 76:
 
|}
 
|}
  
NKit adds the following items to an area that is always empty – 0x200. See the NKit Format documentation
+
The NKit Format adds it's own [[NKit/NKitFormat#Header|items]] to the header in an area that is always empty – 0x200.
  
 
=== Partition Header ===
 
=== Partition Header ===
  
Most of the info in the partition header are ignored by NKit. The following are used:  
+
Most of the info in the [https://wiibrew.org/wiki/Wii_Disc#Partition partition header] are ignored by NKit. The following are used:  
  
 
{| class="wikitable"
 
{| class="wikitable"
Line 87: Line 91:
 
|-
 
|-
 
| 3 || 0x2bc || 0x4 || Partition Size  
 
| 3 || 0x2bc || 0x4 || Partition Size  
 +
|-
 +
| 4 || || || H3 Table (pointed to by 0x2b4)
 
|}
 
|}
  
The Partition Size is the size of the data partition including the hash sectors.
+
The Partition Size is the size of the data partition including the hash sectors. The NKit format writes it as the size without the hashes and preserves the original size in the partition data at 0x210 (data address after dehashing).
 
+
The NKit format stores it as the size without the hashes and preserves the original size in the partition data at 0x210 (data address after dehashing).  
+
  
 
=== Partition Data ===
 
=== Partition Data ===
Line 99: Line 103:
 
==== Encryption ====
 
==== Encryption ====
  
The encryption covers the partition data only. NKit does not do anything special with it. The NKit format removes the encryption, when restoring the encryption is reapplied. When recovering an image it encrypts any repaired data groups the standard way. Scrubbing wipes the encryption with a value – usually 00 or FF. NKit can preserve scrubbing when converting.  
+
The encryption covers the partition data only. NKit does not do anything special with it. The NKit format removes the encryption, when restoring, the encryption is reapplied. When recovering an image it encrypts any repaired data groups the standard way. Scrubbing wipes the encryption with a value – usually 00 or FF. NKit can preserve scrubbing when converting.
  
 
==== Data (Hashes, Groups, Blocks and Sectors) ====
 
==== Data (Hashes, Groups, Blocks and Sectors) ====
Line 107: Line 111:
 
* Each group is 64 32KiB blocks     
 
* Each group is 64 32KiB blocks     
 
* Each block is 32 1KiB sectors     
 
* Each block is 32 1KiB sectors     
* The first sector is the hash sector – H0, H1, H2 tables    
+
* The first sector is the hash sector – H0, H1, H2 tables ([https://wiibrew.org/wiki/Wii_Disc#Encrypted Wii Docs])
 
* Sectors 2 to 32 are data     
 
* Sectors 2 to 32 are data     
 
* Offsets for files and junk generation offsets ignore the hash sectors (group 2 data start offset would be 0x7c00 = 31KiB)     
 
* Offsets for files and junk generation offsets ignore the hash sectors (group 2 data start offset would be 0x7c00 = 31KiB)     
 
* A group can be less than 2 MiB and is always split on a block boundary (32KiB)  
 
* A group can be less than 2 MiB and is always split on a block boundary (32KiB)  
 +
  
 
The hash tables are linked, they hash the data and each other. This is described in the Wii Docs. NKit obeys those rules.  
 
The hash tables are linked, they hash the data and each other. This is described in the Wii Docs. NKit obeys those rules.  
Line 116: Line 121:
 
Hashes are checked and removed when converting to NKit, hashes that don't conform are preserved so they can be restored accurately. This includes hashes that not complete due to part group scrubbing are also preserved. Redump images have 100% recreatable hashes.  
 
Hashes are checked and removed when converting to NKit, hashes that don't conform are preserved so they can be restored accurately. This includes hashes that not complete due to part group scrubbing are also preserved. Redump images have 100% recreatable hashes.  
  
NKit removes all hashes from the data partition. Non recreatable hashes found in scrubbed, hacked and corrupt images are stored at the end of the NKit partition data.  
+
 
 +
NKit removes all hashes from the data partition. Non recreatable hashes found in scrubbed, hacked and corrupt images are stored at the end of the NKit partition data.
  
 
==== GameCube Disc / Wii Raw Partition Data ====
 
==== GameCube Disc / Wii Raw Partition Data ====
  
The Wii Raw Partition Data (no encryption or hashes) is almost exactly the same as a GameCube disc. Offsets used in the data need to be multiplied by 4 for Wii whereas GameCube don't. There are other minor differences for example bi2.bin not being in a Wii Partition etc but they are not important for this document.  
+
The Wii Raw Partition Data (no encryption or hashes) is almost exactly the same as a [http://hitmen.c02.at/files/yagcd/yagcd/chap13.html GameCube disc]. Offsets used in the data need to be multiplied by 4 for Wii whereas GameCube don't. There are other minor differences for example bi2.bin not being in a Wii Partition etc, but they are not important for this document.  
 +
 
  
 
A raw data partition contains the following items  
 
A raw data partition contains the following items  
Line 166: Line 173:
 
Located at the position pointed to by 0x420 in the boot.bin  
 
Located at the position pointed to by 0x420 in the boot.bin  
  
The length is calculated by looping 18 4 byte values. Based at 0x90 and 0x0. Store the greatest 0x90 based value where the 0x0 related based value is not 0. This is the only method proven to work on all images as it caters for the one image has the second value being the largest. Normally it's the last populated.  
+
The length is calculated by looping 18 dol sections and lengths. Section offsets are based at 0x0, section lengths are based at 0x90. Where the section offset value is not 0x0, add the corresponding section length to it. Store the largest value over the 18 items. This is effectively finding the '''last''' dol offset and adding the length. This is the only reliable method proven to work on all images as it caters for the one image has the second value being the largest. Normally it's the last populated.
  
 
===== Fst.bin =====
 
===== Fst.bin =====
Line 178: Line 185:
 
NKit modifies this file when compacting or restoring images for Wii and GameCube.  
 
NKit modifies this file when compacting or restoring images for Wii and GameCube.  
  
'''Wii:''' Only hacks and customs have modified file positions which are not recoverable due to the file content being modified. NKit recovery does not support moved files as there have been 0 images found that require it. Wii Recovery works at group and block level and only looks at the file system to recover gaps (junk and trailing nulls).  
+
'''Wii:''' Only hacks and customs have modified file positions that are not recoverable due to the file content being modified. NKit recovery does not support moved files as there have been 0 images found that require it. Wii Recovery works at group and block level and only looks at the file system to recover gaps (junk and trailing nulls).  
  
'''GameCube:''' There are many GameCube scrubber / compacter apps that move files. NKit can recover these using the correct recovery files.  
+
'''GameCube:''' There are many GameCube scrubber / compacter apps that move files. NKit can recover these using the correct recovery files.
  
 
===== Files =====
 
===== Files =====
This is documented on the File System page and covers Files, Gaps, Junk and alignment information.
+
This is documented under [[NKit/DiscFileSystem|DiscFileSystem]]

Latest revision as of 18:13, 25 December 2019

NKit supports GameCube and Wii disc images. They are named .iso, but are Nintendo's own formats. This documentation will describe known layouts and focuses on key items that NKit uses or information that is not well documented elsewhere. Read the Glossary for terminology.

Disc Information

Here we will take a look at the parts of the Disc NKit is interested in. Wii will be the main focus as it's the most technical with more complexity. The GameCube disc is covered in the Partition Data section of this page as the contained data is basically a GameCube image.

Note: Offset Addresses and lengths are denoted with the 0x prefix to indicate hexadecimal. 

Knowledge for the overall layouts was gained from code, Dolphin contributors and the docs below. This document will refer to them as Wii Docs and GameCube Docs.

Wii Docs: https://wiibrew.org/wiki/Wii_Disc 
GameCube Docs: http://hitmen.c02.at/files/yagcd/yagcd/chap13.html 

Since the above documents were written the full catalogue of games has been released. Scans have been performed. This document will aim to point out confirmed facts where possible.

Disc Layout

The table below shows a Wii disc with an extra channel partition. A typical Wii disc is items 1 to 7 only. Each partition has a header and a data section. 0x-------- denotes that the offset follows the previous item.

Item No. Offset Length ID Name Wii Docs
1 0x0 0x50000 Game ID Disc Header https://wiibrew.org/wiki/Wii_Disc#Header
2 0x50000 0x20000 Update [Header] https://wiibrew.org/wiki/Wii_Disc#Partition
3 0x70000 RELS Update [Data] https://wiibrew.org/wiki/Wii_Disc#Partition_Data
4 0x-------- Gap
5 0xF800000 0x20000 Data [Header] https://wiibrew.org/wiki/Wii_Disc#Partition
6 0xF820000 [Game ID] Data [Data] https://wiibrew.org/wiki/Wii_Disc#Partition_Data
7 0x-------- Gap Junk
8 0x-------- Channel [Header]
9 0x-------- _INS Channel [Data]
10 0x-------- Gap Junk

There are variants of this arrangement:-

  • Channel then Data partitions (reverse order)
  • Multiple channels (Virtual Console) + gap junk (items 8 to 10 would be repeated as 11+)
  • One series of games has a _WIN partition that contains the bulk of the data. It would be in place of the Channel in this above example


The above table is an example the offsets of the various items specified in the partition table. The partition table is located in the disc header and should be used when parsing discs.


Redump image scans show that unless they have been modified, official releases never deviate from the update partition being at 0x50000 and the first channel or data partition is at 0xF800000. There are a couple of service discs that have invalid update partitions. In this case they have an entry in the partition table, have a valid partition header and the partition data is encrypted with valid hashes. The data is just incrementing 32 bit values starting at 0x00000000.

Disc Header

The main items of interest in the disc header are:

Item No. Offset Length Name
1 0x0 0x4 ID
2 0x4 0x2 Maker Code
3 0x5 0x1 Disc No
4 0x6 0x1 Disc Version
5 0x20 0x40 Title
6 0x40000 0x120 ? Partition Info - https://wiibrew.org/wiki/Wii_Disc#Partitions_information
7 0x4E000 0x20 https://wiibrew.org/wiki/Wii_Disc#Region_setting

The NKit Format adds it's own items to the header in an area that is always empty – 0x200.

Partition Header

Most of the info in the partition header are ignored by NKit. The following are used:

Item No. Offset Length Name
1 0x2b4 0x4 H3 Table Offset
2 0x2b8 0x4 Partition Data Offset
3 0x2bc 0x4 Partition Size
4 H3 Table (pointed to by 0x2b4)

The Partition Size is the size of the data partition including the hash sectors. The NKit format writes it as the size without the hashes and preserves the original size in the partition data at 0x210 (data address after dehashing).

Partition Data

The partition data is encrypted and split into blocks containing hash data. This section will cover the encryption, hashing then file system.

Encryption

The encryption covers the partition data only. NKit does not do anything special with it. The NKit format removes the encryption, when restoring, the encryption is reapplied. When recovering an image it encrypts any repaired data groups the standard way. Scrubbing wipes the encryption with a value – usually 00 or FF. NKit can preserve scrubbing when converting.

Data (Hashes, Groups, Blocks and Sectors)

This requires a high level overview of how the partition data is set out for clarity.

  • The partition data is split in to 2MiB groups
  • Each group is 64 32KiB blocks
  • Each block is 32 1KiB sectors
  • The first sector is the hash sector – H0, H1, H2 tables (Wii Docs)
  • Sectors 2 to 32 are data
  • Offsets for files and junk generation offsets ignore the hash sectors (group 2 data start offset would be 0x7c00 = 31KiB)
  • A group can be less than 2 MiB and is always split on a block boundary (32KiB)


The hash tables are linked, they hash the data and each other. This is described in the Wii Docs. NKit obeys those rules.

Hashes are checked and removed when converting to NKit, hashes that don't conform are preserved so they can be restored accurately. This includes hashes that not complete due to part group scrubbing are also preserved. Redump images have 100% recreatable hashes.


NKit removes all hashes from the data partition. Non recreatable hashes found in scrubbed, hacked and corrupt images are stored at the end of the NKit partition data.

GameCube Disc / Wii Raw Partition Data

The Wii Raw Partition Data (no encryption or hashes) is almost exactly the same as a GameCube disc. Offsets used in the data need to be multiplied by 4 for Wii whereas GameCube don't. There are other minor differences for example bi2.bin not being in a Wii Partition etc, but they are not important for this document.


A raw data partition contains the following items

Boot.bin (Data Header)
Item No. Offset Length Name
1 0x0 0x4 ID
2 0x4 0x2 Maker Code
3 0x5 0x1 Disc No
4 0x6 0x1 Disc Version
5 0x20 0x40 Title
6 0x420 0x4 Main.dol location
7 0x424 0x4 Fst.bin location
8 0x428 0x4 Fst.bin size
9 0x42C 0x4 Max Fst.bin size

It's worth noting that the Max Fst.bin size is the same as the Fst.bin size in most cases. It can be different for 2 disc games though where the other disc has a larger fst.bin. Fst.bin size is always the size of the fst.bin on the same disc.

Bi2.bin

A GameCube file containing region info, this is not populated on the Wii. It's listed here for completeness

Appldr.bin

Extracted from GameCube images to form the Recover Files.

Located at 0x2440

The length of this file is calculated by adding the value 32 to the 4 byte values stored at 0x14 and 0x18 of the appldr.bin. 32 bytes being the length of the appldr header.

Main.dol

Not used by NKit. GameCube Recovery will scrub after this file before the fst.bin when attempting to brute force a dat CRC match.

Located at the position pointed to by 0x420 in the boot.bin

The length is calculated by looping 18 dol sections and lengths. Section offsets are based at 0x0, section lengths are based at 0x90. Where the section offset value is not 0x0, add the corresponding section length to it. Store the largest value over the 18 items. This is effectively finding the last dol offset and adding the length. This is the only reliable method proven to work on all images as it caters for the one image has the second value being the largest. Normally it's the last populated.

Fst.bin

Located at the position pointed to by 0x424 in the boot.bin

The length is located at the position pointed to by 0x428 in the boot.bin

The Fst.bin contains all the information about the GameCube disc / Wii Partition Data files positions and sizes.

NKit modifies this file when compacting or restoring images for Wii and GameCube.

Wii: Only hacks and customs have modified file positions that are not recoverable due to the file content being modified. NKit recovery does not support moved files as there have been 0 images found that require it. Wii Recovery works at group and block level and only looks at the file system to recover gaps (junk and trailing nulls).

GameCube: There are many GameCube scrubber / compacter apps that move files. NKit can recover these using the correct recovery files.

Files

This is documented under DiscFileSystem