- 10
- Posts
- 6
- Years
- Age 30
- Seen Oct 11, 2023
Since late February 2018, I've been looking through Pokemon Mystery Dungeon: Red Rescue Team and would like to share what I've found so far.
Leveling System
Offsets 031DD9C-0357B87 in Red and system.sbin in Blue are the level maps (headed "lvmp"). When decompressed, each of these files contain 100 definitions of 12 bytes each. Each definition is made of a 4-byte integer that defines the cumulative experience to reach a level, 6 bytes defining stat gains, and a null short. The order of the stats is HP, Attack, Special Attack, Defense, and Special Defense. The second stat gain byte is always null. It likely would have been the speed stat if it were used in this game.
Learnsets
Learnset tables are located at offsets 0360C06-036799A. The pointer table for the learnsets is located at offsets 0372600-037333F. Each Pokemon gets 2 entries. The first entry is for level up, the second is for TM's. Each entry is null terminated. In the level up entries, each attack has a 1- or 2-byte attack identifier followed by a 1-byte level. In the TM entries, the level bytes are omitted. Which identifiers correspond to which attacks is unknown. Note that each Unown has its own entries.
Cutscene Scripting
List of commands
D0 and D1 script commands are used to pick dialogue based on certain conditions. For example, when 2 or 3 different messages may appear depending on your partner, the first option uses D0 00 01 00, the second option (if there are 3) uses D0 00 03 00, and the last option (regardless of if there are 2 or 3) uses D1 00 00 00. These are preceded by CF if the resultant dialogue box is a statement or D5 for questions. The method location for D0 and D1 are the same as the command parse loop, meaning nothing is executed by these commands. Hypothesis: In the process of parsing, their pointer argument is pushed to the stack and isn't popped when execution moves on to the next command. The CF command opens a dialogue box that waits for a pointer to be pushed to the stack, then it displays the string associated with anything it finds.
Questions use a D9 command for each option. This command has a pointer to the text of the answer and a label that execution jumps to when that option is chosen. The code for these branches are seen immediately following the question that calls them. This may or may not be required in order for execution to find the labels. If the text box closes when the question is answered, the C0 command is after the label, not with the commands setting up the question box. In the unmodified code, branches created by questions typically end by jumping to the same label to return execution to a linear path. This is not required, enabling ROM hacks to set up multiple story routes based on answers given by the player.
Hypotheses:
Commands 0B, 0E, and 0F may be unused.
Commands 32, 49, and E1 are typically (maybe always) seen together and are likely related in some way.
Command 52 is commonly used near the end of the threads. It may be cleanup related.
Commands 86, 98, and 99 are used in the camera thread. They may be pan, init pan, and end pan, respectively.
Command 9B is commonly used near the end of the player thread. It may be cleanup related.
Commands B3 and CC are used in the map thread. B3 is seen before return and then a label. Skip/branch related? (18C858-18C968)
B9 appears branch related. Used many times in a row. Switch/case? Tends to appear in one scene per area, usually the third scene in ROM. Likely selects scenes and behaviors based on story flags. B7, B8, and BA may assist in this(18C858-18C968)
DD appears after 54 in Dugtrio burrow and Wobbuffet salute. Stop animation on current frame? (18C05C-18C4CC, 18D3C8-18D578)
EA is seen before branch leading to end of script. Likely branch condition (18C858-18C968)
Every thread starts with F6. This is likely a thread initialization command. Player threads follow this with some combination of E8, 2D, and E3 commands followed by DF. This may be a scene init completion signal command.
Scripts are multithreaded. There is a main thread that sets up a cutscene, dispatches threads for each character (and maybe the environment), waits for them to finish, and cleans up after the cutscene before exiting. Each thread can only affect the character associated with that thread. General actions such as dialogue boxes run in the player's thread. Synchronization is achieved using a flag system. The E4 command is used to set a flag. This is often followed by the E3 to wait for a flag. The main thread typically uses flag 3, the player thread typically uses flag 5, the partner thread typically uses flag 6, and other threads continue from there. It is possible for a thread to wait for any flag and multiple threads can wait for the same flag at once. Text data is typically located immediately after the code for the thread that references it. Each main thread has a 12-byte footer. This consists of 3 unknown bytes (in Tiny Woods they're always 900107), 5 null bytes, and a pointer to the first line of the main thread.
Map and scene data are broken up into what I've dubbed "calls". Most maps and scenes have a single call. The first call is run with 0C in the main thread during scene initialization. Subsequent calls are run with 0D by one of the character threads (possibly only the player thread) and result in additional characters being initialized and entering the scene.
Character IDs
Format for overworld map data for cutscenes and interactions(idle behaviors and trigger points under research) Note that the order is not important, this simply shows the typical order.
Interaction data (typical location in data unknown):
- Size of the interactable area (4 bytes)
- Location of interactable area (4 bytes)
- 3 pointers or null as applicable. All pointers go to script data and the difference between them is unknown.
- 4 null bytes
Graphics(updated)
Dungeon tiles are located at offsets 03B1CCC-049DBF3. Each set of tiles is part of a structure made of several parts:
Some of the dungeons that use these have been identified. They are (starting with 0):
Data for overworld background graphics begins at 1890000. Offsets 1890018-89153F are the pointer table. There are 0x2A5 (1890008-189000B) pairs of pointers. The first in a pair is the component's debug name (1891540-1892C6F). The second points to the beginning of the component's data. There are 4 types of components, which can easily be identified by the default debug name's suffix. Palettes have no suffix, animated excerpts have the suffix 1, block/chunk data has the suffix c, and chunk arrangement has the suffix m. The types of components used for each graphic fall into 6 main types:
Type 0- The basic type. It has a palette, block/chunk data, and chunk arrangement.
Type 1- Graphics with an excerpt. It has a palette, excerpt data, block/chunk data, and chunk arrangement. Examples include Rub-a-Dub River, Hill of the Ancients, and town areas.
Types 2 and 3- Type 0 graphics with an overlay of type 0 or 1, respectively. They have a palette, data for their overlay, block/chunk data, and chunk arrangement. Examples include underwater friend areas and Luminous Cave.
Types 4 and 5- Same as types 2 and 3, but the base graphic is type 1. The only known example is Crater (type 4).
Dream World/Dream Eater, the Aged Chambers, and the team base are special cases. Dream World and Dream Eater share block/chunk and chunk arrangement data but have different palettes. The Aged Chambers share block/chunk data but have different chunk arrangements and palettes. The team base is a series of type 1 graphics, each with multiple excerpts. The graphic used is determined by the player's type and construction progress while the excerpts used are determined by the player's species. Details are under investigation.
Palette data consists of a short representing the number of palettes, an unknown short, and the palettes. Each palette is 15 32-bit colors in RGBA format. Many graphics have several palettes that are identical but shifted over. This is likely to swap out palettes in order to animate tiles that change color but don't move.
Graphics are broken into chunks, which are further broken into 8x8-pixel blocks. Each block in a chunk and each chunk in a graphic is tiled in left-to-right rows going top-to-bottom.
Block data is preceded by a 16-byte header. This consists of the number of blocks in a chunk row (short), the number of blocks in a chunk column (short), the number of defined blocks (short), 8 unknown bytes, and the number of defined chunks (short).
Block data is 4bpp linear in reverse order. Some animated sections of a graphic are placed before the header of the main block data. These include flowers and lily pads seen in town, the sun at Hill of the Ancients, and the water wheel at Rub-a-Dub River. Their header consists of 2 shorts representing the excerpt's width and height in blocks followed by what is likely a series of integers representing the duration of each frame. What determines frame count is unknown.
In the chunk data, each block gets 2 bytes. The first byte and the 2 lsb in the second byte are the block index. Blocks are assigned consecutive indices based on their order in data. The first index is usually 1. However, in fully animated graphics and the aforementioned animated excerpts, the first index varies. For excerpts, the indices continue where the main part ended. Bit 2 of the second byte flips the block horizontally. Bit 3 of the second byte flips the block vertically. The upper nybble of the second byte selects the palette. There are several instances of tiles and palettes that look like they belong to Silent Chasm. This appears to be used as a placeholder when defining the layout of a static dungeon event map (which then uses the tiles for the relevant dungeon). Fully animated graphics such as attack animations precede the tiling data of each frame with an unidentified integer followed by a 16-byte header identical to the block data header (which is omitted from its typical location).
The data that encodes chunk order is preceded by a 12-byte header. This consists of the camera location (2 bytes, horizontal then vertical), the chunk width and height measured in blocks (2 shorts, redundant), the image width and height measured in chunks (2 bytes), an unknown integer (always 1?), and an unknown (boolean?) short. In the data, chunk values are in 3-byte pairs. If each letter is half of a byte, values ABC and abc are encoded as BC cA ab. These values are defined in rows according to the following rules:
- 0x inserts 0 (x+1)*2 times.
- 8x inserts the following pair of values x+1 times.
- Cx inserts the following x+1 pairs of values.
- Dx inserts the following x+17 pairs of values.
The resultant values are xor'ed with the respective indices of the previous row to get the indices for the current row. The first row assumes all previous indices were 0.
Misc.
Boolean 0x09 in dungeon data is true if the last staircase takes you to a rest area and false if it takes you out of the dungeon.
Short 0x04 in attack data is the strength factor. Non-damaging moves usually default to 2. This matches official Prima guide book and has been confirmed experimentally.
I've made a detailed map of the ROM and tagged the songs for sappy. I've also created a generalized AT4PX compressor/decompressor based on Zhorken's original kaomado tool. These files are attached.
I've also created a tool that automatically parses map data as it's understood so far. It should help with further research and will eventually become a full editing tool. If you want to help develop it, tag specific scenes and areas, or use it to discover anything new about map data, let me know. You can find its source code here.
My next goals are (in no particular order):
- Identify tracks 117, 119, 122, and 127 and find the tracks for Final Island, Legendary Island, Healing Forest, and Enclosed Island
- Identify the purpose/effect of each script command
- Analyze map data for idle behaviors and trigger points
- Determine Celebi's character code
- Research team base graphics
Credits:
Some sections of the ROM map and command table were taken from the Data Crystal wiki
The AT4PX tool is a derivative of Zhorken's kaomado program. I have email permission to make a derivative work.
Leveling System
Offsets 031DD9C-0357B87 in Red and system.sbin in Blue are the level maps (headed "lvmp"). When decompressed, each of these files contain 100 definitions of 12 bytes each. Each definition is made of a 4-byte integer that defines the cumulative experience to reach a level, 6 bytes defining stat gains, and a null short. The order of the stats is HP, Attack, Special Attack, Defense, and Special Defense. The second stat gain byte is always null. It likely would have been the speed stat if it were used in this game.
Learnsets
Learnset tables are located at offsets 0360C06-036799A. The pointer table for the learnsets is located at offsets 0372600-037333F. Each Pokemon gets 2 entries. The first entry is for level up, the second is for TM's. Each entry is null terminated. In the level up entries, each attack has a 1- or 2-byte attack identifier followed by a 1-byte level. In the TM entries, the level bytes are omitted. Which identifiers correspond to which attacks is unknown. Note that each Unown has its own entries.
Cutscene Scripting
List of commands
Spoiler:
D0 and D1 script commands are used to pick dialogue based on certain conditions. For example, when 2 or 3 different messages may appear depending on your partner, the first option uses D0 00 01 00, the second option (if there are 3) uses D0 00 03 00, and the last option (regardless of if there are 2 or 3) uses D1 00 00 00. These are preceded by CF if the resultant dialogue box is a statement or D5 for questions. The method location for D0 and D1 are the same as the command parse loop, meaning nothing is executed by these commands. Hypothesis: In the process of parsing, their pointer argument is pushed to the stack and isn't popped when execution moves on to the next command. The CF command opens a dialogue box that waits for a pointer to be pushed to the stack, then it displays the string associated with anything it finds.
Questions use a D9 command for each option. This command has a pointer to the text of the answer and a label that execution jumps to when that option is chosen. The code for these branches are seen immediately following the question that calls them. This may or may not be required in order for execution to find the labels. If the text box closes when the question is answered, the C0 command is after the label, not with the commands setting up the question box. In the unmodified code, branches created by questions typically end by jumping to the same label to return execution to a linear path. This is not required, enabling ROM hacks to set up multiple story routes based on answers given by the player.
Hypotheses:
Commands 0B, 0E, and 0F may be unused.
Commands 32, 49, and E1 are typically (maybe always) seen together and are likely related in some way.
Command 52 is commonly used near the end of the threads. It may be cleanup related.
Commands 86, 98, and 99 are used in the camera thread. They may be pan, init pan, and end pan, respectively.
Command 9B is commonly used near the end of the player thread. It may be cleanup related.
Commands B3 and CC are used in the map thread. B3 is seen before return and then a label. Skip/branch related? (18C858-18C968)
B9 appears branch related. Used many times in a row. Switch/case? Tends to appear in one scene per area, usually the third scene in ROM. Likely selects scenes and behaviors based on story flags. B7, B8, and BA may assist in this(18C858-18C968)
DD appears after 54 in Dugtrio burrow and Wobbuffet salute. Stop animation on current frame? (18C05C-18C4CC, 18D3C8-18D578)
EA is seen before branch leading to end of script. Likely branch condition (18C858-18C968)
Every thread starts with F6. This is likely a thread initialization command. Player threads follow this with some combination of E8, 2D, and E3 commands followed by DF. This may be a scene init completion signal command.
Scripts are multithreaded. There is a main thread that sets up a cutscene, dispatches threads for each character (and maybe the environment), waits for them to finish, and cleans up after the cutscene before exiting. Each thread can only affect the character associated with that thread. General actions such as dialogue boxes run in the player's thread. Synchronization is achieved using a flag system. The E4 command is used to set a flag. This is often followed by the E3 to wait for a flag. The main thread typically uses flag 3, the player thread typically uses flag 5, the partner thread typically uses flag 6, and other threads continue from there. It is possible for a thread to wait for any flag and multiple threads can wait for the same flag at once. Text data is typically located immediately after the code for the thread that references it. Each main thread has a 12-byte footer. This consists of 3 unknown bytes (in Tiny Woods they're always 900107), 5 null bytes, and a pointer to the first line of the main thread.
Map and scene data are broken up into what I've dubbed "calls". Most maps and scenes have a single call. The first call is run with 0C in the main thread during scene initialization. Subsequent calls are run with 0D by one of the character threads (possibly only the player thread) and result in additional characters being initialized and entering the scene.
Character IDs
Spoiler:
Format for overworld map data for cutscenes and interactions(idle behaviors and trigger points under research) Note that the order is not important, this simply shows the typical order.
Spoiler:
Interaction data (typical location in data unknown):
- Size of the interactable area (4 bytes)
- Location of interactable area (4 bytes)
- 3 pointers or null as applicable. All pointers go to script data and the difference between them is unknown.
- 4 null bytes
Graphics(updated)
Dungeon tiles are located at offsets 03B1CCC-049DBF3. Each set of tiles is part of a structure made of several parts:
Spoiler:
Some of the dungeons that use these have been identified. They are (starting with 0):
Spoiler:
Data for overworld background graphics begins at 1890000. Offsets 1890018-89153F are the pointer table. There are 0x2A5 (1890008-189000B) pairs of pointers. The first in a pair is the component's debug name (1891540-1892C6F). The second points to the beginning of the component's data. There are 4 types of components, which can easily be identified by the default debug name's suffix. Palettes have no suffix, animated excerpts have the suffix 1, block/chunk data has the suffix c, and chunk arrangement has the suffix m. The types of components used for each graphic fall into 6 main types:
Type 0- The basic type. It has a palette, block/chunk data, and chunk arrangement.
Type 1- Graphics with an excerpt. It has a palette, excerpt data, block/chunk data, and chunk arrangement. Examples include Rub-a-Dub River, Hill of the Ancients, and town areas.
Types 2 and 3- Type 0 graphics with an overlay of type 0 or 1, respectively. They have a palette, data for their overlay, block/chunk data, and chunk arrangement. Examples include underwater friend areas and Luminous Cave.
Types 4 and 5- Same as types 2 and 3, but the base graphic is type 1. The only known example is Crater (type 4).
Dream World/Dream Eater, the Aged Chambers, and the team base are special cases. Dream World and Dream Eater share block/chunk and chunk arrangement data but have different palettes. The Aged Chambers share block/chunk data but have different chunk arrangements and palettes. The team base is a series of type 1 graphics, each with multiple excerpts. The graphic used is determined by the player's type and construction progress while the excerpts used are determined by the player's species. Details are under investigation.
Palette data consists of a short representing the number of palettes, an unknown short, and the palettes. Each palette is 15 32-bit colors in RGBA format. Many graphics have several palettes that are identical but shifted over. This is likely to swap out palettes in order to animate tiles that change color but don't move.
Graphics are broken into chunks, which are further broken into 8x8-pixel blocks. Each block in a chunk and each chunk in a graphic is tiled in left-to-right rows going top-to-bottom.
Block data is preceded by a 16-byte header. This consists of the number of blocks in a chunk row (short), the number of blocks in a chunk column (short), the number of defined blocks (short), 8 unknown bytes, and the number of defined chunks (short).
Block data is 4bpp linear in reverse order. Some animated sections of a graphic are placed before the header of the main block data. These include flowers and lily pads seen in town, the sun at Hill of the Ancients, and the water wheel at Rub-a-Dub River. Their header consists of 2 shorts representing the excerpt's width and height in blocks followed by what is likely a series of integers representing the duration of each frame. What determines frame count is unknown.
In the chunk data, each block gets 2 bytes. The first byte and the 2 lsb in the second byte are the block index. Blocks are assigned consecutive indices based on their order in data. The first index is usually 1. However, in fully animated graphics and the aforementioned animated excerpts, the first index varies. For excerpts, the indices continue where the main part ended. Bit 2 of the second byte flips the block horizontally. Bit 3 of the second byte flips the block vertically. The upper nybble of the second byte selects the palette. There are several instances of tiles and palettes that look like they belong to Silent Chasm. This appears to be used as a placeholder when defining the layout of a static dungeon event map (which then uses the tiles for the relevant dungeon). Fully animated graphics such as attack animations precede the tiling data of each frame with an unidentified integer followed by a 16-byte header identical to the block data header (which is omitted from its typical location).
The data that encodes chunk order is preceded by a 12-byte header. This consists of the camera location (2 bytes, horizontal then vertical), the chunk width and height measured in blocks (2 shorts, redundant), the image width and height measured in chunks (2 bytes), an unknown integer (always 1?), and an unknown (boolean?) short. In the data, chunk values are in 3-byte pairs. If each letter is half of a byte, values ABC and abc are encoded as BC cA ab. These values are defined in rows according to the following rules:
- 0x inserts 0 (x+1)*2 times.
- 8x inserts the following pair of values x+1 times.
- Cx inserts the following x+1 pairs of values.
- Dx inserts the following x+17 pairs of values.
The resultant values are xor'ed with the respective indices of the previous row to get the indices for the current row. The first row assumes all previous indices were 0.
Misc.
Boolean 0x09 in dungeon data is true if the last staircase takes you to a rest area and false if it takes you out of the dungeon.
Short 0x04 in attack data is the strength factor. Non-damaging moves usually default to 2. This matches official Prima guide book and has been confirmed experimentally.
I've made a detailed map of the ROM and tagged the songs for sappy. I've also created a generalized AT4PX compressor/decompressor based on Zhorken's original kaomado tool. These files are attached.
I've also created a tool that automatically parses map data as it's understood so far. It should help with further research and will eventually become a full editing tool. If you want to help develop it, tag specific scenes and areas, or use it to discover anything new about map data, let me know. You can find its source code here.
My next goals are (in no particular order):
- Identify tracks 117, 119, 122, and 127 and find the tracks for Final Island, Legendary Island, Healing Forest, and Enclosed Island
- Identify the purpose/effect of each script command
- Analyze map data for idle behaviors and trigger points
- Determine Celebi's character code
- Research team base graphics
Credits:
Some sections of the ROM map and command table were taken from the Data Crystal wiki
The AT4PX tool is a derivative of Zhorken's kaomado program. I have email permission to make a derivative work.
Attachments
Last edited: