Research: Loading DS rom resources[c++]

[Scaryghoul]

Introduction/What I'm using this thread for
Greetings all, I'm Scaryghoul and I'm trying to do something that I can't find a whole lot of information on. I'm overall unsure whether this belongs in Game Development or R&D but I feel as though this is the correct place to be. The way I justify it in my head is I'm doing my own R&D unrelated to a game yet, and it technically fits the three forum rules. I'll probably use this thread as a place to consolidate my findings/information if you guys don't mind.

What I'm trying to do
I'm trying to not exactly extract the resources but more so use load them into my own project(It will be open source, because I hate the level of closed source around here).

Ways not to help me - Apologies if this title comes off slightly snide.
Please don't refer me to closed source tools that will extract the resources because that's not what I'm looking to do exactly. Open sourced tools would still benefit me since I could peek at their code.

What I'm looking to do
The roms I want to work with: Pokemon HG or Pokemon BW
As for what I want to extract/load from them immediately:

• Tilesets
• Music
• Player/pokemon sprites and animations
• Move animations
• Pokemon sounds

Places I've already looked/researched
So far I've only read up on the DS Rom and GBA Rom specifications here(I can't post links so remove spaces, also I apologise for circumventing your forum rules.): ht tp: // nocash . emubase . de/ gbatek. htm

I'm not exactly looking to be spoon-fed but more so a community place to get feedback and to consolidate my information.

 

[Scaryghoul]

Fantastic, my post was finally approved. Well I started the project originally that morning of the post and spent a good day or so working on it. I would edit this into the original post and it will eventually be added when I merge all data into the original post to share. Although for now I'd like to get to 15 posts so I can start posting links xD Also remember, this is acting as my dev-log(Since most of you probably know the things I'm discovering already)

Anyways, as I said in the original post, I started by finding out how the information on the DS is stored and how the roms are laid out.
So I visited these two places - They have most of the same information, but one is formatted and sectioned, and one has everything in one big link.
htt p :/ / w w w. daftcode. n et/ gbatek/ds
h ttp : / / noca sh. emubase. de/ gbatek. htm

From these I gathered information the header of roms first(Hopefully the spoiler tags will keep the information contained)

Spoiler:

Header Overview (loaded from ROM Addr 0 to Main RAM 27FFE00h on Power-up)
Address Bytes Expl.
000h 12 Game Title (Uppercase ASCII, padded with 00h)
00Ch 4 Gamecode (Uppercase ASCII, NTR-<code>) (0=homebrew)
010h 2 Makercode (Uppercase ASCII, eg. "01"=Nintendo) (0=homebrew)
012h 1 Unitcode (00h=Nintendo DS)
013h 1 Encryption Seed Select (00..07h, usually 00h)
014h 1 Devicecapacity (Chipsize = 128KB SHL nn) (eg. 7 = 16MB)
015h 9 Reserved (zero filled)
01Eh 1 ROM Version (usually 00h)
01Fh 1 Autostart (Bit2: Skip "Press Button" after Health and Safety)
(Also skips bootmenu, even in Manual mode & even Start pressed)
020h 4 ARM9 rom_offset (4000h and up, align 1000h)
024h 4 ARM9 entry_address (2000000h..23BFE00h)
028h 4 ARM9 ram_address (2000000h..23BFE00h)
02Ch 4 ARM9 size (max 3BFE00h) (3839.5KB)
030h 4 ARM7 rom_offset (8000h and up)
034h 4 ARM7 entry_address (2000000h..23BFE00h, or 37F8000h..3807E00h)
038h 4 ARM7 ram_address (2000000h..23BFE00h, or 37F8000h..3807E00h)
03Ch 4 ARM7 size (max 3BFE00h, or FE00h) (3839.5KB, 63.5KB)
040h 4 File Name Table (FNT) offset
044h 4 File Name Table (FNT) size
048h 4 File Allocation Table (FAT) offset
04Ch 4 File Allocation Table (FAT) size
050h 4 File ARM9 overlay_offset
054h 4 File ARM9 overlay_size
058h 4 File ARM7 overlay_offset
05Ch 4 File ARM7 overlay_size
060h 4 Port 40001A4h setting for normal commands (usually 00586000h)
064h 4 Port 40001A4h setting for KEY1 commands (usually 001808F8h)
068h 4 Icon_title_offset (0=None) (8000h and up)
06Ch 2 Secure Area Checksum, CRC-16 of [ [20h]..7FFFh]
06Eh 2 Secure Area Loading Timeout (usually 051Eh)
070h 4 ARM9 Auto Load List RAM Address (?)
074h 4 ARM7 Auto Load List RAM Address (?)
078h 8 Secure Area Disable (by encrypted "NmMdOnly") (usually zero)
080h 4 Total Used ROM size (remaining/unused bytes usually FFh-padded)
084h 4 ROM Header Size (4000h)
088h 38h Reserved (zero filled)
0C0h 9Ch Nintendo Logo (compressed bitmap, same as in GBA Headers)
15Ch 2 Nintendo Logo Checksum, CRC-16 of [0C0h-15Bh], fixed CF56h
15Eh 2 Header Checksum, CRC-16 of [000h-15Dh]
160h 4 Debug rom_offset (0=none) (8000h and up) ;only if debug
164h 4 Debug size (0=none) (max 3BFE00h) ;version with
168h 4 Debug ram_address (0=none) (2400000h..27BFE00h) ;SIO and 8MB
16Ch 4 Reserved (zero filled) (transferred, and stored, but not used)
170h 90h Reserved (zero filled) (transferred, but not stored in RAM)

From this we only really needed a few things but it allowed me to load the initial rom information and test what versions of roms we're loading. Not a WHOLE lot of this information is useful or relevant though since we're only aiming at a handful of roms. Anyways I got the initial rom information loaded and now my next goal was understanding the filesystem used.

To understand the filesystem I first set out to figure out what filesystem is actually used in the DS perhaps as a standard but I couldn't actually find one(Which was shocking since almost everyone refers to something called "the DS Filesystem"). In the specifications it says they use NitroROM File system so I started there.

Now my search from there lead me to an old project called PPRE(Project pokemon rom editor).
Their main website is here. h ttp : / / projectpokemon. or g/

I know this forum is familiar with this tool since I saw some threads during my search relating to this here. Anyways, they wanted to make a general purpose rom editor by modifying source code from "Nitro Explorer 2".

So I was interested in whether this project was open source so I immediately started searching around for some source code and found a post on their forums from a few years back linking to a public SVN, so their project was open source at one time. Alas the SVN link was a 404. So I went to IRC talked with some people in the PPRE irc channel and they said the project was abanonded a couple years ago.

So I looked for public forks and I found this gem
htt ps: / / github .co m /magical/ppre
This guy cloned PPRE back in 2010 and started cleaning up their code. It's not the most recent code but it will have to do because there is no more public SVN. I did a lot of code reading to get a general understanding but it's really difficult to follow the spaghetti code that PPRE is. Magical seemed to do a good job at cleaning up some of it, but a lot of it is just a giant mess. So right now I'm working on creating a general purpose rom library in c++ based off of some of the stuff in PPRE(Not going to directly port it because that'd get unnecessarily messy). Having a bit of trouble but hopefully it won't be terrible for too long.

Right now I'm working on creating a filesystem table class that can read the rom's file locations and everything so I can create a virtual filesystem within the library.
Also since I won't know if I'm reading the filetables correctly, I have to write a NARC class since it seems most of the files in pokemon roms are stored in NARC archives. Luckily that's something that is somewhat understandable from the ppre source. I'll upload the project when I get to at least reading NARC files appropriately.

 

[esperance]

You might want to look at "ndstool." If I remember correctly it is open-source, and it would do everything you would need, with a few modifications, of course.