• Our software update is now concluded. You will need to reset your password to log in. In order to do this, you will have to click "Log in" in the top right corner and then "Forgot your password?".
  • Forum moderator applications are now open! Click here for details.
  • Welcome to PokéCommunity! Register now and join one of the best fan communities on the 'net to talk Pokémon and more! We are not affiliated with The Pokémon Company or Nintendo.

Tool: Nerketur's Pile of Tools

2
Posts
10
Years
  • Age 24
  • Seen Oct 11, 2014
This thread is dead, right? That's bad, I realy liked this tool! Why are you not working on this anymore... :'(
Please, work on this AWESOME tool again!
 

DrFuji

[I]Heiki Hecchara‌‌[/I]
1,691
Posts
14
Years
KB13, please don't bump threads, especially if you think they're inactive.

Nerketur, if you would like this thread to be reopened then just message either myself or karatekid552.
 

Nerketur

PokéScripter
104
Posts
13
Years
Maaan, it's been a while. A LONG while. over a year since I last posted here. Over a year even since I last signed in! So all of you need a well-deserved update.

First of all, oh god I have barely any time to spend on this anymore. But I stand by what I originally said. I will see this through to completion! Whether or not I'm active here, know that I have absolutely every intention to stick this through to the end, even if it takes YEARS to complete. I'm that serious.

Secondly, I've made a few decisions about what games I'm going to include support for. It will be able to parse Explorers of Time/Darkness/Sky out-of-the-box. I will focus my time on these games, finding out all the codes, and figuring out what they do. Blue Rescue Team will be supported as well, but it will most likely not be in the beta, and I'm going to definitely need a bit of help on that. I don't plan to support Red Rescue team or anything with 3DS and up, but that plan may change in the future.

Thirdly... Since I know my timing here will probably be a bit sporadic, I DO in fact plan to release my source code soon. I figure having it open will allow other more experienced hackers to comment on making it more efficient and because people can just disassemble it anyway, so there's no real reason to hide it. I just want it to be somewhat efficent before I do. Right now it work, but It's not really modular enough

Fourthly... I don't usually like to do this, but I'd like to officially request anyone willing to help to really help me with figuring out the more tricky opcodes. It would be nice to have the debug version of NO$GBA or any other emulator with debugging capability, so I can more easily figure out the codes myself. What free emulators include this functionality? Does anyone have recommendations? It would make all this easier for sure. I'll do some searching on my own, too, so I can do better than just edit and restart emulation.

All that said, I'm rather excited to be working on this again, finally, after so long! I don't fully understand everything, and might not ever really do so, but at least I'm trying. I was the first to start this venture into hacking PMD EoT, and i hopefully won't be the last. Soon enough we will see Pokemon hacks of games other than the canon series! And man, that in and of itself makes me happy. That is why, even if this tool doesn't turn out to be the best, at least it's a damn good start! Famous already huh? =P

I plan to give somewhat regular updates now, though if you don't see me for a week don't fret. It's hard to keep me down but I do have a thesis to work on and research to do, so I might not be able to work on it every day like I used to. Thank you for all the support so far, and I hope for happy times in the future! Lets make PMD hackable together!

PS: I was right, the goto locations were off because of the way I implemented actions. Man, that was kind of silly. Oh well, it's completely fixed now, gotos are correct and now it's just correcting the incorrect commands. So I'm actually rather close. =D
 

Platinum Lucario

The Legendary Master of [color=#D8D48C]Light[/colo
1,607
Posts
16
Years
Fourthly... I don't usually like to do this, but I'd like to officially request anyone willing to help to really help me with figuring out the more tricky opcodes. It would be nice to have the debug version of NO$GBA or any other emulator with debugging capability, so I can more easily figure out the codes myself. What free emulators include this functionality? Does anyone have recommendations? It would make all this easier for sure. I'll do some searching on my own, too, so I can do better than just edit and restart emulation.

I actually have the debug version of No$GBA 2.6a, but despite it's debugging functions... either I don't know how to operate the debugging functions or they just simply don't work at all. It's a very old program and very buggy.

Anyways, best of luck with your PMD tool, someday I might use it to change it's scripts one way or another. I've also been rather curious of the sound data in the ROMs of Time, Darkness and Sky, as they have a completely different way of using it. In fact they don't have an .sdat, as you may already know. They have three folders, which are, BGM, SE and ME, all using a completely different format to what .sdat files use.
 

Nerketur

PokéScripter
104
Posts
13
Years
Small tiny update.

I might release more and more alphas. Currently my version is slightly more modular, and can semi-parse SSA, SSE, and SSS files. I found out a bit more about them too! Going to be updating the info thread soon! You never know, sometimes when you don't know something, small breakthroughs pave the way for big ones! All in all, just figuring out everything I DO know so as to see if I overlooked something.
 
5,256
Posts
16
Years
These tools certainly are intriguing! I was under the impression nobody cared enough to research into the Mystery Dungeon games, but I had always said to myself that I would dedicate some time to comb through what little documentation there is to see if I could expand upon it but never seemed to get around to it. This really is interesting though, I'm alarmed to see what you can achieve next! Keep up the great work.
 
36
Posts
10
Years
Hi ! I've been lurking for a while, and decided to try to help.

I'd like to encourage as much as I can development of tools for PMD EoS, given so little information is available! So we can all have an easier time ! I just began a little side project for PMD Explorers of Sky, a runtime(for now) sprite/data injector/editor. And, if I can find anything that you might find useful I could probably put it in this thread. Though, my main focus are the sprites and avatars.

But to get back to the topic, I think I might have a few good suggestions for helping you out if you're looking for a debugger! I don't know if you know about those things already though.

I can't find much details about a good NDS debugger, well besides vague mention of a debugger from the nds devkit and people debugging using visual studio and building DeSmuME from sources. (this also : vgcoding.blogspot dot ca/2011/01/how-to-debug-your-nds-rom-using-visual.html)

The only thing with building from source, is the amount of messing around you'd have to do to setup the whole thing properly, which can kinda sap your morale and time..
But as someone who's been messing around with Pokemon XD: Gale of Darkness for a while and dealing with the limited debugger in Dolphin, I developed some workarounds.

DeSmuME 0.9.10 has some basic features that could help out kinda like Dolphin, namely a disassembler, a memory viewer, and it can run lua scripts. Then, I highly suggest you'd get Cheat Engine. It features a great memory search feature and has data breakpoints, which are like the best thing ever ! And Cheat Engine runs lua scripts, with which you can automate tedious tasks and even write a little program/UI for doing things in cheat engine, or just running your scripts etc..

Then, the best way I found to find the data you're looking for is usually to make a few hypothesis as to how the data is stored in memory. There are only so many ways to store data, so chances are you'll guess right ! Throughout coding languages and platforms, some common data structures are universally used. Like tables, lookup tables, linked lists etc..

I hope it doesn't sounds patronizing though, its not my intention.
And of course, I'm not a great hacker or anything really. I'm mainly a C++ coder that got into reversing a short while ago XD
So there might be easier ways of doing all this ! But I hope this was of some use to you at least !
 

Nerketur

PokéScripter
104
Posts
13
Years
Hi ! I've been lurking for a while, and decided to try to help.
Yay! I love people that like to help! =)
I'd like to encourage as much as I can development of tools for PMD EoS, given so little information is available! So we can all have an easier time ! I just began a little side project for PMD Explorers of Sky, a runtime(for now) sprite/data injector/editor. And, if I can find anything that you might find useful I could probably put it in this thread. Though, my main focus are the sprites and avatars.
Actually a LOT of that could be useful. I already know where the sprites are stored, but I am unsure of the complete file format therein. Still, understanding them, and how the Pokémon are mapped to each file would be useful. And that's something found out easier with what you're doing =D
DeSmuME 0.9.10 has some basic features that could help out kinda like Dolphin, namely a disassembler, a memory viewer, and it can run lua scripts. Then, I highly suggest you'd get Cheat Engine. It features a great memory search feature and has data breakpoints, which are like the best thing ever ! And Cheat Engine runs lua scripts, with which you can automate tedious tasks and even write a little program/UI for doing things in cheat engine, or just running your scripts etc..
I've looked into DeSmuME, and it does have few good features, but I don't understand all of them that well. I think I've looked into Cheat Engine, too. If it's the one used to debug windows programs. =P The problem with using that is the games scripting engine is coded in ARM, and the program emulating the DS is running in machine code. So I'd be debugging the machine code that emulates the ARM processor, that runs ther games code, that has the script engine. It's three steps inside. Granted, once I learn how it works I can basically make my own debugger, but... well... it would just be easier if I could debug the NDS ARM code and find the script engine, then figure out what each command does from there.

Things are just never that easy, are they? =P
Then, the best way I found to find the data you're looking for is usually to make a few hypothesis as to how the data is stored in memory. There are only so many ways to store data, so chances are you'll guess right ! Throughout coding languages and platforms, some common data structures are universally used. Like tables, lookup tables, linked lists etc..
This is actually similar to how I'm looking for command length and structure + how the format of various files works. Things that I want to find out but don't know how they are store or where they are include 'dungeon data', 'battle data', 'object data', and a few others.
I hope it doesn't sounds patronizing though, its not my intention.
And of course, I'm not a great hacker or anything really. I'm mainly a C++ coder that got into reversing a short while ago XD
So there might be easier ways of doing all this ! But I hope this was of some use to you at least !
Hee! Its alright. I'm mostly an all around programmer that only got into reversing because of PMD. And yes, I think you can definitely help! Even if I'm not focusing on sprites, I already know a lot about how they are stored. I was a sprite-ripper before this! In truth, I'm kind of honored!

Though, I would have to say any help not tool-based might be better put in my other thread instead. I might even make it an overall info thread. Who knows? I basically work on what interests me. =P
 
173
Posts
11
Years
  • Seen Jan 2, 2015
Hmm... Looks really interesting. If possible, you should make a music editor part of the program! Like, make it able to export Midi's. That would be absolutely AMAZING! There is too much good music in PMD: EoS that I'd love, but am unable to get. That would be incredible.
 
36
Posts
10
Years

Though, I would have to say any help not tool-based might be better put in my other thread instead. I might even make it an overall info thread. Who knows? I basically work on what interests me. =P
Alright then. I'll post what I find over there ! Though, at this point I don't have much. I can't find any documentation about Crystal Tile2's method of displaying pixels which would help a little figuring out the whole thing. It also seems that some of the sprites get misaligned in that program..
Though, I still managed to find a few interesting tidbits by looking at the raw data.


Yay! I love people that like to help! =)

Actually a LOT of that could be useful. I already know where the sprites are stored, but I am unsure of the complete file format therein. Still, understanding them, and how the Pokémon are mapped to each file would be useful. And that's something found out easier with what you're doing =D

I guess you mean that m_ground.bin file ? Its really an odd file. It seems to contain "SIR0" files/chunks.
I guess its some form of container, with a "table of content" at the beginning. But it has a strange and long header.. There should be some form of identification for the every subfiles in there.

And I found out EoS is using the exact same object color palette as Pokemon Mystery Dungeon Red Rescue team on the GBA ! That really helped finding those sprites XD
I still can't figure out where the face portraits are though.. Or if they use the same palette..


I've looked into DeSmuME, and it does have few good features, but I don't understand all of them that well. I think I've looked into Cheat Engine, too. If it's the one used to debug windows programs. =P The problem with using that is the games scripting engine is coded in ARM, and the program emulating the DS is running in machine code. So I'd be debugging the machine code that emulates the ARM processor, that runs ther games code, that has the script engine. It's three steps inside. Granted, once I learn how it works I can basically make my own debugger, but... well... it would just be easier if I could debug the NDS ARM code and find the script engine, then figure out what each command does from there.

Things are just never that easy, are they? =P

What did you have trouble understanding ? Maybe I could help ?

About that, I thought you were only editing the game's script ?
Because, you can still use DeSmuMe's Disassembler to see the assembly code directly, if you really want to.
But, if you're looking only for the part that runs the scripts for the dialogue and game logic, then its easier to just look for the data used by the software to run those scripts. At least IMO.

Cheat engine is really good at finding data, and finding what data changes at what time, and is changed by what function.

But, if you have a good IDE and grab the source code for DeSmuMe and compile it yourself, and run it in debug, that would be optimal, even though it would require some work. There is also a couple of tutorials on how to setup visual studio and a windows port of the GNU GBD debugger to debug the actual ARM code, thanks to some piece of code in DeSmuMe that allows to do that.

This is actually similar to how I'm looking for command length and structure + how the format of various files works. Things that I want to find out but don't know how they are store or where they are include 'dungeon data', 'battle data', 'object data', and a few others.

Hee! Its alright. I'm mostly an all around programmer that only got into reversing because of PMD. And yes, I think you can definitely help! Even if I'm not focusing on sprites, I already know a lot about how they are stored. I was a sprite-ripper before this! In truth, I'm kind of honored!
Honored ? What do you mean ?
Its kinda sad PMD isn't more popular.. Its such a charming game, and it would have been reversed years ago if it had been more popular.. Seeing as most other pokemon game got reversed really quickly !
Actually the whole reason I got into this, is to encourage people to add in the missing faces and sprites for the non official starter pokemon! But, if I can help others along the way, why not ! :)

I'll try to dump some of my notes so far in the thread you've pointed out. They're not 100% sure things, but I guess it can help to try figuring those things out as a group !
 

Nerketur

PokéScripter
104
Posts
13
Years
Alright then. I'll post what I find over there ! Though, at this point I don't have much. I can't find any documentation about Crystal Tile2's method of displaying pixels which would help a little figuring out the whole thing. It also seems that some of the sprites get misaligned in that program..
Though, I still managed to find a few interesting tidbits by looking at the raw data.

I guess you mean that m_ground.bin file ? Its really an odd file. It seems to contain "SIR0" files/chunks.
I guess its some form of container, with a "table of content" at the beginning. But it has a strange and long header.. There should be some form of identification for the every subfiles in there.
Yes, I mean that file. I already "unpacked it" using Tinke. the sir0 files in particular are what I'm actually decoding at the moment. I know a looooooooot about it now. animations, initial locations, you name it. everything for that sprite is stored in the SIR0 file, which is within the .bin file, and I can give you the format of both.

Come to think of it, you should read through both of my threads. I explain how the .bin files work as well as a bit of the sir0 for images in this post of the other thread. Though at that point I didn't know everything that I know now.
What did you have trouble understanding ? Maybe I could help ?

In terms of DeSmuMe, I'd have to look at them again. But in terms of the SIR0 format, there's one main section. It's near the beginning, with all those FFs and F9, FC, etc. it's really weird and i cannot for the life of me logically infer what it signifies. I do know however, that the length of the section is exactly 10 * the length of the pointers to the images section (at least in the file for female Pikachu, the 26th packed file), which means there may be a connection there. I'm unsure at this point. I'm looking deeper into it myself.

About that, I thought you were only editing the game's script ?
Because, you can still use DeSmuMe's Disassembler to see the assembly code directly, if you really want to.
But, if you're looking only for the part that runs the scripts for the dialogue and game logic, then its easier to just look for the data used by the software to run those scripts. At least IMO.
I am indeed currently only working on editing the games script, but there are a few commands that I have no idea what do, and I figure if I understand the scripting engine then I'll understand what every command does. It's all to understand parameters that I don't know what they modify, like jumps and why they don't work for everything everywhere.
But, if you have a good IDE and grab the source code for DeSmuMe and compile it yourself, and run it in debug, that would be optimal, even though it would require some work. There is also a couple of tutorials on how to setup visual studio and a windows port of the GNU GBD debugger to debug the actual ARM code, thanks to some piece of code in DeSmuMe that allows to do that.
I will look into doing it this way. It may be the only way, after all, I was just hoping for an easy solution. Oh well. That's what I get for being the pioneer for hacking PMD EoT/D/S xD
Honored ? What do you mean ?
Its kinda sad PMD isn't more popular.. Its such a charming game, and it would have been reversed years ago if it had been more popular.. Seeing as most other pokemon game got reversed really quickly !
Actually the whole reason I got into this, is to encourage people to add in the missing faces and sprites for the non official starter pokemon! But, if I can help others along the way, why not ! :)

I'll try to dump some of my notes so far in the thread you've pointed out. They're not 100% sure things, but I guess it can help to try figuring those things out as a group !
By honored, I mean I nerver really expected people were going to be helping me with any of this. I'm not a super-genius or anything, I'm just a programmer and even then, just a mediocre one that cares more about efficiency and short code than anything else.

Naturally, this means I'm really surprised and excited that i get the ability to work with someone that could very well be better than I am. I just started this, after all. =P

As for why I started? It alllll started because I wanted to find a way to allow people to actually battle...
Spoiler:
(spoiler'd because it gives plot points away). and the first commamnd i learned was 0x77 from Explorers of Time.

PS: The palette for each pokémon is stored in the SIR0 file, just after the image data. and the scheme is indeed basically the same, but there are a few differences.

PPS: I'll prolly update my other thread with SIR0 stuff in the future.

PPPS: i might actually remember where i saw those portraits. I'll do some searching later and find out if I was right.
 
36
Posts
10
Years

Yes, I mean that file. I already "unpacked it" using Tinke. the sir0 files in particular are what I'm actually decoding at the moment. I know a looooooooot about it now. animations, initial locations, you name it. everything for that sprite is stored in the SIR0 file, which is within the .bin file, and I can give you the format of both.

Come to think of it, you should read through both of my threads. I explain how the .bin files work as well as a bit of the sir0 for images in this post of the other thread. Though at that point I didn't know everything that I know now.

Yikes. I kinda feel a little irrelevant with my notes now XD

I looked over that thread before, but somehow I missed that part :/
And you probably figured whether it is or not by now, but the Z param might just be the alpha channel.

I didn't attempt unpacking the Bin files using tinke though. I didn't even think they could be compressed, given the file was mostly readable.. That could explain a lot of things XD
But, why does it outputs only 149 files ? I'm probably doing something wrong.. :/
I opened them as a "pack" in Tinke and extracted the content..


In terms of DeSmuMe, I'd have to look at them again. But in terms of the SIR0 format, there's one main section. It's near the beginning, with all those FFs and F9, FC, etc. it's really weird and i cannot for the life of me logically infer what it signifies. I do know however, that the length of the section is exactly 10 * the length of the pointers to the images section (at least in the file for female Pikachu, the 26th packed file), which means there may be a connection there. I'm unsure at this point. I'm looking deeper into it myself.
I was speaking about DeSmuMe. I don't know much about SIR0, given I found very little on the web for some reasons..

And those might be padding bytes maybe ? However, I never really found much evidences that the files were byte aligned at all. Or any reasons for them to be, given they're on a flash catridge, not on a physical disc :/


I am indeed currently only working on editing the games script, but there are a few commands that I have no idea what do, and I figure if I understand the scripting engine then I'll understand what every command does. It's all to understand parameters that I don't know what they modify, like jumps and why they don't work for everything everywhere.

Oh, well, its just that at one point I wasn't sure what you were after, when you started mentioning OP codes and the ARM architecture. I thought you meant literally cpu op codes, but scripts are much higher level than that. I just wanted to clarify that.


I will look into doing it this way. It may be the only way, after all, I was just hoping for an easy solution. Oh well. That's what I get for being the pioneer for hacking PMD EoT/D/S xD

There no such things as easy in the reversing world as far as I know XD
I have posted the link, but I can't until I hit 15 posts, and I don't know if mods are picky enough to take offense if I bypass that..

But honestly, you'd have to have a good knowledge of ARM assembly, and look through pages and pages of opcode to really have a lot to gain from that. I was mostly only saying that, as far as debugger goes, this is probably one of the best way to do it.


By honored, I mean I nerver really expected people were going to be helping me with any of this. I'm not a super-genius or anything, I'm just a programmer and even then, just a mediocre one that cares more about efficiency and short code than anything else.

Naturally, this means I'm really surprised and excited that i get the ability to work with someone that could very well be better than I am. I just started this, after all. =P

Oh, well, for one you're probably the only person I stumbled on still working on something for PMD:EoS ! That shows you've got motivation! And in the end, skills can be developed and improved, but motivation is another story..

I wouldn't hold my breath about me being much better ! XD Now that we discussed a little, I guess were pretty close in terms of knowledge. Though, you seem to be much farther ahead than I though in term of reversing the actual game files.


As for why I started? It alllll started because I wanted to find a way to allow people to actually battle...
Spoiler:
(spoiler'd because it gives plot points away). and the first commamnd i learned was 0x77 from Explorers of Time.

That's something I wish we could have done in the game ! Battling these guys ! But the plot just didn't want to :(
Spoiler:



PS: The palette for each pokémon is stored in the SIR0 file, just after the image data. and the scheme is indeed basically the same, but there are a few differences.

PPS: I'll prolly update my other thread with SIR0 stuff in the future.

PPPS: i might actually remember where i saw those portraits. I'll do some searching later and find out if I was right.
Yeah, I saw those palettes, but they seemed incomplete for each pokemon oddly. They just had some of the most preeminent color on the pokemon. Or at least, from what I saw they did. While the palette extracted from red rescue team using VBALink had the entire correct lines for most pokemon. And the palette looked the same as the one DeSmuMe would read from the EoS game.(I'm not even sure why it won't let me build a palette from the current scene and export it, instead of just displaying it..)

And that sounds good! If you can get the specs written down, I might be able to write a little program to display them!
Btw, I was wondering, what language are you using to code this in ? Because, I could probably write that in the same language ? I'm more partial to C++ for working with bytes and such, but it might be easier for cooperation to use the same thing ?
 

Nerketur

PokéScripter
104
Posts
13
Years
And you probably figured whether it is or not by now, but the Z param might just be the alpha channel.
Not sure what Z-param you mean here.
I didn't attempt unpacking the Bin files using tinke though. I didn't even think they could be compressed, given the file was mostly readable.. That could explain a lot of things XD
But, why does it outputs only 149 files ? I'm probably doing something wrong.. :/
I opened them as a "pack" in Tinke and extracted the content..
They aren't compressed. Unpacking is like taking items from a box they were packed in, like when you move from an old house to a new one. Decompressing is something completely different in Computer Science terms. If you look at the format, the number of files parameter is at offset 4, and that is why the files is wrong. Change the num files offset to 4, then change the "start offset" to 8 (because again, that's in the pack format). That way you'll get all the files. However, make sure you select "Start offset/length" as the kind. if you get all SIR0 files, then you unpacked it properly and should hit "apply". =) I explain this indirectly in the post I linked to at the bottom.
And those might be padding bytes maybe ? However, I never really found much evidences that the files were byte aligned at all. Or any reasons for them to be, given they're on a flash catridge, not on a physical disc :/
I sincerely doubt it. If it was padding there would be no reason to have an offset to it!
Oh, well, its just that at one point I wasn't sure what you were after, when you started mentioning OP codes and the ARM architecture. I thought you meant literally cpu op codes, but scripts are much higher level than that. I just wanted to clarify that.
I do realize this. I'm a Computer Science masters student. =)

I say op codes because the scripting is very similar to assembler where the first byte is the op code and the remaining ones are parameters. It's scripting only because it isn't ARM assembler. Granted, my terminology is a bit wonky, but eh. I'm using basically the same terminology Hackmew used, I believe.
But honestly, you'd have to have a good knowledge of ARM assembly, and look through pages and pages of opcode to really have a lot to gain from that. I was mostly only saying that, as far as debugger goes, this is probably one of the best way to do it.
All part of my plan. Honestly, there's not much else I can gain from guessing. I know almost everything I can (scriptwise) from guessing already. It's quite a lot, but it's not enough to really make a good hack. Given what I know, you can do almost anything cutscene related, except ruin other scripts. I don't know the details on doing that, sadly, even if I do know the commands used. That is why I need the debugger. So I can figure out why it just... doesn't work for certain things.
Oh, well, for one you're probably the only person I stumbled on still working on something for PMD:EoS ! That shows you've got motivation! And in the end, skills can be developed and improved, but motivation is another story..

I wouldn't hold my breath about me being much better ! XD Now that we discussed a little, I guess were pretty close in terms of knowledge. Though, you seem to be much farther ahead than I though in term of reversing the actual game files.
In terms of programming knowledge I'm rather advanced. But this is my first venture into figuring out how the scripting works in anything =P, So in that respect I'm a newbie/intermediate.
Yeah, I saw those palettes, but they seemed incomplete for each pokemon oddly. They just had some of the most preeminent color on the pokemon. Or at least, from what I saw they did. While the palette extracted from red rescue team using VBALink had the entire correct lines for most pokemon. And the palette looked the same as the one DeSmuMe would read from the EoS game.(I'm not even sure why it won't let me build a palette from the current scene and export it, instead of just displaying it..)
Incomplete? I have the complete palette in my sir0 files. 16 colors, the format of the palette is RRGGBB??*16. I open the file (currently) in YY-CHR and use that palette (which I manually remove the ?? bytes), adding the required number of bytes to make it a "full palette" of 256 colors, then load it into YY-CHR, letting me view the "official" colors for each pokémon.
And that sounds good! If you can get the specs written down, I might be able to write a little program to display them!
Btw, I was wondering, what language are you using to code this in ? Because, I could probably write that in the same language ? I'm more partial to C++ for working with bytes and such, but it might be easier for cooperation to use the same thing ?
I do have the full specs already written down but there is a lot of extra stuff in there right now. If you look at my linked post (relinked here for convenience) I explain the SIR0 files in it on a basic level. =)

As for language, I use C#. I refuse to program in C++ if I don't have to, and I honestly hate using it. C# is so much cleaner and better in so many ways. If you don't know C#, I highly recommend it! You'll never go back to C++, I guarantee it =P
 
36
Posts
10
Years

Not sure what Z-param you mean here.

Nevermind about that.


They aren't compressed. Unpacking is like taking items from a box they were packed in, like when you move from an old house to a new one. Decompressing is something completely different in Computer Science terms. If you look at the format, the number of files parameter is at offset 4, and that is why the files is wrong. Change the num files offset to 4, then change the "start offset" to 8 (because again, that's in the pack format). That way you'll get all the files. However, make sure you select "Start offset/length" as the kind. if you get all SIR0 files, then you unpacked it properly and should hit "apply". =) I explain this indirectly in the post I linked to at the bottom.

Well, I just assumed it was compressed, because I don't really see any advantage to grouping all those files into one, if its not for compression. Unless, they're just loading everything straight into memory..

Thanks for the offsets ! I got 599 of them now ! Honestly, I never really used Tinke all that much, and I'm still not very familiar with the formats on the DS.
As for the link, I can't seem to find those offset you just gave me. I know you've said indirectly, but I'm not sure what you mean in this case. :/

I sincerely doubt it. If it was padding there would be no reason to have an offset to it!

Why not ? The offset you're mentioning could be simply a "pointer" to the end of the data +1, which is used relatively commonly. The rest could be padding to align the next data block in the packed bin file ?

Or maybe its a string of some kind with a fixed length ? I know that in some of the main series games, in pokemon name fields, when some names are shorter than the 12 characters (I think), the rest is basically *pokemon name* 0x0 0xFF 0xFF ... repeat the 0xFF until all the 12 bytes are filled. It depends of the gen though. Sometimes other values might gets left in there, the so-called trash bytes.



I do realize this. I'm a Computer Science masters student. =)

Yes, I was just explaining why I was asking this. Its hard to understand eachothers if we twist new meanings for words, so I was just trying to understand what you meant. I didn't mean to belittle you or anything. Sorry if it sounded that way.. :(


I say op codes because the scripting is very similar to assembler where the first byte is the op code and the remaining ones are parameters. It's scripting only because it isn't ARM assembler. Granted, my terminology is a bit wonky, but eh. I'm using basically the same terminology Hackmew used, I believe.
Yes, but the only issue here is that opcodes commonly refer to CPU opcodes, or sometimes their mnemonics. So its slightly hard for someone relatively familiar with them to get what you mean. Well in my case anyways.
Of course, I don't mean to tell you what to do. And you just cleared everything up anyways.


All part of my plan. Honestly, there's not much else I can gain from guessing. I know almost everything I can (scriptwise) from guessing already. It's quite a lot, but it's not enough to really make a good hack. Given what I know, you can do almost anything cutscene related, except ruin other scripts. I don't know the details on doing that, sadly, even if I do know the commands used. That is why I need the debugger. So I can figure out why it just... doesn't work for certain things.
Alright, sounds good ! I wasn't sure what you needed the debugger for, and tried to find you simpler to setup alternatives, but you seem to have everything worked out already.


In terms of programming knowledge I'm rather advanced. But this is my first venture into figuring out how the scripting works in anything =P, So in that respect I'm a newbie/intermediate.
Well, same with me for reversing scripts XD

Which kinda makes me wonder, you explained those "opcodes" you were talking about earlier, and it kinda made me realize something.

Maybe the dev used some kind of virtual machine to run their game scripts ? Or used some scripting language that was compiled into bytecode?
It would explain the "opcode" like structure of the scripts you're talking about ! Given, on a VM programs are first transformed into bytecode then run by the VM. And I think some other scripting language generate intermediate bytecode like that. Not to mention that bytecode is very similar to opcodes.

Then again, I don't know enough about the scripts you've found to really say this for certain.
Those were just some random thoughts I suddenly had however. So, just take that as me brainstorming a little. XD


Incomplete? I have the complete palette in my sir0 files. 16 colors, the format of the palette is RRGGBB??*16. I open the file (currently) in YY-CHR and use that palette (which I manually remove the ?? bytes), adding the required number of bytes to make it a "full palette" of 256 colors, then load it into YY-CHR, letting me view the "official" colors for each pokémon.
Well, I was mainly talking about those shades of color before the actual sprites. I never found the full palette..
I had a few potential candidates, but I didn't feel like entering random tables of data into an image manually to see if they're forming a color palette. XD
Of course now that I can see its not a huge single potentially compressed file, it helps putting things in perspective! XP


I do have the full specs already written down but there is a lot of extra stuff in there right now. If you look at my linked post (relinked
here for convenience) I explain the SIR0 files in it on a basic level. =)
Right, but I meant the sprite animations, the headers, the palettes, etc.. In this post you linked, you only discuss the pixel format for the images.

And for the overall SIR0 file you describe it like this :
SIR0:
Some type of data
picture
data for above picture
(repeat above ad nauseum)
some pointer data
I'm sorry if I sound difficult, but that, and the pixel format leaves me a lot to figure out again. And I say "again", because you mentioned having found out about those.
I mean, I can most likely figure it out myself in a few days, but are you sure you don't have some notes somewhere you could just copy paste in here ?

If not, well thanks anyways. I'll just go through it the hard way. At least now I have the individual files to work with, thanks to your help ! :)

And if I understand correctly, all sprites are 32x32 pixels, right ? And the image's pixels are read from the file and then organized like this ? :

0x00 0x01 0x02 0x03 0x10 0x11 0x12 0x13 ...
0x04 0x05 0x06 0x07 0x14 0x15 0x16 0x17 ...
0x08 0x09 0x0A 0x0B 0x18 0x19 0x1A 0x1B ...
0x0C 0x0D 0x0E 0x0F 0x1C 0x1D 0x1E 0x1F ...
...

Then it goes on like that on 32 columns, and 32 rows or 1024 pixels of that pattern ?

And again if I understand correctly each pixel is a 4 bits long value referring to a color in the palette, and stored in bunches of 64 (8x8) over several blocks of 12 bytes that specify a data length for the pixels?
But wait, only 24 pixels would fit in 12 bytes, no ?
I guess I'll have to take a look at this when its not 3am.. >_<


As for language, I use C#. I refuse to program in C++ if I don't have to, and I honestly hate using it. C# is so much cleaner and better in so many ways. If you don't know C#, I highly recommend it! You'll never go back to C++, I guarantee it =P
I did a lot of C# already. Its a nice tool !

I'm not sure what you're talking about with the "cleaner" part though. C++ will let you do nasty things, but its up to you to avoid doing them. There are many idioms to help avoid doing unsafe, dirty things.

Namely my favorite idiom, that I can't use reliably in C# or Java unfortunately because of the garbage collector, the RAII idiom! I also love the CRTP (Curiously Recurring Template Pattern) the name especially ! The last one gave me headaches for days to try to understand why and how it worked XD

I had a really passionate teacher, and he showed us so much mind boggling stuff, and how to make incredibly clean things. And more importantly, to not code the C way in C++, and to use objects as much as possible. But honestly, some languages are better suited for some tasks, so I wouldn't say any is really much better than another. I kinda stopped being picky about languages after being forced to use languages I wasn't a big fan of in various little jobs..

With that said, I really don't mind using C#, if it would make collaboration easier ! :)

*EDIT*: Nevermind about the 0xFF thing I wrote. I thought you meant something else.. XD Given there are so many extremes like 0 and FF, maybe its some kind of opacity map ?
 
Last edited:

Nerketur

PokéScripter
104
Posts
13
Years
Well, I just assumed it was compressed, because I don't really see any advantage to grouping all those files into one, if its not for compression. Unless, they're just loading everything straight into memory..

Thanks for the offsets ! I got 599 of them now ! Honestly, I never really used Tinke all that much, and I'm still not very familiar with the formats on the DS.
As for the link, I can't seem to find those offset you just gave me. I know you've said indirectly, but I'm not sure what you mean in this case. :/
There should be 593 exactly. I'm really not sure where you got the extra 6 files. Maybe you mistyped?

I don't see much of an advantage either. I'd assume memory too, since every Pokémon has to be available at a moments notice, but I'm unsure, since dungeon info is the same way. It could just be they wanted it really hard to open! Or grouped by format, or many other reasons. Code-wise, there is a limit to the number of files in the nitro system, and I also know it can be slower so it could be either of those reasons.

Also, more a side note, but everything I know about NDS files was self-taught, specifically because of this very venture! Hehe. (Well, except for the NDS header itself. It was more of a 'look on GBATek thing') Really, other than SIR0, the files I know the most about are the .SS? files and .LSD. other than that I know close to zip. Other than the fact .bin usually means packed file.
*EDIT*: Nevermind about the 0xFF thing I wrote. I thought you meant something else.. XD Given there are so many extremes like 0 and FF, maybe its some kind of opacity map ?
3,360 bytes in total (in the 26th file (holds female pikachu)), an offset to it and after it, pointed to by the same offset group. The offsets behind it are offsets pointing to location data, the offsets after it are pointers to animation data. I'm thinking its some sort of flags for how to change the sprites, or map sprites to directions or something, since in the file I checked the size is EXACTLY 10*the LENGTH of the pointers for the images. All the evidence points to it being used. I just don't know how. When I'm done I might run the game with that section zeroed out and see what happens.

I've thoooought about opacity maps. Thats a definite possibility, honestly. Let me check. It would be the only other thing that made sense. (even though they could just do color 0 = transparant)
Yes, I was just explaining why I was asking this. Its hard to understand eachothers if we twist new meanings for words, so I was just trying to understand what you meant. I didn't mean to belittle you or anything. Sorry if it sounded that way.. :(

Yes, but the only issue here is that opcodes commonly refer to CPU opcodes, or sometimes their mnemonics. So its slightly hard for someone relatively familiar with them to get what you mean. Well in my case anyways.
Of course, I don't mean to tell you what to do. And you just cleared everything up anyways.
It's okay. In this case its the mnemonics. They made their own assembly language! =P

In fact, this is ALSO what happened with the main games too. It's similar to assembly there just as it is here. =)
Maybe the dev used some kind of virtual machine to run their game scripts ? Or used some scripting language that was compiled into bytecode?
I'm not sure its a virtual machine... but I'm almost POSITIVE it was compiled by a compiler. All the evidence points towards it, and it isn't even a very GOOD compiler. Quite a bit of redundancy, but it explains a lot of things. Like all the u?XXXX.ssb files! everything has the same 'structure', and honestly would be pretty easy (comparatively) to reverse back to the source. (which is usually the case with scripting languages, and languages like Java and C# which ARE basically VMs)
Well, I was mainly talking about those shades of color before the actual sprites. I never found the full palette..
<snip>
Right, but I meant the sprite animations, the headers, the palettes, etc.. In this post you linked, you only discuss the pixel format for the images.
<snip>
...are you sure you don't have some notes somewhere you could just copy paste in here ?
I do, but I'd have to tweak it to post on here. It's LONG because I am literally labeling all the different offsetts with what they do and its probably a bit confusing as is. but it's far more complete. It would be MUCH better to ... Actually, I'll use pastebin. http://pastebin.com/8btvKGDu
I did a lot of C# already. Its a nice tool !

</snip>

With that said, I really don't mind using C#, if it would make collaboration easier ! :)
Firstly... Ahh, language wars. This will be spoiler'd since its mostly off topic.
Spoiler:

Secondly, I'm very glad your teacher did so, but my preferred language to code this in is definitely C#. So I'm glad you like it too! =D
 
Last edited:
36
Posts
10
Years
Well in tinke, I set "Read at" to 4, and "Start offset" to 8. And left everything else to default. And it gives me 599 files. Except, if I press calculate, it climbs way up to 1216 files... That thing is kinda buggy.

And as for the reasons to pack files together, if there's a filesystem file limit, that's probably it. And yeah, I really need to brush up on my NDS knowledge..

About the transparency thing. They could use color 0 as transparency. I can't remember seeing sprites with semi-transparent parts.. But the UI has many parts that are half transparent and such. So I guess that yeah, zeroing it out, or only half of it would pretty much settle this.

Yeah, I'm not sure why I said VM.. So this is probably some script compiled to bytecode, then compiled and run just-in-time ? That makes sense. And its hard to reverse back to the source if we don't know what the original language is :/

And thanks for the notes. So far, I don't get too much of it, but I'll eventually figure them out! XD

Oh, and I didn't mean to start a war of some kind.. But alright, I'll reply to you via pm.

*EDIT*: Ok, so I counted all the mentions of SIR0 in the m_ground.bin file, and there are indeed 599 of them, not 593 as you said earlier. And in the header, the number of file is 0x0257 which is 599. I began writing something too for loading the sprites and all, though I'm finding out my C# is rusty a little..
However, at this point, maybe I should make a separate thread :/

But, tell me, what do you think I could do right now that could help you out ?
 
Last edited:
10
Posts
9
Years
  • Age 26
  • Seen Aug 29, 2015
I require assistance. There's a problem I have encounted while testing one of the areas. The beginning character says the first three letters of the file, such as "M02" Then, the sequential characters say the last three letters of the senquencing lines. Such as this...
Chatot:M02
Chatot:oor.
Chatot:ork.
[partner]:ere.
Chatot:ide!
And so on and so forth... It's in an area majorly part of the story, so I can't turn a blind eye... Do you know what this is?
 

Nerketur

PokéScripter
104
Posts
13
Years
I'm fairly sure I know exactly what happened there. You likely decided to change the number of "constants" when the number was originally none. And if so, then that's a bug I diddn't realize existed until quite recently, but it should be fixed in my new update, which "should" be released by new years. But nno promises on that. For now, changing the number of constants can mess things up, because of a slight error I made when calculating how the file was put together. (And I subsequently fixed it when I realized what I was doing, getting the WRONG offsets, so the whole thing screws up, even though my program will always read it correctly, because the game uses the offsets.)

May I know which scene you're speaking of? At least then I can reproduce what you did and see if it really is fixed :P
 
10
Posts
9
Years
  • Age 26
  • Seen Aug 29, 2015
Thanks for replying. It's the scene where Chatot takes you and your partner to the guild's second underground floor to meet with Wigglytuff and register the team. It's in folder G01P04A. The error was first found in the file m02a0109.ssb, but I recently found it again in m02a0202.ssb, where the first guild briefing is held. So I've suspected that the error affects the entire folder. Not anywhere else, as they work fine. And only the text is affected, not the positions or anything like that.

I gotta admit, I didn't think you'd reply this fast. I'm grateful you did <3
 

Nerketur

PokéScripter
104
Posts
13
Years
Yeah, it's the error I thought. This is fixed in the current program I have that's not yet released. In the meantime, you have two options. Option A.) reload a backup of the ROM you changed. Option B.) Fix all the pointers in the changed ROM manually. This isn't excruciatingly hard, but it's really annoying. I'm just glad I figured out this bug in time (read: before I gave up and assumed such a change was impossible).

I don't remember all the conditions that cause the bug to pop up, but its REALLY obvious if that strings table or constants table ever changes. For the longest time I thought that meant you couldn't successfully add constants, but I was wrong!

Evenm so, please do continue giving me these errors if new ones pop up. I'm sure as many as I fixed, there are bound to be others that I have yet to fix :P
 
Back
Top