Research & Development Quick Research/Development Thread

[diegoisawesome]

Here's one thing that troubles everyone: badge hacking.
I'm talking about hacking what levels of obedience, what HM is usable, and things like that.
I'm sure that SOMEONE has done it and, hopefully, they can share it, here, with us.

[colcolstyles]

Originally Posted by diegoisawesome

Here's one thing that troubles everyone: badge hacking.
I'm talking about hacking what levels of obedience, what HM is usable, and things like that.
I'm sure that SOMEONE has done it and, hopefully, they can share it, here, with us.

I would guess that the code is spread throughout the ROM. For example, in this post, JPAN revealed that he found the routine which, during a battle, checks the player's acquired badges. However, the routine which checks what badges the player has in order to prevent the player from using Surf too early, for example, is probably located elsewhere. I don't know if the badge flags are at a fixed address in the RAM or not but if they are, you should put a break-on-read on those addresses and then disassemble the routines around wherever the game breaks.

[diegoisawesome]

Originally Posted by colcolstyles

I would guess that the code is spread throughout the ROM. For example, in this post, JPAN revealed that he found the routine which, during a battle, checks the player's acquired badges. However, the routine which checks what badges the player has in order to prevent the player from using Surf too early, for example, is probably located elsewhere. I don't know if the badge flags are at a fixed address in the RAM or not but if they are, you should put a break-on-read on those addresses and then disassemble the routines around wherever the game breaks.

The flag locations are probably DMA-protected, as with the variables...
Any other ideas?

[colcolstyles]

Originally Posted by diegoisawesome

The flag locations are probably DMA-protected, as with the variables...
Any other ideas?

You could disassemble the routine that is executed when a 'checkflag' command is encountered in a script in order to try to find where the flags are located. I've never worked with flags on the ASM-level before so I'm afraid I can't be of much help.

[diegoisawesome]

Hm... Turns out, the checkflag routine (the actual one that does the calculations) is run a lot of times in the OW (I know, duh, the people event flags) so I got the flag location (or at least, the memory pointer to it). In Emerald, it's at the address pointed at by 0x03005D8C plus 0x1270.
Now, I have to find the bit that designates the badge flags..
EDIT: 0x0809C7EC in Emerald contains the surf-check-routine... at least for the tile. I'm not sure about the PKMN menu one.
EDIT2: 0x081B54E8 (again, in Emerald) contains the badge-check-routine for the menu. I'm trying to find out where the numbers to add to the first badge are obtained from...
EDIT3: Well, apparently they're loaded from 0x02000020, but I can't find how it gets the value...
Anybody, feel free to help me out with this. :/
EDIT4: Well, I hacked the routine and made it load different flag numbers for each of the old badge+base number. And it works! :D
To get all of the flags to work out on the field, however, you'll need to edit all of the scripts for, say, Rock Smash, Strength, and Cut so that they have the new flags. And then you'll need to hack the surf routine, like I said above.
Also, with the Set Disobedience findings, all we need to control the badges completely is to find out where the Attack/Defense... stats are increased.Even though that doesn't matter much, it would still be cool to be able to control the badges completely.

[JPAN]

I found something disturbing. There are no variables above the first 0x4000 set. Variables nearing the 0x5000 set start by overwriting the pokemon data at the Breeding center, and variables from 0x5ef4 to 0x7fff are inside Box space, meaning that there is nearly no variables usable in-game.
The reason we can use those variable spaces is because the game has no variable check for values other than 0x4000 and 0x8000. So, up until now, all variables we use in the upper scale are permanently damaging the game file.
Also, trainer flags correspond to the normal flags 0x500 to 0x700.

[Chaos Rush]

Originally Posted by JPAN

I found something disturbing. There are no variables above the first 0x4000 set. Variables nearing the 0x5000 set start by overwriting the pokemon data at the Breeding center, and variables from 0x5ef4 to 0x7fff are inside Box space, meaning that there is nearly no variables usable in-game.
The reason we can use those variable spaces is because the game has no variable check for values other than 0x4000 and 0x8000. So, up until now, all variables we use in the upper scale are permanently damaging the game file.
Also, trainer flags correspond to the normal flags 0x500 to 0x700.

I really hope that's just FR/LG... because I'm generally using variables from 0x5000 to 0x7FFF in my hack.

[JPAN]

Originally Posted by Chaos Rush

I really hope that's just FR/LG... because I'm generally using variables from 0x5000 to 0x7FFF in my hack.

I've searched the other ROM versions for it, and I have bad news:
In Emerald, from 0x5536 forward affect the Boxes, and at Ruby, although not directly problematic, from 0x55a0 the variables aren't saved, and from 0x415c, you start overwriting names, sayings and Overworld data.
This fact has made me think, how can we get usable, permanent variables? So, I've studied the FlashRom write routines, and found something interesting, and since I was on a multi-ROM streak, I confirmed it for Ruby, Fire Red and Emerald.

The Pokemon games, as we know, use a 128K Flash ROM, that can be accessed by using the correct key at the 0x0e00xxxx addresses. A 128kB Flash has a total of 0x20000 possible addresses, so it uses a Bank system to allow access to the full memory. More details can be find at this useful document here.

The save File is always written in 4kB blocks (0x1000 addresses), so the pokemon Game considers a Block to be a set of 0x1000 bytes of information. Those blocks always have the same configuration:
0x0 - 0xf80 -> Data to store. (maximum spotted, can be less)
0xff4 -> Block number
0xff6 -> Checksum, calculated by adding all words in the copyed memory, then adding both Upper and lower halfword toguether.
0xff8 -> Seems like a pointer, but needs further investigation
0xffc -> a byte that seems to indicate how many saves since last load.

The Games seem to use a round-Robin list to change the location of the Save data at each save, most likely to avoid chip degradation and unfortunate hacking attempts. Emerald and Fire Red have two lists (ruby is not confirmed), that have this Saving pattern:
00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d
0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b
The game has a complex pattern of choosing the start. I'll leave you with examples:

07 08 09 0a 0b 0c 0d 00 01 02 03 04 05 06
16 17 18 19 1a 1b 0e 0f 10 11 12 13 14 15
09 0a 0b 0c 0d 00 01 02 03 04 05 06 07 08
18 19 1a 1b 0e 0f 10 11 12 13 14 15 16 17
0b 0c 0d 00 01 02 03 04 05 06 07 08 09 0a
1a 1b 0e 0f 10 11 12 13 14 15 16 17 18 19

The only data saved on these blocks are the data that in Fire Red and Emerald are located in the Dynamic pointers at 0x3005008, 0x300500c and 0x3005010.

The Fire Red Table is as follows

0x0 - Personal Data (at 0x300500c) Size: 0xf24
0x1 to 0x4 - Map Data (at 0x03005008) Size: 0xf80*3 + 0xee8 = 0x3d68
0x5 to 0xd - Box Data (at 0x03005010) Size: 0xf80*8 + 0x7d0 = 0x83d0

Both Ruby and Emerald have similar tables, although the sizes of the ending packets may vary between ROMs. Layout is always this one.

Now, why does this matter? Well, if you notice the lists presented, you can see that blocks 1c, 1d, 1e and 1f are not used, and that is true.
So, if anyone wants, we can use this area to save and load our own pieces of data, just like the game.
That can be used to increase the number of seen-caught pokemon, get some real empty Variables we can use, and even save our own data structures that we need to fit in pre-used variables to use.

As a complementary note, I will now post the location of the "Save block" Routine in the three mainly used US versions

Ruby -> 0x081DFE74
Fire Red -> 0x081DF070
Emerald -> 0x082E20AC

[diegoisawesome]

Originally Posted by JPAN

I've searched the other ROM versions for it, and I have bad news:
In Emerald, from 0x5536 forward affect the Boxes, and at Ruby, although not directly problematic, from 0x55a0 the variables aren't saved, and from 0x415c, you start overwriting names, sayings and Overworld data.
This fact has made me think, how can we get usable, permanent variables? So, I've studied the FlashRom write routines, and found something interesting, and since I was on a multi-ROM streak, I confirmed it for Ruby, Fire Red and Emerald.

The Pokemon games, as we know, use a 128K Flash ROM, that can be accessed by using the correct key at the 0x0e00xxxx addresses. A 128kB Flash has a total of 0x20000 possible addresses, so it uses a Bank system to allow access to the full memory. More details can be find at this useful document here.

The save File is always written in 4kB blocks (0x1000 addresses), so the pokemon Game considers a Block to be a set of 0x1000 bytes of information. Those blocks always have the same configuration:
0x0 - 0xf80 -> Data to store. (maximum spotted, can be less)
0xff4 -> Block number
0xff6 -> Checksum, calculated by adding all words in the copyed memory, then adding both Upper and lower halfword toguether.
0xff8 -> Seems like a pointer, but needs further investigation
0xffc -> a byte that seems to indicate how many saves since last load.

The Games seem to use a round-Robin list to change the location of the Save data at each save, most likely to avoid chip degradation and unfortunate hacking attempts. Emerald and Fire Red have two lists (ruby is not confirmed), that have this Saving pattern:
00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d
0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b
The game has a complex pattern of choosing the start. I'll leave you with examples:

07 08 09 0a 0b 0c 0d 00 01 02 03 04 05 06
16 17 18 19 1a 1b 0e 0f 10 11 12 13 14 15
09 0a 0b 0c 0d 00 01 02 03 04 05 06 07 08
18 19 1a 1b 0e 0f 10 11 12 13 14 15 16 17
0b 0c 0d 00 01 02 03 04 05 06 07 08 09 0a
1a 1b 0e 0f 10 11 12 13 14 15 16 17 18 19

The only data saved on these blocks are the data that in Fire Red and Emerald are located in the Dynamic pointers at 0x3005008, 0x300500c and 0x3005010.

The Fire Red Table is as follows

0x0 - Personal Data (at 0x300500c) Size: 0xf24
0x1 to 0x4 - Map Data (at 0x03005008) Size: 0xf80*3 + 0xee8 = 0x3d68
0x5 to 0xd - Box Data (at 0x03005010) Size: 0xf80*8 + 0x7d0 = 0x83d0

Both Ruby and Emerald have similar tables, although the sizes of the ending packets may vary between ROMs. Layout is always this one.

Now, why does this matter? Well, if you notice the lists presented, you can see that blocks 1c, 1d, 1e and 1f are not used, and that is true.
So, if anyone wants, we can use this area to save and load our own pieces of data, just like the game.
That can be used to increase the number of seen-caught pokemon, get some real empty Variables we can use, and even save our own data structures that we need to fit in pre-used variables to use.

As a complementary note, I will now post the location of the "Save block" Routine in the three mainly used US versions

Ruby -> 0x081DFE74
Fire Red -> 0x081DF070
Emerald -> 0x082E20AC

JPAN, in Emerald, what variables are safe, then? Anything under 0x5536, or are there other used places?

[JPAN]

Originally Posted by diegoisawesome

JPAN, in Emerald, what variables are safe, then? Anything under 0x5536, or are there other used places?

The main problem here is that it's impossible to truly know which are safe at this point. I would bet the ones originaly used by the game outside the ASM Routines(0x4050-0x40c0) should be safe, but the area saved is widely unknown. The Variables are stored between People data on the OW and the Breeding Daycare store. But that area is large, and filled with used data. As they are indirectly referenced in-game, it may take a long while to find all possibilities out.

[diegoisawesome]

Originally Posted by JPAN

The main problem here is that it's impossible to truly know which are safe at this point. I would bet the ones originaly used by the game outside the ASM Routines(0x4050-0x40c0) should be safe, but the area saved is widely unknown. The Variables are stored between People data on the OW and the Breeding Daycare store. But that area is large, and filled with used data. As they are indirectly referenced in-game, it may take a long while to find all possibilities out.

Wow, this is really bad...
I wish I could help, but I can barely do anything in ASM.

Also, up to what point are the 0x5000 variables okay, would you say?

[JPAN]

Originally Posted by diegoisawesome

Wow, this is really bad...
I wish I could help, but I can barely do anything in ASM.

Also, up to what point are the 0x5000 variables okay, would you say?

I think what would be best right now is to choose variables in an area you are not going to use. For instance, if your hack doesn't use the breeding center, you can use 0x4e4a to 0x4ed6. The 0x5000 variable is located in a location that is unknown, so I can't say what it will break, if it breaks anything at all.

On a more positive sidenote, I managed to make the game save a location of my choice to the 1f bank, and it affected nothing adversly, so it should be possible to recreate the variables there, with some work.

[diegoisawesome]

Originally Posted by JPAN

I think what would be best right now is to choose variables in an area you are not going to use. For instance, if your hack doesn't use the breeding center, you can use 0x4e4a to 0x4ed6. The 0x5000 variable is located in a location that is unknown, so I can't say what it will break, if it breaks anything at all.

On a more positive sidenote, I managed to make the game save a location of my choice to the 1f bank, and it affected nothing adversly, so it should be possible to recreate the variables there, with some work.

Amazing! Still, Game Freak needed to do better with the coding of the routine in the first place, even if they weren't planning on using those variables.

BTW, what would you suggest for a hack of Emerald that's used variables up to around, say, 0x5030? Would you recommend switching them all out (somehow), or will we see a fix in the near future?

[JPAN]

Originally Posted by diegoisawesome

Amazing! Still, Game Freak needed to do better with the coding of the routine in the first place, even if they weren't planning on using those variables.

BTW, what would you suggest for a hack of Emerald that's used variables up to around, say, 0x5030? Would you recommend switching them all out (somehow), or will we see a fix in the near future?

Luckily, all Save functions seem identical (with the exceptions of a few pointers), so saving should be easily solved. Repointing variables to solve the problem should also be simple. It's the loading that will take a while to crack, as I've yet to find that function. Also, the new variables will be limited to 0x800 (one block), and finding 0x1000 bytes in the RAM that are free in a joint space should also be problematic.

But, for now, keep using that 0x5030 variables. A fix shouldn't be far out.
By the way, if you can find an empty RAM location with the needed space(should be near the end), it would make things easier.

[diegoisawesome]

Originally Posted by JPAN

Luckily, all Save functions seem identical (with the exceptions of a few pointers), so saving should be easily solved. Repointing variables to solve the problem should also be simple. It's the loading that will take a while to crack, as I've yet to find that function. Also, the new variables will be limited to 0x800 (one block), and finding 0x1000 bytes in the RAM that are free in a joint space should also be problematic.

But, for now, keep using that 0x5030 variables. A fix shouldn't be far out.
By the way, if you can find an empty RAM location with the needed space(should be near the end), it would make things easier.

How would you go about seeing if a RAM location is empty? Is there a special method, or do you just look and try to find a ton of 00s?
If the second one is right, try 0x02FF0000. Found it randomly.

[Chaos Rush]

Originally Posted by JPAN

I've searched the other ROM versions for it, and I have bad news:
In Emerald, from 0x5536 forward affect the Boxes, and at Ruby, although not directly problematic, from 0x55a0 the variables aren't saved, and from 0x415c, you start overwriting names, sayings and Overworld data.
This fact has made me think, how can we get usable, permanent variables? So, I've studied the FlashRom write routines, and found something interesting, and since I was on a multi-ROM streak, I confirmed it for Ruby, Fire Red and Emerald.

So in an Emerald hack, is it okay to use any variable below 0x5536? And would a 100% safe way is to use 0x4050-0x40C0?

[JPAN]

Originally Posted by diegoisawesome

How would you go about seeing if a RAM location is empty? Is there a special method, or do you just look and try to find a ton of 00s?
If the second one is right, try 0x02FF0000. Found it randomly.

RAM only goes from 0x02000000 to 0x0203ffff. 0x0203f000 seems open in Emerald, but the only way to be sure is to search for the value 00 several times, with VBA, and try and see if the values closer to it change by doing different things (for example, fill a box with pokemon, and check if it touches there. Then try other things like teaching an attack, a wild battle, safari zone... If it remains 00 all that time, chances are it's safe).

Originally Posted by Chaos Rush

So in an Emerald hack, is it okay to use any variable below 0x5536? And would a 100% safe way is to use 0x4050-0x40C0?

Like I said before, 0x5536 is the first box position. Lower than that it's the Map Data, that include several random events. The only variables you can use with certainty are the ones they used in-game. All others may either damage your save or be ok. So, use the any you want. Then, If when testing, you see it affected something (like battle frontier records, daycare center, "trendy phrase", Rival name), check if it's the variable. If so, try another.

[JPAN]

Ok, I finished a preliminary version of a replacement for the unsafe Variables. Depending on if you want, or not, to replace the Variable system, there are two features you can get.
First, the new block saved at the end of the Flash ROM, that contains 4096 bytes of usable, saveable space. That block is located at 0x0203e000 (as it seems free on both Fire Red and Emerald), and can be accessed alone by the use of the byte manipulating commands (in the 0x10 range).
Second, the ability to use that new area as Variables. The current solution destroys the access to the old ones, So if you're not OK with it, I'm sure we can work something out (after all, we still have 0x9000-0xffff to work with).
Installing informations are inside the Zip below, as well as the source code.

Before you go and install that directly in your Hack, I only tested it slightly. The save-load function alone is harmless, but the Variable one needs further testing. So, if you could, please test it in a normal ROM and see if it doesn't break anything.

[diegoisawesome]

Originally Posted by JPAN

Ok, I finished a preliminary version of a replacement for the unsafe Variables. Depending on if you want, or not, to replace the Variable system, there are two features you can get.
First, the new block saved at the end of the Flash ROM, that contains 4096 bytes of usable, saveable space. That block is located at 0x0203e000 (as it seems free on both Fire Red and Emerald), and can be accessed alone by the use of the byte manipulating commands (in the 0x10 range).
Second, the ability to use that new area as Variables. The current solution destroys the access to the old ones, So if you're not OK with it, I'm sure we can work something out (after all, we still have 0x9000-0xffff to work with).
Installing informations are inside the Zip below, as well as the source code.

Before you go and install that directly in your Hack, I only tested it slightly. The save-load function alone is harmless, but the Variable one needs further testing. So, if you could, please test it in a normal ROM and see if it doesn't break anything.

The hack (the one that makes the new variables work) broke my ROM. I couldn't start the game (once I pressed Continue, it crashed), I couldn't load my save states (take one step, it freezes) and other stuff.
My ROM is Emerald.

[JPAN]

Originally Posted by diegoisawesome

The hack (the one that makes the new variables work) broke my ROM. I couldn't start the game (once I pressed Continue, it crashed), I couldn't load my save states (take one step, it freezes) and other stuff.
My ROM is Emerald.

Sorry about that. It seems I forgot to include some steps in the Emerald aplication. The problem comes from the Emerald Var_decrypt function being too simple, and not pushing r4-r6. Fixed instructions are posted.

[diegoisawesome]

Originally Posted by JPAN

Sorry about that. It seems I forgot to include some steps in the Emerald aplication. The problem comes from the Emerald Var_decrypt function being too simple, and not pushing r4-r6. Fixed instructions are posted.

There, it seems to be working fine now. :D
I was testing this out awhile, and so far, it seems to be working fine.
Of course, all of my variables (0x5000 and up) were set to 0x0, which told me it was repointed.
It works great! :D

[diegoisawesome]

JPAN! I have a HUGE problem!!
It appears that the RAM address you specified (0x0203E000) is used by the D/N system!!
What would be a good RAM address to use instead? (This is using BPEE)
EDIT: Also, the RAM address I changed it to doesn't seem to get refreshed upon starting a new game, with a game already having been saved!

[JPAN]

Originally Posted by diegoisawesome

JPAN! I have a HUGE problem!!
It appears that the RAM address you specified (0x0203E000) is used by the D/N system!!
What would be a good RAM address to use instead? (This is using BPEE)
EDIT: Also, the RAM address I changed it to doesn't seem to get refreshed upon starting a new game, with a game already having been saved!

Well, 0x0203e000 is the start, but in Emerald, all up to 0x0203ffff seems free. So, just choose 0x1000 area or Ram from where the D/N lets up and use that.
Also, Emerald appears to load Flash-to-RAM only once, at the start of the first screen. Previously unused areas of the RAM shouldn't be deleted by themselves on New Game. So, use a script that only happens on the new game (set by one of the old 0x40xx variables, that the game clears for you) that clears the entire memory area for you.

[diegoisawesome]

Originally Posted by JPAN

Well, 0x0203e000 is the start, but in Emerald, all up to 0x0203ffff seems free. So, just choose 0x1000 area or Ram from where the D/N lets up and use that.
Also, Emerald appears to load Flash-to-RAM only once, at the start of the first screen. Previously unused areas of the RAM shouldn't be deleted by themselves on New Game. So, use a script that only happens on the new game (set by one of the old 0x40xx variables, that the game clears for you) that clears the entire memory area for you.

Okay then. Thanks for the fast response.
But, my question is:
How would I go about writing a script (or an ASM routine, if necessary) that clears the RAM?
I remember seeing something about ldstia or something like that in some ASM routines, and heard it does something with an increasing value... Would that help?

[JPAN]

You can use this routine to clear 0x1000 bytes. Just replace the last pointer for the one you will be using.

.align 2
.thumb
Fill_memory: push {r4-r7, lr}
    mov r4, #0x10
    lsl r4, r4, #0x8 /*value = 0x1000*/
    ldr r5, start_addr
    add r4, r5, r4
    mov r0, #0x0
    mov r1, #0x0
    mov r2, #0x0
    mov r3, #0x0
fill_loop:  stmia r5!, {r0-r3}
    cmp r4, r5
    bgt fill_loop
    pop {r4-r7,pc}
.hword 0x0000    
start_addr:  .word 0x0203f000 /*change as needed*/

or, in byte code, this:

f0 b5 10 24 240205 4d 2c 19 00 20 00 21 00 22
00 23 0f c5 ac 42 fc dc f0 bd 00 00 00 f0 03 02

The last bolded bytes are the pointer you wish to use. replace for your own. The bolded number above is the instruction needed to replace to make different sizes. For instance, if your replace 10 with 20, you will clear 0x2000 bytes. This code will always cover 0x100 bytes minimum (0x1).