Tcoppy
Favorite Stand
- 548
- Posts
- 14
- Years
- Age 27
- New England
- Seen Feb 6, 2023
Let's talk about
Hex
Introduction:
Hello I'm Tcoppy, the co-leader/owner/maker/developer for Pokemon Discovery and as of late, I've been working on a way to expand the move table in Emerald. While I was doing that, great people along the way taught me a lot of Hex and how Hex works. For a while now, my main Hex editor is HxD so any pictures that you will see will be from HxD but there are other great Hex editors too like Hex Workshop.
What is Hex:
What do I need to get started:
Well right off the bat you need:
-An emulator (VBA's are good)
-A rom (I'm gonna use Fire Red for this tut cause it has more coverage for now)
-A Hex Editor (Like I said, HxD is great, many hackers use this)
-A calculator you can use Hex with (Windows has a great built in calculator)
-Tools (Hacking tools, there everywhere)
-Patience (Trust me stuff can and will go wrong eventually, it just takes time to learn)
Lesson 1: Opening the rom and an introduction on offsets and bytes:
I assume you installed your Hex editor and opened your rom in it, now this is what it looks like:
Yep, that's the game, a bunch of numbers and letters, that's pretty much the whole game in itself. Beautiful ain't it? Let's break this down to a more simple way of understanding this.
So in the green box you see two variables, these variables make up a byte, these are very important and you will need to play with them eventually.
On the right of the blue vertical line is the first part of an offset and you match it with one of the parts above the yellow horizontal line to find a particular offset, an offset will have the byte.
For example, I need to find offset 00(or 0x)000012 (offset 12), I would look where it says 00000010 and 02 and highlight the bytes that match up to that and all you'll have to do is look at the bottom left and it will say what offset it is. In the picture with the color I highlighted the offset there in red, it says offset 0, with the bytes corresponding as 7F and is on 00000000.
There is a very easier way to find offsets though, the other way was manually, but this is way easier. Hit ctrl-g or under the search tab click Goto and you'll see something like this.
In the black square is the Goto box I typed in 00800000 so it will search for offset 0x800000, the place where all the free space is.
Here I am!
We'll get back to these offsets and bytes later, but for now, let's talk about what happens when you screw up.
Lesson 2: Backups! Backups! Oh, and the horrors of ctrl-v (paste):
I'm going to be blunt, if you screw up extremely bad, which you will do eventually, and you have no backups. You are completely screwed. I'm going to show you guys a very common mistake when copying bytes and putting them in free space, and that's pasting, or ctrl-v. To show you this let me back up my rom.
Now that I did that, I can show you how bad you can screw up everything with that simple command.
I'll be inserting Hackmew's Firered ASM routine into Firered and I take no credit for this ASM routine, it is completely Hackmews. You can find the routine and learn about the basics of ASM on the tutorial thread Hackmew's Knowledge.
Anyway, I just copied the ASM and I'm going to paste it onto 0x800000 OR 00800000
Looks fine doesn't it? Nothing could've went wrong right? Let's open of Advance Map created by LU-HO.
Nope, I didn't use paint to create those blocks, this happened because I pasted instead of using ctrl-b, which is the right way to insert it by the way.
When you hit ctrl-v or paste, you accidentally re-arranged a ton of bytes that all in all, broke the rom, and since I saved in HxD, this rom is useless, which is why I create backups so I can start the rom back to when I didn't do ctrl-v. This is why you never use ctrl-v and always make backups.
Lesson 3: QUICK Lesson on how to use a calculator with Hex.
I know, I lied, I'm sorry. I just wanted to get this out of the way as fast as possible. I'll tell you from experience, you need to know how to use a calculator with Hex compatibility, and for people who have Windows, I'm going to show you a great calculator to use, other computers may have this too or have a slightly different version, but since I have a Windows, I'm going to use my calculator. I'm sorry.
First you open up the calculator, in the start/menu button on the lower left will lead you to a directory of sorts, shown here.
I know, you see the paint label on the top, deal with it. :P
In the black box that's where you type to search for the calculator and above the yellow is what you click.
When you open the calculator you should be in standard mode (this):
We were talking about the Battle Frontier by the way if you were wondering. lol
Anyway, we need to make this set up for a programmer so click on the view tab and click programmer and then and this should appear.
This isn't right yet, we need to switch this to Hex, so click the hex button and it'll show up in Hex. I like keeping it to qword too because I'm use to it but work with whatever you want.
Now let's try a simple problem, kinda regarding math.
0x800000 + C
....
Huh?
Let's try this on the calculator
0x800000 just turns into 800,000 or 80 0000
Then you just add C and you get
80 000C or 0x80000C
It's not difficult to use and it's quite useful to find exact offsets. Now I can get on to the good stuff. Strings and Repointing!
Lesson 4: Strings
Now we're getting into more complicate things. First, to find a string you first press Find under the search tab or ctrl-f.
After you press this it should lead you to the next picture, which I already set up to find a string, so you find the string you're looking for just like this but instead of my random string, it'd be yours.
Once you find the string it will be highlighted like the picture below.
Now let's break that down.
In the light green, it means that that's the string. In the dark green however, that separates the string into bytes. Remember when I said in Lesson 1 that a byte has two variables, a string has four bytes, that means eight variables in total make up a string.
Here's another quote from Itari that also explains strings, actually, even more clearly.
That's essentially what a string is, just a line of bytes that can have 16-bit integers to 64-bit integers. Simple, right? Now let's talk about pointers and repointing.
Lesson 5: Pointers and Repointing
Now that I have time to do this right, I can actually explain pointers and repointing.
Remember when I explained in Lesson 1 what the offsets are. Pointers are essentially the offsets written backwards. They are bytes in the rom that point to specific offsets. Let's say your expanding moves for Fire Red, you have to copy a lot of bytes starting from a particular offset into a bunch of free space. This offset, it's something different, but let's call it 0x9902C3. Pointers is this backwards In roms, you will find a lot of pointers that direct to offsets. 0x9902C3 backwards would be C3029908, with x switching to 8. The pointer would be found in this order in the rom but in byte format, meaning, C3 02 99 08. Those bytes would point to a specific offset, this one being 0x9902C3 or 009902C3, whichever you prefer.
Now for repointing. When you take the bytes of a particular offset let's say again, 0x9902C3 and move the bytes to an area with free space, the bytes there containing FF's. The pointers from before, C3 02 99 08, need to be changed so that the offset can be repointed. Let's say I put 0x9902C3's data into 0x80092A. The pointers to 0x80092A would be 2A 09 80 08, thanks to the rule above stating that offsets that have pointers, the pointers are backwards. All you do is replace C3 02 99 08 with 2A 09 80 08, to make it so that the rom knows that data is at the new offset.
Here is another quote from Itari that sums up pointers and repointing too:
Does that make sense? Do you need me to clarify it more? It can be a little bit difficult to grasp but you'll get the hang of it eventually.
Lesson 6?
Credits:
For now this is who helped me learn Hex and provided insight on Hex editing.
Le Pug: He pushes me to strive to do whatever I need to do for hacking, whether it'd be learning ASM or telling me to expand the Emerald Move Table, which a tutorial by me will happen when I get the time to fully expand the table. Oh and he let me join Team Discovery and I ended up working along side him as the co-leader/owner/maker/developer, so that's cool.
Itman: I've been following his tutorial for the expansion of the move-table in Fire Red and I just use different offsets for Emerald, this tutorial greatly impacted me to write this. And his explanation of repointing with GoGo's helped me learn repointing.
DarkSneasel: He found the attack data for Emerald off of PGE
Gamer2020: For PGE. :P
Gogojjtech: GoGo mostly taught me all I needed to know about Hex editing, and I can't thank him enough for helping me and answering my stupid questions. :P
Hackmew: Even though I never spoke to him, that mistake I talked about on Lesson 2 actually did happen when I was first getting started and then GoGo told me what I did wrong. lol
Either way, I screwed up my rom inserting an ASM routine with ctrl-v, and since it was Hackmew's Knowledge tutorial I was following, I have to thank him for my own screw ups. lol
LU-HO: For Advance map 1.92
Ipatix: He told me that Maël Hörz created HxD, who I wanted the makers to have credit for the tool.
Itari: She (I think) has allowed me to use her quotes to explain Hex and a lot of other insight on stuff and deserves the credit too.
Maël Hörz and anyone else who was involved with HxD: They made HxD and if this wasn't developed or was taught to use this, I don't think I'd be here at this point in time right now.
Spherical Ice: Okay, so I did something stupid while explaining pointers and repointing, people have those days they just completely say something completely stupid, I had one of those days, Spherical Ice saw my mistake, and I fixed it.
Everyone on #GoGo chat and #Discovery chat: They make me laugh and make me even more determined to hack. You know exactly who you are.
If I missed anybody please let me know. :)
Collaboration Tutorial?:
I really want this tutorial to grow and help people, so I decided that whoever wants to be involved, they can post "Lessons" in this thread and I can put them on the main post with proper credit and so people that know about Hex can also teach others about it and make it easier on them. So just tell me or write your lesson or nitpicks in this tutorial and I'll put it on the main post.
Coming up soon:
Lesson 6: ???
Hex
Introduction:
Hello I'm Tcoppy, the co-leader/owner/maker/developer for Pokemon Discovery and as of late, I've been working on a way to expand the move table in Emerald. While I was doing that, great people along the way taught me a lot of Hex and how Hex works. For a while now, my main Hex editor is HxD so any pictures that you will see will be from HxD but there are other great Hex editors too like Hex Workshop.
What is Hex:
Hex, short for hexadecimal, is not a programming language. Hexadecimal is base 16 number system, like decimal is a base 10 number system or binary is a base 2 number system. What this means is that each number is made of a combination of digits using one of sixteen (hence base 16) digits: 0-9 and A-F. It is used in computers when dealing with file manipulation because it converts quite well to binary and back, because each additional hex digit allows for an even number of binary digits (bits) to make it up (4 bits per hex digit). Therefore, a hex editor is merely a tool that takes the bits that make up a file, or in this case the ROM, and show them as a collection of bytes, which are two hex digits (meaning 8 binary bits).
Basically, hex is just a way for programmers to visualize the raw data that makes up the ROM, and free space would just be blank data that is not used by the ROM, represented by the maximum value for a byte, 0xFF (255 in decimal).
What do I need to get started:
Well right off the bat you need:
-An emulator (VBA's are good)
-A rom (I'm gonna use Fire Red for this tut cause it has more coverage for now)
-A Hex Editor (Like I said, HxD is great, many hackers use this)
-A calculator you can use Hex with (Windows has a great built in calculator)
-Tools (Hacking tools, there everywhere)
-Patience (Trust me stuff can and will go wrong eventually, it just takes time to learn)
Lesson 1: Opening the rom and an introduction on offsets and bytes:
I assume you installed your Hex editor and opened your rom in it, now this is what it looks like:
Yep, that's the game, a bunch of numbers and letters, that's pretty much the whole game in itself. Beautiful ain't it? Let's break this down to a more simple way of understanding this.
So in the green box you see two variables, these variables make up a byte, these are very important and you will need to play with them eventually.
On the right of the blue vertical line is the first part of an offset and you match it with one of the parts above the yellow horizontal line to find a particular offset, an offset will have the byte.
For example, I need to find offset 00(or 0x)000012 (offset 12), I would look where it says 00000010 and 02 and highlight the bytes that match up to that and all you'll have to do is look at the bottom left and it will say what offset it is. In the picture with the color I highlighted the offset there in red, it says offset 0, with the bytes corresponding as 7F and is on 00000000.
There is a very easier way to find offsets though, the other way was manually, but this is way easier. Hit ctrl-g or under the search tab click Goto and you'll see something like this.
In the black square is the Goto box I typed in 00800000 so it will search for offset 0x800000, the place where all the free space is.
Here I am!
We'll get back to these offsets and bytes later, but for now, let's talk about what happens when you screw up.
Lesson 2: Backups! Backups! Oh, and the horrors of ctrl-v (paste):
I'm going to be blunt, if you screw up extremely bad, which you will do eventually, and you have no backups. You are completely screwed. I'm going to show you guys a very common mistake when copying bytes and putting them in free space, and that's pasting, or ctrl-v. To show you this let me back up my rom.
Now that I did that, I can show you how bad you can screw up everything with that simple command.
I'll be inserting Hackmew's Firered ASM routine into Firered and I take no credit for this ASM routine, it is completely Hackmews. You can find the routine and learn about the basics of ASM on the tutorial thread Hackmew's Knowledge.
Anyway, I just copied the ASM and I'm going to paste it onto 0x800000 OR 00800000
Looks fine doesn't it? Nothing could've went wrong right? Let's open of Advance Map created by LU-HO.
Nope, I didn't use paint to create those blocks, this happened because I pasted instead of using ctrl-b, which is the right way to insert it by the way.
When you hit ctrl-v or paste, you accidentally re-arranged a ton of bytes that all in all, broke the rom, and since I saved in HxD, this rom is useless, which is why I create backups so I can start the rom back to when I didn't do ctrl-v. This is why you never use ctrl-v and always make backups.
Lesson 3: QUICK Lesson on how to use a calculator with Hex.
I know, I lied, I'm sorry. I just wanted to get this out of the way as fast as possible. I'll tell you from experience, you need to know how to use a calculator with Hex compatibility, and for people who have Windows, I'm going to show you a great calculator to use, other computers may have this too or have a slightly different version, but since I have a Windows, I'm going to use my calculator. I'm sorry.
First you open up the calculator, in the start/menu button on the lower left will lead you to a directory of sorts, shown here.
I know, you see the paint label on the top, deal with it. :P
In the black box that's where you type to search for the calculator and above the yellow is what you click.
When you open the calculator you should be in standard mode (this):
We were talking about the Battle Frontier by the way if you were wondering. lol
Anyway, we need to make this set up for a programmer so click on the view tab and click programmer and then and this should appear.
This isn't right yet, we need to switch this to Hex, so click the hex button and it'll show up in Hex. I like keeping it to qword too because I'm use to it but work with whatever you want.
Now let's try a simple problem, kinda regarding math.
0x800000 + C
....
Huh?
Let's try this on the calculator
0x800000 just turns into 800,000 or 80 0000
Then you just add C and you get
80 000C or 0x80000C
It's not difficult to use and it's quite useful to find exact offsets. Now I can get on to the good stuff. Strings and Repointing!
Lesson 4: Strings
Now we're getting into more complicate things. First, to find a string you first press Find under the search tab or ctrl-f.
After you press this it should lead you to the next picture, which I already set up to find a string, so you find the string you're looking for just like this but instead of my random string, it'd be yours.
Once you find the string it will be highlighted like the picture below.
Now let's break that down.
In the light green, it means that that's the string. In the dark green however, that separates the string into bytes. Remember when I said in Lesson 1 that a byte has two variables, a string has four bytes, that means eight variables in total make up a string.
Here's another quote from Itari that also explains strings, actually, even more clearly.
A string is a collection of characters, of variable length, which are the letter values mapped by the computer to specific byte values.
What was described is how to find a 32-bit integer, which is a number made consisting of (obviously) 32 bits, or 4 bytes. 8 bytes make up a 64 bit integer, and 2 bytes make up a 16 bit integer.
Furthermore, these numbers have names we refer to them as. A 16-bit integer is a "word", making a 32-bit integer a "double word" (dword), and a 64-bit integer a "quadruple word" (qword). In programming, a word is often called a "short" and a dword is just the generic integer, and a qword is called a "long".
On the GBA (and most systems), a word is stored "backwards" because it is following a convention for word storage known as "little-endian," meaning what is considered the "least significant" (in terms of rounding) digit is stored first, followed by the others.
So, 0x12345678 would be 78 56 34 12 in little-endian format.
That's essentially what a string is, just a line of bytes that can have 16-bit integers to 64-bit integers. Simple, right? Now let's talk about pointers and repointing.
Lesson 5: Pointers and Repointing
Now that I have time to do this right, I can actually explain pointers and repointing.
Remember when I explained in Lesson 1 what the offsets are. Pointers are essentially the offsets written backwards. They are bytes in the rom that point to specific offsets. Let's say your expanding moves for Fire Red, you have to copy a lot of bytes starting from a particular offset into a bunch of free space. This offset, it's something different, but let's call it 0x9902C3. Pointers is this backwards In roms, you will find a lot of pointers that direct to offsets. 0x9902C3 backwards would be C3029908, with x switching to 8. The pointer would be found in this order in the rom but in byte format, meaning, C3 02 99 08. Those bytes would point to a specific offset, this one being 0x9902C3 or 009902C3, whichever you prefer.
Now for repointing. When you take the bytes of a particular offset let's say again, 0x9902C3 and move the bytes to an area with free space, the bytes there containing FF's. The pointers from before, C3 02 99 08, need to be changed so that the offset can be repointed. Let's say I put 0x9902C3's data into 0x80092A. The pointers to 0x80092A would be 2A 09 80 08, thanks to the rule above stating that offsets that have pointers, the pointers are backwards. All you do is replace C3 02 99 08 with 2A 09 80 08, to make it so that the rom knows that data is at the new offset.
Here is another quote from Itari that sums up pointers and repointing too:
As for pointers, there is a little more to it. As you know, a pointer is a dword pointing to a specific address, which can be anywhere that is within the GBA's memory. The "prefix" as I will call it actually tells the GBA where in its various memory banks the pointer refers, be it the ROM, RAM, etc.
The various prefixes are:
Source: GBATEK
- 00 - BIOS (if offset < 0x4000)
- 02 - WRAM (Work RAM) (if offset < 0x40000)
- 03 - IRAM (2nd Work RAM) (if offset < 0x8000)
- 04 - I/O Registers (if offset < 0x3FF)
- 05 - Palette RAM (if offset < 0x400)
- 06 - VRAM (Video RAM) (if offset < 0x18000)
- 07 - OAM (Object attributes) (if offset < 0x400)
- 08-0D - ROM (what you modify) (if offset is < 0x6000000)
So, if you wanted to make a pointer to offset 0x1763214 in an expanded ROM, you add 0x08000000 to it and you get 0x09763214. Or, if you wanted a pointer to 0x3A54 in the WRAM, you add 0x02000000 to it and get 0x02003A54.
Repointing is simply changing a pointer to a different offset, so that whatever data is expected by the GBA there is read from said different offset. You do this for when you need to replace something that is larger than the original without overwriting surrounding data.
Does that make sense? Do you need me to clarify it more? It can be a little bit difficult to grasp but you'll get the hang of it eventually.
Lesson 6?
Credits:
For now this is who helped me learn Hex and provided insight on Hex editing.
Le Pug: He pushes me to strive to do whatever I need to do for hacking, whether it'd be learning ASM or telling me to expand the Emerald Move Table, which a tutorial by me will happen when I get the time to fully expand the table. Oh and he let me join Team Discovery and I ended up working along side him as the co-leader/owner/maker/developer, so that's cool.
Itman: I've been following his tutorial for the expansion of the move-table in Fire Red and I just use different offsets for Emerald, this tutorial greatly impacted me to write this. And his explanation of repointing with GoGo's helped me learn repointing.
DarkSneasel: He found the attack data for Emerald off of PGE
Gamer2020: For PGE. :P
Gogojjtech: GoGo mostly taught me all I needed to know about Hex editing, and I can't thank him enough for helping me and answering my stupid questions. :P
Hackmew: Even though I never spoke to him, that mistake I talked about on Lesson 2 actually did happen when I was first getting started and then GoGo told me what I did wrong. lol
Either way, I screwed up my rom inserting an ASM routine with ctrl-v, and since it was Hackmew's Knowledge tutorial I was following, I have to thank him for my own screw ups. lol
LU-HO: For Advance map 1.92
Ipatix: He told me that Maël Hörz created HxD, who I wanted the makers to have credit for the tool.
Itari: She (I think) has allowed me to use her quotes to explain Hex and a lot of other insight on stuff and deserves the credit too.
Maël Hörz and anyone else who was involved with HxD: They made HxD and if this wasn't developed or was taught to use this, I don't think I'd be here at this point in time right now.
Spherical Ice: Okay, so I did something stupid while explaining pointers and repointing, people have those days they just completely say something completely stupid, I had one of those days, Spherical Ice saw my mistake, and I fixed it.
Everyone on #GoGo chat and #Discovery chat: They make me laugh and make me even more determined to hack. You know exactly who you are.
If I missed anybody please let me know. :)
Collaboration Tutorial?:
I really want this tutorial to grow and help people, so I decided that whoever wants to be involved, they can post "Lessons" in this thread and I can put them on the main post with proper credit and so people that know about Hex can also teach others about it and make it easier on them. So just tell me or write your lesson or nitpicks in this tutorial and I'll put it on the main post.
Coming up soon:
Lesson 6: ???
Last edited: