[Other] 1st Gen Hacking: Useful Links

[Sawakita]

1ST GEN HACKING: USEFUL LINKS

This is a list of links to documents and tools; it aims to help people solving the "hard" task of finding information about 1st gen hacking, which means Pokemon Red/Blue (and sometimes Yellow) versions. At the end of the post you'll also find some really useful documents about Game Boy hardware specifications.


PREMISE
Bits and Bytes[link]
The link contains a fast explanation of bits and bytes, and binary/hexadecimal format conversions.

Hex Editing [link] [tool-list link]
I know all the misconceptions that exist around hex editing, but you've got to understand that getting used to it is really easy (after all, you learned decimal numbers when you were, like, 5-6 years old). I can't think of linking any page about learning hex editing. Ok, try THIS, if you really can't handle a simply different way of counting numbers. If it helps you, you can think of a hex editor as if it's a book, but instead of reading words, formed by letters, you find organized data (and code, of course), formed by numbers. The hard thing could be figuring out what the formats of the various data structures are, but many of them (and the most important ones), are documented and explained, so, it's not that bad really.

Game Boy Pointers [link]
A very clear explanation of GB pointers, since they can be hard to understand for some people.


GENERAL DATA LOCATION
Red ROM Map [link]
This document helps you orientate in the ROM, listing where many things are located (be it graphics, texts or other data-tables). Certainly a doc to keep an eye on, during your hacking sessions.
Notice that it's for Red version, although Blue offsets are almost identical. Yellow has, instead, completely different addresses (even though it's important to say that general data-structure are the same, as well as the game engine, except obviously for some added features)

Red/Blue RAM Map [link]
Similar to the ROM Map, but this lists several RAM locations used by the game. It's especially useful, when you're trying to figure out a certain part of code, since most of what ASM code does is moving values between ROM and RAM, and between RAM and RAM. So, once you know what is a certain RAM location used for, figuring out the code is an easy task.
Yellow RAM Map is pretty similar to this: the order of data is almost the same, but the location seems to be shifted by 1-2 bytes, compared to the R/B RAM Map.

RBY Text Table [link][alt. link]
If you apply the text table to a hex editor that supports it (I can suggest WindHex or Hexeccute; with some work you can get HexWorkshop to be a useful text editor too), you can localize and edit text, in almost no time! (especially if the hex editor supports relative search, which most likely should)

Giegue's Master Guide... [link]
...to Hacking Pokemon Games Red-Crystal is an old, incomplete document, filled with errors. Nonetheless, it's a somehow acceptable answer to many questions about editing specific things in the game (wild pokemon, mapping, base stats, evolutions,...)


SPECIFIC DATA STRUCTURES
RGBY Map Headers... [link]
...& Stuff That Goes With It. In my opinion this is the most important, helpful and instructive 1st-gen-specific document (for things that can be modified just by hex editing). If you want to get confident with the ROM, read this doc. Then read it again, and again.

Base Stats [link]
A clear explanation of the Base Stats' structure, byte by byte, and its location in ROM. Notice that the Base Stats follow the same order of the Pokedex.

Maps' Indexes And Names [link]
A list of maps' IDs and their corresponent name to help orientate you through the maps.

Pokemon Indexes And Names [link]
One of the things that annoy people about 1st gen Pokemon is that there are two different orders: Pokemon can be ordered by Pokedex, or by their index (or ID) number. The second order can be confusing since it doesn't seem to follow any logic, so having a list of pokemon by their internal ID along with the Dex ID can help.

Wild Pokemon Data [link] [link]
Thanks to IIMarckus for saving this conversation where you can figure out how and where wild pokemon data is stored. Check the second link too, it might help you understanding how to edit this data.

Pokedex Data Structure [link]
In this doc you can find a dump of the pokedex info (name of species, height & weight, dex's text entry pointer).

Attacks Data Structure [link]
A good documentation of each attack's specifics, including a description of what each effect-of-move byte means.

Town Map Editing [link] [tool's link]
Town Map layout consists of a RLE-compressed data which can use only 16 (=0x10) different tiles. Read the linked document to make your own Town Map for your 1st-gen hack. If you don't want to compress the data by hand, IIMarckus wrote a C program that can both compress or decompress RLE files (notice that the file must contain only the town map data, you still need a hex editor to create the uncompressed map and insert it in the game's ROM).


GRAPHICS
Game Boy graphics format [link]
Game Boy's GFX format is 2BPP (two bit per plane), which means that 2 bits are required to display a single pixel in the screen.

Tile Layer Pro [tool link][fix-patch link]
You can use this cool tool (no pun intended) to edit uncompressed graphics: tilesets, OWs, attacks' graphics, titlescreen's title. Fonts and "red version" text are uncompressed too, but are stored in 1BPP format, so make sure you choose the correct format in TLP's menu.
TLP also can import Bitmap images.

Compression Tool [link]
This tool, written by stag019, compresses pictures from 2BPP (Game Boy) format into the format which is compatible with Red, Blue, Green and Yellow versions.

Decompression Algorithm [link] [rbgfx.c][sprites.c][disasm]
This document written by Tauwasser (both in german and in english) explains the "complex" algorithm that compress in a such efficient way all the in-battle sprites (trainers' and pokemon'). There are a couple of C programs that perform the decompression:
- rbgfx.c (IIMarckus said: "a C program that rips the front‐ and back‐pics of all 151 Pokémon and converts them to bitmaps. (The code is written badly and is buggy on 64‐bit compilers, though.)").
- sprites.c (wrote by Ubitux for his great program Pokanalysis; I'm not sure it can be compiled as a stand-alone).
I've also linked my incomplete disassembly of the decompression routine.


MUSIC
RBY Music Doc [link]
This doc was made to explain GSC music structure, but was expanded to explain RBY's as well.

RBY Music Importer [link]
This tool is supposed to convert music previously composed with FamiTracker.


SCRIPTING
R/B Text-Scripting Commands [link]
Several text-commands are listed with a short description. There might be some ASM reference, so you might want to give a look at the Assembly Zone, first (not necessarily, though).

R/B Scripting Guidelines [link]
An overview of the level-scripts, the way are built in a 1st gen game (no scripting commands like newer generations, sorry). This document pre-requires ASM knowledge.


ASSEMBLY: DOCS
The ASM School [link]
This is a list of lessons for newbies that attempt to learn Assembly. It is a very good starting point, for people who don't know anything about GB-z80 ASM.

GameBoy Opcode Summary [link]
Here you can find listed all the opcodes along with a schematic explanation of their meaning/function.

Game Boy Official Programming Manual [link]
This is the official programming manual. It simply explains everything about GB/GBC/SGB hardware. The only thing I couldn't find is the SGB border arrangement, so I'm putting it here:

SGB-Border Tile Arrangement:
1st byte = tile ID
2nd byte:
| bit |      meaning       |
 [7]   Horizontal flipping [|]
 [6]   Vertical flipping [-]
 [5]   Unused
 [4]   Unused
 [3]   Unused
 [2][1][0]: %000 = Pal 0, %010 = Pal 1, %100 = Pal 2

Gameboy Unofficial Programming Manual [link]
This doc doesn't cover everything, unlike the previous link; anyway it might help you since it's less challenging if you're trying to get your first grasp of the GB hardware.

Pandocs [link]
Another link to GB hardware documentation. Since it has an index at its beginning, it might help you trying orientate in this great amount of info.

Gameboy Cribsheet [link]
This well-packed .pdf file is a great fast reference when you have to look for the meaning of a register or anything hardware-related, in few seconds. IIMarckus pointed out that it has some wrong values for opcodes, so for the correct hex correspondencies you should read THIS.

THE Disassembly [link]
This is a disassembly of Pokemon Red, work in progress by IIMarckus. It holds tons of information about almost all the things that have been listed in this post, so far. So if you don't understand or don't find something you're interested in in the previous links, you can give a look here, even if you still don't know assembly. This disassembly really contains some pieces of information that you won't find anywhere else.


ASSEMBLY: TOOLS
BGB [link]
This is an excellent Game Boy Emulator/Debugger that I suggest to anyone's working on a GB/C hack, instead of using other inaccurate emulators like VBA (that could lead to bugs in your hacks: example). Some useful debugging features are:
- code/RAM/stack/registers viewer, that helps you keeping an eye on everything is happening on run-time;
- a (dis)assembler, that allows you to inject code directly;
- a VRAM/OAM/Palette viewer;
- an ASM dumper;
But there is much more: you can put breakpoints on code-access or on data-access, and you can also put conditional breakpoints. That's not all, so the best thing is starting using this debugger.
Other accurate emulators are Gambatte and KiGB, which are multi-platform (BGB runs only on Windows, if I'm not wrong), but they seem to lack a debugger. KiGB's home site declares it's the most accurate emulator around.

Assembly Editor [link]
This tool made by Jigglypuff works almost like a hex editor but allows you to read/write the code both in bytecode (hex) and in mnemonics' form. Doesn't support labeling.

RGBDS [Linux: tool][Linux: guide] [Win: tool][Win: guide]
My favorite assembler/linker. This is actually best to be used for large-project, rather than for small ASM hacks, probably. Anyway it's a very powerful toolkit: great macro language, supports labeling (of course!), and also includes a program to fix the ROM header. It might take a while to understand how everything works, but once you get used to it, it's the best tool you can use to build your Game Boy hack (especially if you have an already started disassembly, which link you can find in the previous section).


CONCLUSION
I hope I'm not forgetting anything. Oh well, I'll update it in the future, in case there's something I left out. Have a nice hacking time!

[miksy91]

Great job and

Spoiler:

thanks for the link to ASM School

[Sawakita]

Thanks and

Spoiler:

you're welcome.

I've added an "ASM tools" section (debugger, assembler/linker). Also fixed some layout errors.

[Mephodicus]

This is very useful. I've never really got into R/B hacking and I've only dabbled with it so it's useful for those who really need it.

[IIMarckus]

Originally Posted by Sawakita

Decompression Algorithm [link]
This document written by Tauwasser (both in german and in english) explains the "complex" algorithm that compress in a such efficient way all the in-battle sprites (trainers' and pokemon'). No compressing tool exists, so far, to insert new sprites, so you'd have to compress sprites by hand (I've never seen it done before) or ASM-hack the game engine (already done before).

If you want to see the decompression algorithm in action, rbgfx.c is a C program that rips the front‐ and back‐pics of all 151 Pokémon and converts them to bitmaps. (The code is written badly and is buggy on 64‐bit compilers, though.)

Originally Posted by Sawakita

Gameboy Cribsheet [link]
This well-packed .pdf file is a great fast reference when you have to look for the meaning of a register or anything hardware-related, in few seconds.

As I recall, it has some wrong values for opcodes, so keep that in mind.

[psyxe]

Originally Posted by IIMarckus

If you want to see the decompression algorithm in action, rbgfx.c is a C program that rips the front‐ and back‐pics of all 151 Pokémon and converts them to bitmaps. (The code is written badly and is buggy on 64‐bit compilers, though.)As I recall, it has some wrong values for opcodes, so keep that in mind.

im quite confused. i read through it as well as i could, but... xD i got nothing. would the rbgfx.c be something that could make changing the sprites easier? or is there another way to do it?

ive been researching how to hack red for a while now so this was EXTREMELY helpful info.

[Sawakita]

Originally Posted by IIMarckus

If you want to see the decompression algorithm in action, rbgfx.c is a C program that rips the front‐ and back‐pics of all 151 Pokémon and converts them to bitmaps. (The code is written badly and is buggy on 64‐bit compilers, though.)As I recall, it has some wrong values for opcodes, so keep that in mind.

Thank you, I'll fix the first post.

Originally Posted by psyxe

im quite confused. i read through it as well as i could, but... xD i got nothing. would the rbgfx.c be something that could make changing the sprites easier? or is there another way to do it?

:) ive been researching how to hack red for a while now so this was EXTREMELY helpful info.

Considering that it's a program that decompress sprites from ROM it's obvious that it doesn't help inserting edited sprites into the ROM. It might help figuring out how the algorithm works. As it has already been said you have some options:
- compress your sprites by hand;
- build a compressing tool;
- ASM hack the game engine so that it allows the support for uncompressed sprites.

Anyway I've added the source code of the C program written by Ubitux, if it helps.
And also added my partial disassembly of the decompression routine.

Check first post for everything.

[psyxe]

well, i meant changing sprites easier as in being able to use it, edit them, and throw em back in somehow... i read through, its just being new to hacking pokemon, its hard to grasp some of this.

im going to look through and try to read through every step required to edit graphics of any kind... if any of you have time, i would greatly appreciate a slightly "dumbed down" explanation for sprite editing on it if not ill suffer through.. im not giving up

[Sawakita]

I know, understanding the compression algorithm isn't easy, but I don't think it can be explained in a clearer way than how it is in the document written by Tauwasser.

Too bad, currently, there are just two options: figuring out the algorithm, or learning ASM.

[psyxe]

okay... *sigh* i knew it wouldnt be easy anyways.. well, if theres anything you could do to help me i would really appreciate it... otherwise it looks like im in for a lot of studying...


------edit----------
ive looked through stuff... i know the basics of how to change stats ect... using the hex....

so i assume the pokemon's sprites are stored in there too... is there a way to edit the sprites through hex?

[IIMarckus]

Originally Posted by psyxe

ive looked through stuff... i know the basics of how to change stats ect... using the hex....

so i assume the pokemon's sprites are stored in there too... is there a way to edit the sprites through hex?

Editing the pics in hex still requires them to be compressed.

[psyxe]

well, ive learnt quite a bit through this... but im still no closer to changing the sprites...

i read through everything i saw that had to do with the asm.....

Decompression Algorithm [link]
This document written by Tauwasser (both in german and in english) explains the "complex" algorithm that compress in a such efficient way all the in-battle sprites (trainers' and pokemon'). No compressing tool exists, so far, to insert new sprites, so you'd have to compress sprites by hand (I've never seen it done before) or ASM-hack the game engine (already done before).

- ASM hack the game engine so that it allows the support for uncompressed sprites.

how exactly is this done? im sorry if these seem really noobish but im sincerely trying...

[Sawakita]

I've added a link about bits and bytes. Thanks to kkj1116 for suggesting it.

Originally Posted by psyxe

well, ive learnt quite a bit through this... but im still no closer to changing the sprites...
i read through everything i saw that had to do with the asm.....
how exactly is this done? im sorry if these seem really noobish but im sincerely trying... :)

You do it through an assembler or a hex editor, if you mean the program. If you meant how you do it technically, you can start by disassembling the routine using a debugger (the decompression routine is at $251A in a Red/Blue ROM, while in Yellow should be located at $2410).

[giradialkia]

This seems extremely useful, so stickying :)

[jvpski3]

Dude, you are the best! Thank you Thank you Thank you! THANK YOU! You are the king of Gen I-II hacking!! Woo!

[stag019]

Originally Posted by Sawakita

Decompression Algorithm
No compressing tool exists, so far, to insert new sprites, so you'd have to compress sprites by hand (I've never seen it done before)

This has been done now. I'd show you the link, but "You are only allowed to post URLs to other sites after you have made 15 posts or more." So therefore, you lose out. Unless of course, you visit Skeetendo, in the Devamp Pokemon thread.

I will post the tool when I work out all the bugs/when I'm allowed to post links (see also, March of 2028 at the current rate I'm going).

[hi sir tomato my password is syvniti]

the Bits and Bytes link doesn't work, please fix it.

[Sawakita]

Originally Posted by stag019

This has been done now.

So, did you actually compressed it by hand, one bit at a time? Kudos to you for the patience.

Originally Posted by stag019

So therefore, you lose out.

I already can put sprites in RBY. I also never lose.

Originally Posted by stag019

I will post the tool when I work out all the bugs/when I'm allowed to post links (see also, March of 2028 at the current rate I'm going).

You might not know that they've invented something called post attachment. Just for your information.

Originally Posted by hi sir tomato my password is syvniti

the Bits and Bytes link doesn't work, please fix it.

Fixed, thanks for pointing it out.

[stag019]

Originally Posted by Sawakita

You might not know that they've invented something called post attachment. Just for your information.

Really? I had no idea such technology existed!

However, the biggest problem with that is that because I'm much better at scripting languages (PHP) than programming languages (C, C++, although I'm getting better at it), the current form of the tool is an online tool: a PHP script. And while I could release the source and allow anyone to upload it themselves (or run on their localhost) it would be much more advantageous to simply link to the one I've already uploaded (when I'm finished with it).

At some point I do plan to port it to C or C++ though.

[IIMarckus]

Originally Posted by stag019

Really? I had no idea such technology existed!

However, the biggest problem with that is that because I'm much better at scripting languages (PHP) than programming languages (C, C++, although I'm getting better at it), the current form of the tool is an online tool: a PHP script. And while I could release the source and allow anyone to upload it themselves (or run on their localhost) it would be much more advantageous to simply link to the one I've already uploaded (when I'm finished with it).

At some point I do plan to port it to C or C++ though.

If you release the source code, maybe someone else will port it to C for you

[stag019]

While I do like your thought process there, keep this is mind:
"If Tauwasser documents the decompression routine, maybe someone else will create a tool to recompress graphics for him "

He posted the documentation on the wiki in German on March 25th, 2006. He did a rough English translation on June 12th, 2010. It's September 7th, 2011, and only now are we getting close to a compression tool.

Because I already know what I'm doing (for the most part), I feel obligated, I feel it's my duty to finish this out.

...hopefully before Christmas.

[Sawakita]

I've added the link to my sprite compression tool (see first post).

[stag019]

Okay good then. Then I don't feel obligated to show my tool, since this works almost the exact same way mine does.

[IIMarckus]

Originally Posted by stag019

While I do like your thought process there, keep this is mind:
"If Tauwasser documents the decompression routine, maybe someone else will create a tool to recompress graphics for him "

And sure enough, it is happening. Barely a year after he translated it, too. Can you imagine how long it would have taken if he hadn’t released his doc?

My hacking time has been epsilon for many months now, but since I released much of my work while I was working on it, it has been able to help others (and get improved by others), and will continue to do so long after I stop hacking completely. I heartily encourage this practice wherever possible.

Originally Posted by stag019

Because I already know what I'm doing (for the most part), I feel obligated, I feel it's my duty to finish this out.

...hopefully before Christmas.

This is exactly what I’m talking about!

[Sawakita]

Originally Posted by stag019

Okay good then. Then I don't feel obligated to show my tool, since this works almost the exact same way mine does.

Why not? If yours is multiplatform and performs interpretations 2 and 3 too, it's certainly more efficient than the one I posted. Feel free to post it so people can have a better choice, if you want.