Research: Calling All ASM Experts!

Queen Azura · Jul 23, 2014

I am currently working on a hack of Emerald. After showing some of my progress to my father (a professional software developer),
he had an interest in developing a tool to simplify and make using ASM simpler for average hackers (like me *cough*).

I made this thread to ask, not being well versed in ASM (just sent him every tutorial and resource I could find),
what features hackers out there would like in such a tool. Also, any information on the topic is useful. He has specifically requested:

A ROM map of Emerald (I am aware of only one incomplete one)
The source code of any programs dealing with ASM already out there (if you're willing to share)

Thanks to Touched for letting me know about the GBATEK manual!

Thank you to FlyingWaffle for these links!

devkitPro
ARM Info Center
GBATEK Manual
CowBiteSpec
Tonc - GBA Programming Manual
ARM7TDMI Reference Manual

However, in his words,

"I want every piece of information on Emerald possible. If someone else knows it, I want it. I'll care if it's useful later, but I want to read it first."

I'd like to hopefully make a large repository of all the information we can assimilate here. Links to other threads/posts are perfectly fine if you don't want to post redundant information.
(I'll save copies of other threads and posts on my own.) I would just like to gather as much data on a wide range as possible.

To make this easier and less messy, using a form like this would help:

Topic: ....
Information: ....
Link(s): ....
Anything else?: ....

Thank you in advance to everyone who contributes!

(PS. As of now, this is only for Emerald. Sorry to FireRed hackers.)

Touched · Jul 24, 2014

Automated routine writing? There's a reason we have programming languages, you know. They're abstractions on a system that can't be satisfactorily automated. Creating something that can create anything you want is basically impossible.

A list of commands available in GBA ASM? The GBA uses an ARM7TDMI processor, which has two instruction sets: ARM and THUMB (It can also emulate the Z80 instruction set for GameBoy Color compatibility). Really, a simple Google search for "THUMB opcodes" will give you numerous resources on this. Additionally, there are numerous assemblers out there with their own manuals, namely GCC. However, if it's a GBA hardware manual you want, the extensive GBATek manual helps a lot (I can't post links as of yet, but you should have no problem searching for it)
Most of these resources can be uncovered with a quick forum search.

As far as I'm aware, most of the people who frequently work with ASM here use IDA pro for all their assembly needs. As this is proprietary, I highly doubt you'll have much luck with that second item.

VBA is dead. It is no longer maintained. The VBA-M team has taken over development of this, but the codebase is horribly cluttered and virtually impossible to refactor successfully (it's riddled with globals and other things that make running multiple instances in one process a nightmare). However, there are all sorts of useful bits in there. I'm currently working on a C++ library that will allow a custom emulator to be built with relative ease. The source code for this library is not available as of yet, but I will be releasing it when I actually have made good progress. Otherwise, you can find VBA-M's source on their SVN (again, Google is your friend), or in a multitude of GitHub repositories.

I'm reserving judgement on this though. Your father seems to be asking an awful lot that could have been resolved with a few seconds of searching (a lot less time than writing that list). Wikipedia would have told you to look for VBA-M, and the first thing a seasoned software developer would have done would be to perform an SVN checkout - it is under the GPL, so obviously someone must have the source.

miksy91 · Jul 24, 2014

Simple and user-friendly asm hacking program would work kinda like XSE or other script editors do;

It could disassemble any routine you want by just telling it the address where the routine is, and you could use it write your own code at any rom address you like (and possibly also make it possible for the program to look for space for the routine and insert it there).

If you wanted to "enhance" it by one way or another, I would suggest adding support for other systems (like GB/C CPU (~z80)) as well.

Queen Azura · Jul 24, 2014

Touched said:
Automated routine writing? There's a reason we have programming languages, you know. They're abstractions on a system that can't be satisfactorily automated. Creating something that can create anything you want is basically impossible.

A list of commands available in GBA ASM? The GBA uses an ARM7TDMI processor, which has two instruction sets: ARM and THUMB (It can also emulate the Z80 instruction set for GameBoy Color compatibility). Really, a simple Google search for "THUMB opcodes" will give you numerous resources on this. Additionally, there are numerous assemblers out there with their own manuals, namely GCC. However, if it's a GBA hardware manual you want, the extensive GBATek manual helps a lot (I can't post links as of yet, but you should have no problem searching for it)
Most of these resources can be uncovered with a quick forum search.

As far as I'm aware, most of the people who frequently work with ASM here use IDA pro for all their assembly needs. As this is proprietary, I highly doubt you'll have much luck with that second item.

VBA is dead. It is no longer maintained. The VBA-M team has taken over development of this, but the codebase is horribly cluttered and virtually impossible to refactor successfully (it's riddled with globals and other things that make running multiple instances in one process a nightmare). However, there are all sorts of useful bits in there. I'm currently working on a C++ library that will allow a custom emulator to be built with relative ease. The source code for this library is not available as of yet, but I will be releasing it when I actually have made good progress. Otherwise, you can find VBA-M's source on their SVN (again, Google is your friend), or in a multitude of GitHub repositories.

I'm reserving judgement on this though. Your father seems to be asking an awful lot that could have been resolved with a few seconds of searching (a lot less time than writing that list). Wikipedia would have told you to look for VBA-M, and the first thing a seasoned software developer would have done would be to perform an SVN checkout - it is under the GPL, so obviously someone must have the source.

I realise that I may not have chosen the proper wording. My grasp of programming languages is not nearly on the level of most high-level hackers in the community. I am aware that of course, a programming language cannot be fully automated, but the intent was making certain common routines easier while making a program that can insert them without hassle and multiple steps. However, if tutorials can include sample ASM routines that work simply by assembling them and placing them in free space, then a library of samples for different tasks should not be hard to include. On another hand, I had not seen a GBATek manual, but I will certainly look it up.

I am very aware that the VBA project is dead. I asked if anyone had the 1.8.0 source code because I did indeed read about VBA-M and heard that the code was quite a mess. If finding the 1.8.0 code is impossible, then it is. But I'd like to keep searching. And as I am still gathering information, it seemed a lot simpler to ask "does someone have the file?" because I cannot find it.

Please do not assume that I did not do research, and that I have any intent other than to simply ask people for information in hopes of helping this come to fruition. I have already searched the PC forums and archives as well as other websites, but there is always information missed. An obscure topic that I didn't see. Something someone hasn't written into a document yet. Something where the link is dead. That is the kind of information I am hoping for.

It's a shame that the IDA pro software is not open source, but it can't be helped. However, it's wonderful that you are working on that library, and I hope it progresses well.

miksy91 said:
Simple and user-friendly asm hacking program would work kinda like XSE or other script editors do;

It could disassemble any routine you want by just telling it the address where the routine is, and you could use it write your own code at any rom address you like (and possibly also make it possible for the program to look for space for the routine and insert it there).

This does sound ideal, indeed. Thank you. A feature to search for free space is incredibly convenient.

Touched · Jul 24, 2014

I'm sorry I assumed that you had not done your research. I just thought that since you were asking for 1.8.0 that you didn't know about VBA-M. The latest stable release of the original VBA project is 1.7.2, and its source can be found on their SourceForge page (I can't link to it yet, but it's on Wikipedia).
I would suggest you stay away from it anyway, as it has the same problems as VBA-M. There is actually no advantage to using VBA over VBA-M. You'd just be using an outdated codebase. The VBA-M project just merged a variety of modifications to the original VBA - they didn't perform much refactoring. I plan to get around this mess by encapsulating my new code as best as possible, and then attempting to use multiprocessing to run each instance of the emulator in a separate process.

The best possible (open source) alternative to IDA is radare. I suppose you could look into that, but IDA is pretty unique.

Anyway, may I ask what language you're thinking of writing this in? As well as what languages your father is most experienced in?

Queen Azura · Jul 24, 2014

Touched said:
I'm sorry I assumed that you had not done your research. I just thought that since you were asking for 1.8.0 that you didn't know about VBA-M. The latest stable release of the original VBA project is 1.7.2, and its source can be found on their SourceForge page (I can't link to it yet, but it's on Wikipedia).
I would suggest you stay away from it anyway, as it has the same problems as VBA-M. There is actually no advantage to using VBA over VBA-M. You'd just be using an outdated codebase. The VBA-M project just merged a variety of modifications to the original VBA - they didn't perform much refactoring. I plan to get around this mess by encapsulating my new code as best as possible, and then attempting to use multiprocessing to run each instance of the emulator in a separate process.

The best possible (open source) alternative to IDA is radare. I suppose you could look into that, but IDA is pretty unique.

Anyway, may I ask what language you're thinking of writing this in? As well as what languages your father is most experienced in?

I have already given him the source code for 1.7.2, but I may give him the VBA-M code just in case. I will also look into radare, so thank you.

He plans to write the program in C/C++, which is also the language he is most experienced in. He has experience with a large number of languages (he actually wrote a prominent language for the Atari 8bit computer series, but that's just a fun age check), but is fairly adamant that he wants to use C/C++. We do not have any non-Windows OS' between us, so it's almost certainly only going to have Windows support.

Touched · Jul 24, 2014

Queen Azura said:
We do not have any non-Windows OS' between us, so it's almost certainly only going to have Windows support.

That's sad news indeed. Cross platform capability is certainly something to be considered (necessary, I would argue). Many of the experienced ASM hackers on here use some Linux distro, or another POSIX compliant operating system, and I'm sure they'd agree with me.

Additionally, many people would like to integrate this tool into their own. Maybe you could look into something like SWIG, to allow generation of wrapper libraries. This would allow the majority of the codebase to be written in C/C++, but the GUI to be implemented in Python, or another higher level language. Additionally, any of the low level functionality could be utilised in other tools.
Otherwise, you could at least create a shared library containing the core functionality (written in cross-platform C) and implement your GUI front end separately (with all the Windows-only functions). This would ease porting to another language and integration, particularly if you open sourced it or provided the headers for this library.

Queen Azura · Jul 24, 2014

Touched said:
That's sad news indeed. Cross platform capability is certainly something to be considered (necessary, I would argue). Many of the experienced ASM hackers on here use some Linux distro, or another POSIX compliant operating system, and I'm sure they'd agree with me.

Additionally, many people would like to integrate this tool into their own. Maybe you could look into something like SWIG, to allow generation of wrapper libraries. This would allow the majority of the codebase to be written in C/C++, but the GUI to be implemented in Python, or another higher level language. Additionally, any of the low level functionality could be utilised in other tools.
Otherwise, you could at least create a shared library containing the core functionality (written in cross-platform C) and implement your GUI front end separately (with all the Windows-only functions). This would ease porting to another language and integration, particularly if you open sourced it or provided the headers for this library.

That is definitely something to consider. I asked him about writing it in Python earlier, as I did notice the prevalence of Linux users here, but he was fairly clear that he did not want to use Python. Hopefully he will be open to one of these options.

Team Fail said:
Thing is, if you open-source it, it can be ported to other platforms, or even Java, which runs on almost every OS anyways.

The program will absolutely be open-source. I can't see him not wanting it to be. :)

Touched · Jul 24, 2014

Team Fail said:
Thing is, if you open-source it, it can be ported to other platforms, or even Java, which runs on almost every OS anyways. I'm not a big Java person (I personally find it confusing), but if it means cross-platform compatibility, Java is one of the best for that.

I do C# myself (I've dabbed in C, although I wasn't taught too terribly well...), but yeah, coding it in C or C++, I think there's compilers for different platforms, so if you open-source it, it can be ported with the efficiency it was intended to have.

Yeah, there's a C compiler for basically every processor out there - the language is that ubiquitous. However, porting C/C++ code from one platform to another is not as simple as switching compilers. C/C++ do not have native support for many high level constructs such as GUI, network functionality, etc. Thus you either have to write the code with a cross-platform library that handles all that (GTK+, Qt, etc.), or you have to manually change every piece of code to something that'll work on the target platform. This is why I suggest separating the functionality from the GUI, as it will ease this process significantly.

Another option would be to use Cygwin to force POSIX compliance on Windows.

Tlachtli · Jul 24, 2014

My suggestion for such a tool would be to condense this down into the help document(s). Even with a pretty good understanding of the ideas of programming, I had only ever worked with higher-level languages before trying the GBA's ASM. I reread that document more times than I could count trying to remember what all the useful commands were and how they interacted.

It's really the subtle things: The differences between the lower and upper register numbers. The purposes of r14 and r15. How pushing a register saves the current value for later, and popping eliminates the current value in favor of a previously pushed one. The differences between using "r0" and "[r0]". When to use ldr or mov. The difference between "add r0, r1" and "add r0, r1, r2". When to use b, bl, bx, or blx.

Something akin to XSE's "command help" feature would be invaluable for this sort of thing. A compiler, a decompiler, and a reference guide will go a long way toward helping new ASM hackers.

Queen Azura · Jul 24, 2014

Tlachtli said:
My suggestion for such a tool would be to condense this down into the help document(s). Even with a pretty good understanding of the ideas of programming, I had only ever worked with higher-level languages before trying the GBA's ASM. I reread that document more times than I could count trying to remember what all the useful commands were and how they interacted.

It's really the subtle things: The differences between the lower and upper register numbers. The purposes of r14 and r15. How pushing a register saves the current value for later, and popping eliminates the current value in favor of a previously pushed one. The differences between using "r0" and "[r0]". When to use ldr or mov. The difference between "add r0, r1" and "add r0, r1, r2". When to use b, bl, bx, or blx.

Something akin to XSE's "command help" feature would be invaluable for this sort of thing. A compiler, a decompiler, and a reference guide will go a long way toward helping new ASM hackers.

This is an excellent idea. I will probably get his explanation of things and then write a cohesive document using the information from the GBATek manual and other sources.

Touched said:
Yeah, there's a C compiler for basically every processor out there - the language is that ubiquitous. However, porting C/C++ code from one platform to another is not as simple as switching compilers. C/C++ do not have native support for many high level constructs such as GUI, network functionality, etc. Thus you either have to write the code with a cross-platform library that handles all that (GTK+, Qt, etc.), or you have to manually change every piece of code to something that'll work on the target platform. This is why I suggest separating the functionality from the GUI, as it will ease this process significantly.

Another option would be to use Cygwin to force POSIX compliance on Windows.

I am not familiar with Cygwin, but I will present the idea to him.

daniilS · Jul 25, 2014

Would it be possible to create this in the form of an IDA PRO plugin? Like having an extra option to directly modify anything you see., and then reload related pointers and stuff. Then it could work on FR too.

As for a list of commands: http://problemkaputt.de/gbatek.htm#thumbinstructionset

Queen Azura · Jul 25, 2014

Update: I spent last night teaching him how to play Pokemon. He just received the Pokedex. {XD} He's compiled the source code of VBA 1.7.2 and made his own changes to let him observe how Emerald works. He had to install Visual Studio 2003 to compile the code, though.

daniilS said:
Would it be possible to create this in the form of an IDA PRO plugin? Like having an extra option to directly modify anything you see., and then reload related pointers and stuff. Then it could work on FR too.

As for a list of commands: http://problemkaputt.de/gbatek.htm#thumbinstructionset

I spoke to him, and he made it fairly clear that he isn't familiar with IDA PRO, so unfortunately that is not an option right now. He does not develop cross-platform software. (Linux is almost non-existent in his job). I may look into finding help porting the program to other platforms, but right now we can only assure compatibility with Windows and Emerald.

However, thank you for the list of commands. That manual is a godsend.

Touched · Jul 25, 2014

Queen Azura said:
He's compiled the source code of VBA 1.7.2 and made his own changes to let him observe how Emerald works. He had to install Visual Studio 2003 to compile the code, though.

Yeah, you really shouldn't use such outdated code - as I've said before, there's no advantage to it. I've almost finished my library, anyway. As soon as I can settle on a good (cross-platform) GUI toolkit, I'll be able to release it. I've already implemented a fully-fledged debugger, among other things.

Queen Azura · Jul 25, 2014

Touched said:
Yeah, you really shouldn't use such outdated code - as I've said before, there's no advantage to it. I've almost finished my library, anyway. As soon as I can settle on a good (cross-platform) GUI toolkit, I'll be able to release it. I've already implemented a fully-fledged debugger, among other things.

Congratulations on almost being done! He basically told me that for his purposes, he just cares if the emulator runs Emerald properly and is stable, so the code he is using is working for him.

daniilS · Jul 26, 2014

Queen Azura said:
Update: I spent last night teaching him how to play Pokemon. He just received the Pokedex. {XD} He's compiled the source code of VBA 1.7.2 and made his own changes to let him observe how Emerald works. He had to install Visual Studio 2003 to compile the code, though.

I spoke to him, and he made it fairly clear that he isn't familiar with IDA PRO, so unfortunately that is not an option right now. He does not develop cross-platform software. (Linux is almost non-existent in his job). I may look into finding help porting the program to other platforms, but right now we can only assure compatibility with Windows and Emerald.

However, thank you for the list of commands. That manual is a godsend.

Well, the reason I suggested it is because IDA has almost everything you need when studying something in the game. Now we'd have to use yet another tool, and he'd need to write lots of functions from scratch. Also, IDA isn't that difficult to use at all, I became familiar with it in a short time. (Except for that cursed gdb)

~~damn, I wish my dad would play pokemon~~

FlyingWaffle · Jul 26, 2014

Quick list of things that no-one has mentioned yet that will be pretty necessary:
devkitpro_org - Free development environment for various consoles including GBA and NDS.
infocenter_arm_com/help/index.jsp - Spec sheet for the processor used in the GBA and NDS (you'll have to search for the right one, it's listed on GBATEK.)
problemkaputt_de/gbatek.htm - GBATEK specifications for GBA and NDS.
cs_rit_edu/~tjh8300/CowBite/CowBiteSpec.htm - CowBiteSpec, similar to GBATEK.
coranac_com/tonc/text/toc.htm - Tonc, complete tutorial and manual on GBA programming.
Pretty much everything you'd need other than emulators/flashcarts is listed here.
(Can't post links, replace underscores with periods.)

Touched · Jul 26, 2014

FlyingWaffle said:
Quick list of things that no-one has mentioned yet that will be pretty necessary:

devkitpro.org - Free development environment for various consoles including GBA and NDS.
infocenter.arm.com/help/index.jsp - Spec sheet for the processor used in the GBA and NDS (you'll have to search for the right one, it's listed on GBATEK.)
problemkaputt.de/gbatek.htm - GBATEK specifications for GBA and NDS.
cs.rit.edu/~tjh8300/CowBite/CowBiteSpec.htm - CowBiteSpec, similar to GBATEK.
coranac.com/tonc/text/toc.htm - Tonc, complete tutorial and manual on GBA programming.
Pretty much everything you'd need other than emulators/flashcarts is listed here.

I've altered your post in this quote so that they're links now.

Those links are pretty useful, even if a few of them have been mentioned (GBATEK, for example).
Queen Azura, maybe you should update the main post with all these links?

FlyingWaffle · Jul 26, 2014

Thanks for fixing the links!
Here are two reference manuals for the ARM7 processor used in the GBA:
infocenter_arm_com/help/topic/com.arm.doc.ddi0029g/DDI0029.pdf
infocenter_arm_com/help/topic/com.arm.doc.ddi0210c/DDI0210B.pdf
I'm not quite sure which one is the right one since the revised copy was released right after the GBA itself was released.
(Same deal with links, sorry.)

Queen Azura · Jul 26, 2014

Touched said:
I've altered your post in this quote so that they're links now.

Those links are pretty useful, even if a few of them have been mentioned (GBATEK, for example).
Queen Azura, maybe you should update the main post with all these links?

FlyingWaffle said:
Thanks for fixing the links!
Here are two reference manuals for the ARM7 processor used in the GBA:
infocenter_arm_com/help/topic/com.arm.doc.ddi0029g/DDI0029.pdf
infocenter_arm_com/help/topic/com.arm.doc.ddi0210c/DDI0210B.pdf
I'm not quite sure which one is the right one since the revised copy was released right after the GBA itself was released.
(Same deal with links, sorry.)

Thank you so much for all these resources! I've sent them on to him and browsed through them briefly myself--they look like a nice, solid foundation of information. I've updated the main post with these links for ease of access as you requested, Touched. ♪

daniilS said:
~~damn, I wish my dad would play pokemon~~

He spent twenty minutes complaining about not being able to move diagonally and that characters do nothing but stand around. He sat in the chair next to the character's mom for a minute and I asked, "What are you doing?" He replied, "I want her to make me dinner. Playing Pokemon is hard."

Research: Calling All ASM Experts!

What features would you like in an ASM tool?

Writing directly to ROMs

Automated routine writing

Disassembling

All of the above

Other (Please post)

Mistress of Midnight

Resident ASMAGICIAN

Dark Energy is back in action! ;)

Mistress of Midnight

Resident ASMAGICIAN

Mistress of Midnight

Resident ASMAGICIAN

Mistress of Midnight

Resident ASMAGICIAN

Crit happens.

Mistress of Midnight

busy trying to do stuff not done yet

Mistress of Midnight

Resident ASMAGICIAN

Mistress of Midnight

busy trying to do stuff not done yet

Resident ASMAGICIAN

Mistress of Midnight