Development: Extensible GBA Emulator

[Touched]

As a GBA hacker, an emulator is one of our fundamental tools. Although the options we have are stable, there is no all-in-one solution. You have to resort to an SDL port of VBA to debug ASM, for example. However, there is no reason to think that an emulator can't be extended like any other tool.

However, VBA-M's source (and VBA's source before it) is cluttered and unwieldy. It is difficult at best to write your own port. Thus, I've written my own generic port, and exposed a C interface for you to use. This will allow custom ports to be readily created. One caveat however: You cannot run multiple instances in the same process. The source is too cluttered with global variables to refactor satisfactorily. If you want to get around this, load up the shared library in another process.

The source can be cloned from the GitHub repository:

git clone https://github.com/Touched/libatlantis.git

I'll update this post with a sample application when I've made more progress.

 

[PmHacks]

Why is it based on a version of VBA-M thats 3 years old? You should use the latest svn.

 

[Touched]

Why is it based on a version of VBA-M thats 3 years old? You should use the latest svn.

I could not compile the latest svn due to apparent incompatibilities with my version of SFML. I'm still attempting to rectify the issue, but I just uploaded the older version of the source as a proof-of-concept. I still haven't deduced whether this would be useful to anyone, hence the "Survey" in the title.

UPDATE: I've disabled GBA Link (there's no real use for it?) functionality, thereby removing the SFML dependency. I will update the source to the latest SVN. This issue can be worked out later.

 

[Sierraffinity]

I would personally find this very useful, as VBA-SDL-H can get pretty finicky when it comes to working at multiple computers; usually changing out the SDL.dll will fix it but not always. It would also be nice to see a GUI instead of the usual CMD window.

 

[Full Metal]

I'll work on building it on Windows. (:

 

[Touched]

I would personally find this very useful, as VBA-SDL-H can get pretty finicky when it comes to working at multiple computers; usually changing out the SDL.dll will fix it but not always. It would also be nice to see a GUI instead of the usual CMD window.

I'm not sure that this will fix that problem, as it is based on SDL, too (VBA-SDL-H and VBA-M SDL share a codebase). But it would be possible to add a GUI, using a Python GUI library.

I'll work on building it on Windows. (:

Sounds good :)
I used Python 3.4 to compile this. You'll have to change the required includes and libraries in CMakeLists.txt manually. The find package function doesn't seem to work for Python 3.

You'll also need zlib, libpng, OpenGL and SDL development libraries. Don't bother with the GTK+ port. It includes far less than the SDL version, and Windows's GTK+ support is lacklustre anyway.

 

[Full Metal]

I'm not sure that this will fix that problem, as it is based on SDL, too (VBA-SDL-H and VBA-M SDL share a codebase). But it would be possible to add a GUI, using a Python GUI library.



Sounds good :)
I used Python 3.4 to compile this. You'll have to change the required includes and libraries in CMakeLists.txt manually. The find package function doesn't seem to work for Python 3.

You'll also need zlib, libpng, OpenGL and SDL development libraries. Don't bother with the GTK+ port. It includes far less than the SDL version, and Windows's GTK+ support is lacklustre anyway.

Trust me, I'm plenty familiar with GTK+ on Windows. I realized that I was missing zlib and libpng pretty quickly, but as far as CMake goes, I just opened the solution in Visual Studio? .-.

 

[Phenom2122]

I've been poking around the source for VBA-M (A variation of the popular VBA emulator), and have embedded the Python interpreter into it. Using the C API, one can expose the methods in the source to a Python module. You can then ask the interpreter to load some scripts in a directory, creating a simple plug-in interface for the emulator.

Does anyone think that this would be at all useful? One could potentially create an array of debuggers, or more in-depth tools than the ones already present in VBA.

Oh hell yes. Sounds to me, with a plug-in system like that, there are a million possibilities.
Apologies if this is a stupid question, I am just starting my journey into programming. With a system like this, would it be possible to make hack specific plugins? For example, a bigger better Pokémon PC storage, saved in some file on your computer that the game accesses instead of the normal method? Would that be possible to hijack certain game routines like that to give bigger ram etc? Kind of like a gba emulator that extends games into the pc realm.

 

[Touched]

Trust me, I'm plenty familiar with GTK+ on Windows. I realized that I was missing zlib and libpng pretty quickly, but as far as CMake goes, I just opened the solution in Visual Studio? .-.

The GTK+ port is very minimal and not really worth it if you ask me. I haven't configured the Visual Studio project, so you'll have to make sure to include the Python development binaries and includes (preferably 3.4 - I'm using things from the C API that require 3.3 or greater). Also, remember to disable VBA Link - its apparently very buggy. It might even be possible to write the linking in Python!

Oh hell yes. Sounds to me, with a plug-in system like that, there are a million possibilities.
Apologies if this is a stupid question, I am just starting my journey into programming. With a system like this, would it be possible to make hack specific plugins? For example, a bigger better Pokémon PC storage, saved in some file on your computer that the game accesses instead of the normal method? Would that be possible to hijack certain game routines like that to give bigger ram etc? Kind of like a gba emulator that extends games into the pc realm.

Yes, you could use it for that. The API is very low level at the moment, but it provides access to the different blocks of memory (WRAM, IRAM, ROM, etc.) and to the CPU registers, so you could theoretically inject some ASM routine or something to control what happens in the game. You could also check the ROM header to ensure you only run the plugin on compatible ROMs. I'm not sure about more memory. You could probably change the source to allow more memory, but I'm not sure how useful that would be.
To be honest, I didn't really envision anything like that. I thought it could be used to debug script routines (with breakpoints and stepping) and other functions for testing/debugging purposes (Warp to a specific map, etc.).
But, theoretically, you could add any functionality through Python. As long as someone adds the correct C code for it.

 

[Warpten]

CMake can generate you .sln files super easily, just select the target (MS12 if VS2013, etc) and there shouldn't be any problem. It'd probably be cool if you put this somewhere on GitHub, Bitbucker or any of your preffered CVS :p

Haven't tried it yet (my real work device is dead, i'll need time to get a new one), but this sure sounds great!

 

[Touched]

CMake can generate you .sln files super easily, just select the target (MS12 if VS2013, etc) and there shouldn't be any problem. It'd probably be cool if you put this somewhere on GitHub, Bitbucker or any of your preffered CVS :p

Haven't tried it yet (my real work device is dead, i'll need time to get a new one), but this sure sounds great!

I plan to put this on GitHub, but as I haven't yet reached 15 posts, I couldn't post a link to it or anything. That point about CMake is interesting. I haven't used CMake all that much, but the FindPythonLibs module seems to be broken (at least for Python 3) on Ubuntu. Does it work on Windows? If not, there is no real point in generating .sin files.

 

[Warpten]

Cool, I just lost all I wrote.

Summing it up:
- Adding execution and memory breakpoints, with conditions, would be very nice.
- Also mimic VS's Watches (auto updates value in a memory address, convenient instead of having to "m(h|b|d) offset" in VBA SDL H)

About CMake:
I'm not a CMake guru, but by the looks of it (for old cmake versions), FindPythonModule will only look for Python up to 2.7. Try "find_package(PythonLibs EXACT REQUIRED 3.0)" or "find_package(PythonLibs REQUIRED 3.0)" (EXACT just makes it so that _only_ 3.0 is picked; not above).

cmake.org/Bug/print_bug_page.php?bug_id=13794

also, VS solution files are not .SIN, but SLN :p


[EDIT] You might want to update to latest Cmake:
cmake.org/cmake/help/v3.0/module/FindPythonLibs.html
github.com/Kitware/CMake/blob/master/Modules/FindPythonLibs.cmake

You can also enforce any version of cmake in your CMake scripts by calling "cmake_minimum_required(VERSION 2.8.9)" in the main CMakeLists.txt of your project.

Also, protip: the URL parser looks for "http (s) : / / (w w w)", so strip them out :p

 

[Touched]

Cool, I just lost all I wrote.

Summing it up:
- Adding execution and memory breakpoints, with conditions, would be very nice.
- Also mimic VS's Watches (auto updates value in a memory address, convenient instead of having to "m(h|b|d) offset" in VBA SDL H)

About CMake:
I'm not a CMake guru, but by the looks of it (for old cmake versions), FindPythonModule will only look for Python up to 2.7. Try "find_package(PythonLibs EXACT REQUIRED 3.0)" or "find_package(PythonLibs REQUIRED 3.0)" (EXACT just makes it so that _only_ 3.0 is picked; not above).

cmake.org/Bug/print_bug_page.php?bug_id=13794

also, VS solution files are not .SIN, but SLN :p


[EDIT] You might want to update to latest Cmake:
cmake.org/cmake/help/v3.0/module/FindPythonLibs.html
github.com/Kitware/CMake/blob/master/Modules/FindPythonLibs.cmake

You can also enforce any version of cmake in your CMake scripts by calling "cmake_minimum_required(VERSION 2.8.9)" in the main CMakeLists.txt of your project.

Also, protip: the URL parser looks for "http (s) : / / (w w w)", so strip them out :p

Thanks so much. The FindPythonLibs.cmake in my version finds the shared object, just not the includes. It looks like the latest release of CMake (3.0) supports up to Python 3.4 (the version I am using). The previous release of CMake (2.8.12), which was the one in the Ubuntu PPA, does not seem to have this support. I'll upgrade to that now and create a GitHub repository.

I thought I could bypass the parser (it's probably just a basic URL Regex), but I thought any post doing so would just be removed. I guess that that rule isn't strictly enforced then?

 

[Warpten]

I think it's just a limitation for scamming bots. Hopefully mods don't get back on me D:

 

[hremeraude]

GBA-link is very usesful...

 

[Touched]

I think it's just a limitation for scamming bots. Hopefully mods don't get back on me D:

I just tried pasting a link into a post, and it works - even though I haven't made 15 posts :/

GBA-link is very usesful...

Would you care to have a go at porting it then?

UPDATE: I've added a link to the GitHub repo.

 

[hremeraude]

Yes, thanks (I use it to put events, make trades, use E-Reader on a game).
It can be useful :)

 

[Touched]

Currently, when you start the emulator, it looks for a file in the current directory called vba.py. It loads this file, and calls the load() function inside it. This will form the plug-in loader after the API is complete. Right now, the API consists of several MemoryView buffers, allowing read/write access to the various GBA memory blocks, as well as the CPU and IO registers. Additionally, I've provided read-only access to the BIOS and ROM memory blocks. This already adds quite a lot of functionality. I plan to add other features to make this API contain various other plug-in features, such as event listeners (JQuery style) and hooks (WordPress style). Additionally, I plan to create Python versions of most of the standard emulator functions (Loading and resetting ROMS, automated patching, graphics filters, debugging, etc.)

I've also thought of another fairly cool idea. Using the "API" as it currently stands, one could create a plug-in that writes a particular byte sequence to a free area in the GBA RAM on start up. It could then ensure that this sequence is not over-wrriten, allowing someone to check whether this emulator is being used in-game (from a script or assembly routine or something). This could (possibly) open up avenues for extra features in scripts: we could see what plug-ins are loaded or whatever.

Any input from C/C++ or Python programmers would be greatly appreciated. What would you like to see in the API? If you wish to contribute, you can just send me a pull request.

 

[Full Metal]

What I would do, personally speaking, would be to write the main code in C and adapt it so that it can be compiled to an DLL. Then write a library to access the library from Python, and control it that way. Also, I haven't had time to try building the program again, and I might not for a little bit.

 

[Touched]

What I would do, personally speaking, would be to write the main code in C and adapt it so that it can be compiled to an DLL. Then write a library to access the library from Python, and control it that way. Also, I haven't had time to try building the program again, and I might not for a little bit.

It wouldn't be as convenient. Right now, I can create wrappers for any C function or a pointer to an arbitrary struct or whatever. I can control the emulator's current state by embedding Python. The alternative you speak of is a compiled module, which wouldn't make much sense given multiple instances of the emulator, and exposing memory blocks would be much more difficult.