The PokéCommunity Forums

The PokéCommunity Forums (https://www.pokecommunity.com/index.php)
-   Binary Hack Research & Development (https://www.pokecommunity.com/forumdisplay.php?f=195)
-   -   Development Extensible GBA Emulator (https://www.pokecommunity.com/showthread.php?t=330540)

Touched July 5th, 2014 2:55 AM

Extensible GBA Emulator
 
As a GBA hacker, an emulator is one of our fundamental tools. Although the options we have are stable, there is no all-in-one solution. You have to resort to an SDL port of VBA to debug ASM, for example. However, there is no reason to think that an emulator can't be extended like any other tool.

However, VBA-M's source (and VBA's source before it) is cluttered and unwieldy. It is difficult at best to write your own port. Thus, I've written my own generic port, and exposed a C interface for you to use. This will allow custom ports to be readily created. One caveat however: You cannot run multiple instances in the same process. The source is too cluttered with global variables to refactor satisfactorily. If you want to get around this, load up the shared library in another process.

The source can be cloned from the GitHub repository:
Code:

git clone https://github.com/Touched/libatlantis.git


I'll update this post with a sample application when I've made more progress.

PmHacks July 6th, 2014 7:00 AM

Why is it based on a version of VBA-M thats 3 years old? You should use the latest svn.

Touched July 6th, 2014 8:11 AM

Quote:

Originally Posted by PmHacks (Post 8333371)
Why is it based on a version of VBA-M thats 3 years old? You should use the latest svn.

I could not compile the latest svn due to apparent incompatibilities with my version of SFML. I'm still attempting to rectify the issue, but I just uploaded the older version of the source as a proof-of-concept. I still haven't deduced whether this would be useful to anyone, hence the "Survey" in the title.

UPDATE: I've disabled GBA Link (there's no real use for it?) functionality, thereby removing the SFML dependency. I will update the source to the latest SVN. This issue can be worked out later.

Sierraffinity July 6th, 2014 4:11 PM

I would personally find this very useful, as VBA-SDL-H can get pretty finicky when it comes to working at multiple computers; usually changing out the SDL.dll will fix it but not always. It would also be nice to see a GUI instead of the usual CMD window.

Full Metal July 6th, 2014 5:50 PM

I'll work on building it on Windows. (:

Touched July 6th, 2014 6:38 PM

Quote:

Originally Posted by diegoisawesome (Post 8334239)
I would personally find this very useful, as VBA-SDL-H can get pretty finicky when it comes to working at multiple computers; usually changing out the SDL.dll will fix it but not always. It would also be nice to see a GUI instead of the usual CMD window.

I'm not sure that this will fix that problem, as it is based on SDL, too (VBA-SDL-H and VBA-M SDL share a codebase). But it would be possible to add a GUI, using a Python GUI library.

Quote:

Originally Posted by Full Metal (Post 8334345)
I'll work on building it on Windows. (:

Sounds good :)
I used Python 3.4 to compile this. You'll have to change the required includes and libraries in CMakeLists.txt manually. The find package function doesn't seem to work for Python 3.

You'll also need zlib, libpng, OpenGL and SDL development libraries. Don't bother with the GTK+ port. It includes far less than the SDL version, and Windows's GTK+ support is lacklustre anyway.

Full Metal July 7th, 2014 4:10 AM

Quote:

Originally Posted by Touched (Post 8334392)
I'm not sure that this will fix that problem, as it is based on SDL, too (VBA-SDL-H and VBA-M SDL share a codebase). But it would be possible to add a GUI, using a Python GUI library.



Sounds good :)
I used Python 3.4 to compile this. You'll have to change the required includes and libraries in CMakeLists.txt manually. The find package function doesn't seem to work for Python 3.

You'll also need zlib, libpng, OpenGL and SDL development libraries. Don't bother with the GTK+ port. It includes far less than the SDL version, and Windows's GTK+ support is lacklustre anyway.

Trust me, I'm plenty familiar with GTK+ on Windows. I realized that I was missing zlib and libpng pretty quickly, but as far as CMake goes, I just opened the solution in Visual Studio? .-.

Phenom2122 July 7th, 2014 5:54 AM

Quote:

Originally Posted by Touched (Post 8331182)
I've been poking around the source for VBA-M (A variation of the popular VBA emulator), and have embedded the Python interpreter into it. Using the C API, one can expose the methods in the source to a Python module. You can then ask the interpreter to load some scripts in a directory, creating a simple plug-in interface for the emulator.

Does anyone think that this would be at all useful? One could potentially create an array of debuggers, or more in-depth tools than the ones already present in VBA.

Oh hell yes. Sounds to me, with a plug-in system like that, there are a million possibilities.
Apologies if this is a stupid question, I am just starting my journey into programming. With a system like this, would it be possible to make hack specific plugins? For example, a bigger better Pokémon PC storage, saved in some file on your computer that the game accesses instead of the normal method? Would that be possible to hijack certain game routines like that to give bigger ram etc? Kind of like a gba emulator that extends games into the pc realm.

Touched July 7th, 2014 9:06 AM

Quote:

Originally Posted by Full Metal (Post 8334948)
Trust me, I'm plenty familiar with GTK+ on Windows. I realized that I was missing zlib and libpng pretty quickly, but as far as CMake goes, I just opened the solution in Visual Studio? .-.

The GTK+ port is very minimal and not really worth it if you ask me. I haven't configured the Visual Studio project, so you'll have to make sure to include the Python development binaries and includes (preferably 3.4 - I'm using things from the C API that require 3.3 or greater). Also, remember to disable VBA Link - its apparently very buggy. It might even be possible to write the linking in Python!

Quote:

Originally Posted by Phenom2122 (Post 8335069)
Oh hell yes. Sounds to me, with a plug-in system like that, there are a million possibilities.
Apologies if this is a stupid question, I am just starting my journey into programming. With a system like this, would it be possible to make hack specific plugins? For example, a bigger better Pokémon PC storage, saved in some file on your computer that the game accesses instead of the normal method? Would that be possible to hijack certain game routines like that to give bigger ram etc? Kind of like a gba emulator that extends games into the pc realm.

Yes, you could use it for that. The API is very low level at the moment, but it provides access to the different blocks of memory (WRAM, IRAM, ROM, etc.) and to the CPU registers, so you could theoretically inject some ASM routine or something to control what happens in the game. You could also check the ROM header to ensure you only run the plugin on compatible ROMs. I'm not sure about more memory. You could probably change the source to allow more memory, but I'm not sure how useful that would be.
To be honest, I didn't really envision anything like that. I thought it could be used to debug script routines (with breakpoints and stepping) and other functions for testing/debugging purposes (Warp to a specific map, etc.).
But, theoretically, you could add any functionality through Python. As long as someone adds the correct C code for it.

Warpten July 7th, 2014 10:11 AM

CMake can generate you .sln files super easily, just select the target (MS12 if VS2013, etc) and there shouldn't be any problem. It'd probably be cool if you put this somewhere on GitHub, Bitbucker or any of your preffered CVS :p

Haven't tried it yet (my real work device is dead, i'll need time to get a new one), but this sure sounds great!

Touched July 7th, 2014 11:53 AM

Quote:

Originally Posted by Warpten (Post 8335386)
CMake can generate you .sln files super easily, just select the target (MS12 if VS2013, etc) and there shouldn't be any problem. It'd probably be cool if you put this somewhere on GitHub, Bitbucker or any of your preffered CVS :p

Haven't tried it yet (my real work device is dead, i'll need time to get a new one), but this sure sounds great!

I plan to put this on GitHub, but as I haven't yet reached 15 posts, I couldn't post a link to it or anything. That point about CMake is interesting. I haven't used CMake all that much, but the FindPythonLibs module seems to be broken (at least for Python 3) on Ubuntu. Does it work on Windows? If not, there is no real point in generating .sin files.

Warpten July 8th, 2014 3:02 AM

Cool, I just lost all I wrote.

Summing it up:
- Adding execution and memory breakpoints, with conditions, would be very nice.
- Also mimic VS's Watches (auto updates value in a memory address, convenient instead of having to "m(h|b|d) offset" in VBA SDL H)

About CMake:
I'm not a CMake guru, but by the looks of it (for old cmake versions), FindPythonModule will only look for Python up to 2.7. Try "find_package(PythonLibs EXACT REQUIRED 3.0)" or "find_package(PythonLibs REQUIRED 3.0)" (EXACT just makes it so that _only_ 3.0 is picked; not above).

cmake.org/Bug/print_bug_page.php?bug_id=13794

also, VS solution files are not .SIN, but SLN :p


[EDIT] You might want to update to latest Cmake:
cmake.org/cmake/help/v3.0/module/FindPythonLibs.html
github.com/Kitware/CMake/blob/master/Modules/FindPythonLibs.cmake

You can also enforce any version of cmake in your CMake scripts by calling "cmake_minimum_required(VERSION 2.8.9)" in the main CMakeLists.txt of your project.

Also, protip: the URL parser looks for "http (s) : / / (w w w)", so strip them out :p

Touched July 8th, 2014 4:28 AM

Quote:

Originally Posted by Warpten (Post 8336818)
Cool, I just lost all I wrote.

Summing it up:
- Adding execution and memory breakpoints, with conditions, would be very nice.
- Also mimic VS's Watches (auto updates value in a memory address, convenient instead of having to "m(h|b|d) offset" in VBA SDL H)

About CMake:
I'm not a CMake guru, but by the looks of it (for old cmake versions), FindPythonModule will only look for Python up to 2.7. Try "find_package(PythonLibs EXACT REQUIRED 3.0)" or "find_package(PythonLibs REQUIRED 3.0)" (EXACT just makes it so that _only_ 3.0 is picked; not above).

cmake.org/Bug/print_bug_page.php?bug_id=13794

also, VS solution files are not .SIN, but SLN :p


[EDIT] You might want to update to latest Cmake:
cmake.org/cmake/help/v3.0/module/FindPythonLibs.html
github.com/Kitware/CMake/blob/master/Modules/FindPythonLibs.cmake

You can also enforce any version of cmake in your CMake scripts by calling "cmake_minimum_required(VERSION 2.8.9)" in the main CMakeLists.txt of your project.

Also, protip: the URL parser looks for "http (s) : / / (w w w)", so strip them out :p

Thanks so much. The FindPythonLibs.cmake in my version finds the shared object, just not the includes. It looks like the latest release of CMake (3.0) supports up to Python 3.4 (the version I am using). The previous release of CMake (2.8.12), which was the one in the Ubuntu PPA, does not seem to have this support. I'll upgrade to that now and create a GitHub repository.

I thought I could bypass the parser (it's probably just a basic URL Regex), but I thought any post doing so would just be removed. I guess that that rule isn't strictly enforced then?

Warpten July 8th, 2014 5:22 AM

I think it's just a limitation for scamming bots. Hopefully mods don't get back on me D:

hremeraude July 8th, 2014 9:00 AM

GBA-link is very usesful...

Touched July 8th, 2014 1:40 PM

Quote:

Originally Posted by Warpten (Post 8336920)
I think it's just a limitation for scamming bots. Hopefully mods don't get back on me D:

I just tried pasting a link into a post, and it works - even though I haven't made 15 posts :/

Quote:

Originally Posted by hremeraude (Post 8337173)
GBA-link is very usesful...

Would you care to have a go at porting it then?

UPDATE: I've added a link to the GitHub repo.

hremeraude July 8th, 2014 2:00 PM

Yes, thanks (I use it to put events, make trades, use E-Reader on a game).
It can be useful :)

Touched July 8th, 2014 9:33 PM

Currently, when you start the emulator, it looks for a file in the current directory called vba.py. It loads this file, and calls the load() function inside it. This will form the plug-in loader after the API is complete. Right now, the API consists of several MemoryView buffers, allowing read/write access to the various GBA memory blocks, as well as the CPU and IO registers. Additionally, I've provided read-only access to the BIOS and ROM memory blocks. This already adds quite a lot of functionality. I plan to add other features to make this API contain various other plug-in features, such as event listeners (JQuery style) and hooks (WordPress style). Additionally, I plan to create Python versions of most of the standard emulator functions (Loading and resetting ROMS, automated patching, graphics filters, debugging, etc.)

I've also thought of another fairly cool idea. Using the "API" as it currently stands, one could create a plug-in that writes a particular byte sequence to a free area in the GBA RAM on start up. It could then ensure that this sequence is not over-wrriten, allowing someone to check whether this emulator is being used in-game (from a script or assembly routine or something). This could (possibly) open up avenues for extra features in scripts: we could see what plug-ins are loaded or whatever.

Any input from C/C++ or Python programmers would be greatly appreciated. What would you like to see in the API? If you wish to contribute, you can just send me a pull request.

Full Metal July 9th, 2014 7:13 AM

What I would do, personally speaking, would be to write the main code in C and adapt it so that it can be compiled to an DLL. Then write a library to access the library from Python, and control it that way. Also, I haven't had time to try building the program again, and I might not for a little bit.

Touched July 9th, 2014 7:39 AM

Quote:

Originally Posted by Full Metal (Post 8339029)
What I would do, personally speaking, would be to write the main code in C and adapt it so that it can be compiled to an DLL. Then write a library to access the library from Python, and control it that way. Also, I haven't had time to try building the program again, and I might not for a little bit.

It wouldn't be as convenient. Right now, I can create wrappers for any C function or a pointer to an arbitrary struct or whatever. I can control the emulator's current state by embedding Python. The alternative you speak of is a compiled module, which wouldn't make much sense given multiple instances of the emulator, and exposing memory blocks would be much more difficult.

Phenom2122 July 9th, 2014 10:22 AM

Plus if everything needs to be compiled, that cuts off a certain amount of freedom for the user I would say. Scripts on the other hand could easily be tweaked to individual needs and improved by many.

Shiny Quagsire July 9th, 2014 1:41 PM

Quote:

Originally Posted by Touched (Post 8339063)
It wouldn't be as convenient. Right now, I can create wrappers for any C function or a pointer to an arbitrary struct or whatever. I can control the emulator's current state by embedding Python. The alternative you speak of is a compiled module, which wouldn't make much sense given multiple instances of the emulator, and exposing memory blocks would be much more difficult.

Yes, but if the extension portion of the code is modularized, then you could write libraries for other languages such as Lua and even Java if properly wrapped, allowing choice of language at the slight cost of full control over the application. However, if you expose enough parts of the program, the disadvantage isn't as evident and you gain a lot of possibilities. A project very similar to this one by knizz was made which used Lua. The difference however was that it exposed the methods into the Lua manually, but enough was exposed that you could do a lot of powerful things. Obviously this is still your project but it was just a thought I had.

Touched July 10th, 2014 7:03 AM

Quote:

Originally Posted by Shiny Quagsire (Post 8339722)
Yes, but if the extension portion of the code is modularized, then you could write libraries for other languages such as Lua and even Java if properly wrapped, allowing choice of language at the slight cost of full control over the application. However, if you expose enough parts of the program, the disadvantage isn't as evident and you gain a lot of possibilities. A project very similar to this one by knizz was made which used Lua. The difference however was that it exposed the methods into the Lua manually, but enough was exposed that you could do a lot of powerful things. Obviously this is still your project but it was just a thought I had.

What I was planning to do was create a generic wrapper for the plug-in module, which would include things like the loading of the module, etc. Thus, if someone wanted to change the language that the module was loaded in, all they'd have to do would be rewrite this module. You'd have to know C in order to do this anyway, it'd just be more convenient. However, I do not see why you'd want to use Java. Lua or Ruby I could understand, but if you're using Java, you might as well just use C++? I'm going to try to make this project as modular as possible though. Thanks for that link. I'll look into the source to see how knizz embedded Lua. However, I'm of the opinion that Python is a much more versatile and succinct language than Lua.

Touched July 26th, 2014 5:02 AM

In an attempt to acquiesce to requests here, I've create a C++ library which will form the back-end of the new Python-based emulator. This library contains all the core functions necessary to implement your own emulator, in whatever language you desire (as long as you write the appropriate bindings). I shall attempt to make the code as minimal and as fully functional as possible. Write now, a number of (critical) things do not work properly. I've disabled save states, because I'd like to make them saved to the RAM, rather than a file on the disk. This will be to allow you to decide what you want to do with the file. However, I've modified the processor core to allow for better debugging features. You can now set read/write breakpoints on any address you desire. Anyway, this is basically just a taster. I'll commit more changes soon. Now that I've forked VBA-M's source, I'll be free to edit as much of the source as I please - this is what was slowing my progress down before.

The source can be found on my GitHub repository. I'll update the main post with a link, and hopefully with a sample application. Otherwise, you can look at the test application in the source. It doesn't do much, but it's a proof of concept.

NearEDGE August 8th, 2014 5:35 AM

Quote:

Originally Posted by Touched (Post 8331182)
As a GBA hacker, an emulator is one of our fundamental tools. Although the options we have are stable, there is no all-in-one solution. You have to resort to an SDL port of VBA to debug ASM, for example. However, there is no reason to think that an emulator can't be extended like any other tool.

No one uses NO$GBA apparently XD
Also, you talk about the SDL version of VBA like it's a bad thing.


All times are GMT -8. The time now is 8:50 AM.


Like our Facebook Page Follow us on Twitter © 2002 - 2018 The PokéCommunity™, pokecommunity.com.
Pokémon characters and images belong to The Pokémon Company International and Nintendo. This website is in no way affiliated with or endorsed by Nintendo, Creatures, GAMEFREAK, The Pokémon Company or The Pokémon Company International. We just love Pokémon.
All forum styles, their images (unless noted otherwise) and site designs are © 2002 - 2016 The PokéCommunity / PokéCommunity.com.
PokéCommunity™ is a trademark of The PokéCommunity. All rights reserved. Sponsor advertisements do not imply our endorsement of that product or service. User generated content remains the property of its creator.

Acknowledgements
Use of PokéCommunity Assets
vB Optimise by DragonByte Technologies Ltd © 2023.