• Our software update is now concluded. You will need to reset your password to log in. In order to do this, you will have to click "Log in" in the top right corner and then "Forgot your password?".
  • Welcome to PokéCommunity! Register now and join one of the best fan communities on the 'net to talk Pokémon and more! We are not affiliated with The Pokémon Company or Nintendo.

Modern ROM Hacking: A Call to Action

Akiba

[img]http://i.imgur.com/o3RYT4v.png[/img]
4,262
Posts
13
Years
  • Hey all. Sync here. I've been in and out of the hacking scene for about five years now.

    I've made several attempts in the past to create a comfortable ecosystem for all ROM hackers. All have failed, because the barriers of entry were too high, the approach was wrong, or I simply did not have enough time to get it up and running to a self-sustaining state.

    After much contemplation, I've arrived at a conclusion. We've been doing it all wrong. In the past, I've stressed how our current tooling system is inconsistent, uncooperative, and fragmented. I've tried to solve this by introducing a centralized documentation project (it's still a good read, by the way). But this doesn't reduce the barrier to entry, and without a proper tooling system, could encourage even deeper fragmentation.

    The first step to a better ecosystem is organization. I've said this before. The next step is abstraction and simplicity. How complicated are the current tutorials for making hacking tools? How hard do you think it should be to make one?

    If you ask me, I'd say it should be this easy:
    Code:
    import fs from 'fs';
    
    let file = './BPRE.gba';
    let ROM = fs.readFileSync(file);
    ROM.writeUInt8(0, 0xbd494);
    fs.writeFileSync(file, ROM);

    These several lines of Javascript take care of reading a ROM, writing a byte, and saving it. In other words, we've allowed running indoors on BPRE. Why do we need to go through so much trouble, clicking through wizards and GUI editors, just to do something so simple? These few lines are far less intimidating than an enormous Visual C# tutorial that accomplishes the same thing (and even then, only runs on Windows). Many of these tutorials don't really teach you how to make hacking tools, but rather .NET visual applications. I was fully guilty of committing this as a 13 year-old. As a result, most of the time spent making hacking tools is really spent making visual interfaces, which doesn't really sound like a passionate hacker's job to me. I've seen dozens of people who were really excited about hacking, but didn't know how to work UIs or other things completely unrelated to hacking, and eventually gave up on their precious work. What if we could just have one well-maintained application, that binds together different functionality, written by hackers, for hackers?

    Let's talk about the code above. It probably shouldn't even be your job to take care of reading and writing files. So, delegating that task to our manager, let's simplify it to this:
    Code:
    export default ROM => {
    	ROM.writeUInt8(0, 0xbd494);
    }

    Now, we ship a single function that writes the indoor running byte to a ROM. We don't care where it came from, or what else has been done to it.

    But, since lots of small patches are really simple, writing data to offsets, why don't we allow some shortcuts?
    Code:
    export default {
    	offset: 0xbd494,
    	data: 0
    }

    Now, we have code that is really DRY and orthogonal.

    That thing is a hacking tool. Maybe we shouldn't even call them hacking tools anymore. That would show that we've really matured.

    Now, what I've written above will absolutely work (install node/iojs and babel to run!), but it's also just a proof-of-concept demonstration. You might be wondering how it actually runs. Unfortunately, you need a terminal. However, as I mentioned above, we can create and maintain a tool that binds together these "hacking tools," allowing everyone to hack as they always have (in that you all can still click around colorful views).

    My purpose is to show that, by only worrying about what we need to worry about when making a hacking tool, we can achieve rich composition. Easily said, we can combine tools, make them work together, improve productivity, and greatly reduce the potential for bugs. We can also then stop worrying about pointless and irrelevant things, and move on to higher-level and more intelligent hacking.

    In fact, the ecosystem that I've outlined above introduces possibilities for powerful debugging, testing, and other development practices that we've been missing out on. We even get backups, logs/journals, and revision control for free! I may write a post about that later.

    Please consider what I've said, as it really would move the hacking scene forward.
     

    Alexander Nicholi

    what do you know about computing?
    5,500
    Posts
    14
    Years
  • I have a lot of interest in naming this project, if the names of my past projects are of any appeal at all. Besides that, I also naturally have a high interest in how it would handle the numerous scripting engine bytecodes used in our ROM bases.

    I'm 100% for this. It is about time the amateur stuff stop.
     
    39
    Posts
    8
    Years
    • Seen Jul 29, 2018
    so what you basically want is sort of a tool to create tools? like a language of some kind? like people sharing lines of asm code you want people to share lines of javascript code and other people using them?
    in my opinion this would be much more useful than some strange windows form where you have to click some buttons and stuff and nobody knows what really happens
     

    Akiba

    [img]http://i.imgur.com/o3RYT4v.png[/img]
    4,262
    Posts
    13
    Years
  • My proposition is to use the excellent ecosystem that node/npm already provides us.

    As I said, the code runs above in node.js.

    I see the most value in creating a ROM CLI IDE with Vantage.js, but we could always do a GUI.
     

    Touched

    Resident ASMAGICIAN
    625
    Posts
    9
    Years
    • Age 122
    • Seen Feb 1, 2018
    I do this kind of thing for assembly hacking, but it's usually in Python. They dynamically insert routines and apply hooks/code changes so that I can test/create complex, multi-routine hacks.
    However, whenever I share the scripts, the number one complaint is something about it "just opening and then closing" (They double click the script, which Windows has associated with the interpreter, and it just closes after running the script). People have absolutely no idea what a command line is, so if you show them some REPL like Vantage, they're gonna freak out. They need a tutorial to open the command line, and generally cannot even run basic commands. They ask me for help even though I haven't used Windows in years and have to resort to guessing.

    As much as I'd like something like this, there are like maybe 10 programmers here tops, and the only people fit to use these tools are the people who are capable of making them. JavaScript won't make anything easier sadly, especially if you start throwing in even basic functional stuff like map, reduce, filter, etc. It will only get worse if you start using asynchronous code (callbacks or promises, it doesn't matter). Check out the code quality of some of the people - here it's the opposite DRY - it's WET.

    I guess you're probably not including ASM hackers in this, because most of our tools can't be replaced, so how do you expect to accommodate mappers (a very visual task) with a REPL? I think without a GUI, this idea won't catch on. You've suggested Electron in the past, so why not use that? You could embed a console inside.

    Anyway, I was writing a set of core functionality that would be useful in hacking: the ability to load C structures/enums and use them with IO, free space management, images codecs, MIDI, etc. Maybe it would make sense to start from there?

    TLDR; Use GUI because noobs. I already started on something like this - let's get a repo going.
     

    Akiba

    [img]http://i.imgur.com/o3RYT4v.png[/img]
    4,262
    Posts
    13
    Years
  • Lol.

    -I've seen more than enough WET code recently.

    -A REPL isn't necessary, but it wouldn't be too hard to get in to. Like I said, I'm planning on doing a GUI for it in Electron.

    -Even if we don't do a GUI, we're not really missing out on much. For maps, it's always possible to write some pixbuf to an instantiated Window. We could also use blessed for a curses style UI.

    -UI stuff is trivial when done correctly.

    -I doubt functional or async/promise stuff will be that necessary, since it never really was for this kind of thing.

    -ASM shouldn't be too hard to support.

    -10 programmers is always better than 9.
     

    Touched

    Resident ASMAGICIAN
    625
    Posts
    9
    Years
    • Age 122
    • Seen Feb 1, 2018
    -ASM shouldn't be too hard to support.

    Depends what you mean by that. Something replicating IDA isn't exactly possible unless we use radare or something. Even then it doesn't have the decompiler IDA has.
     

    Blah

    Free supporter
    1,924
    Posts
    11
    Years
  • I really liked the documentation project you started up previously. From what I can tell, your reason for stopping was because it didn't reduce the barrier to entry? I disagree. The purpose of it should not have been to make it easier for new hackers to enter the scene, but rather to organise our knowledge in ROM hacking. Making an easy to access, browse and edit database of ROM hacking information and documentation on ROMs is an important thing which this community lacks. These days, if you wanted to try find something like how to disable Gym badges on the trainer card, where would you go? Your best bet is the search through the Quick research thread, and if it's not there, then you search through the Quick Questions thread, and if it's not there you ask it somewhere and hope someone has it documented for you and if that doesn't work you have to look for it yourself (and if you're not an ASM hacker, you won't find it). This kind of limitation stops the hacker from implementing cool ideas in their hack. I would think a database like this would have a section on the trainer card, and going from there is much easier.

    Your new project, if I can understand properly, is to make some sort of tool which makes other tools. This in your eyes reduces the need to develop GUI and introduces abstraction to simplifies the process. However, I'm in agreement with Touched's point. There are many many people on here who have no further clue how to use a computer besides double clicking the application on their Desktop. Introducing a programming language and asking them to develop something is too much. I understand that the process has been heavily abstracted, but there's a problem here which still remains.

    "Under what conditions have my solution made itself easier than manually inserting the bytes to the ROM?"
    For this running inside case, it hasn't been simplified at all. I can manually do that in a hex editor in 3 seconds.

    The case in which it's better to have a program do it for me is if I have to do it repeatedly. This is why I disagree with most tool functions created these days. "Check out my new tool! It changes 2 bytes in the ROM!" vs "I made a tool which inserts ASM routines given hook locations". The latter is a good tool, the former is a for-noobs-who-don't-know-how-to-use-a-hex-editor-tool. There's a notion of re-usability and dynamic functionality. Before I go off topic, if I were to make this ASM insertion tool with your platform I'd have to introduce variables, file reading, free space finder, and if I was good, the ability to run a thumb compiler in some directory. Of these things, only so much can be abstracted. I would argue the process hasn't been simplified. Writing a python program from scratch to do this and running through the terminal would be equal difficulty. The only advantage from your system, is hopefully you already have ready-to-use libraries for things like "Free space finder" and "Copy/Paste hex string" and that is nullified because the programming community already probably made these libraries :)

    At least this is my view point on the matter. I'd be happy to hear you rebuttal what I've said. I don't know your tool, you know your tool, so its quite possible I've misunderstood.
     

    Alexander Nicholi

    what do you know about computing?
    5,500
    Posts
    14
    Years
  • While I'm not saying we motion to hide the command-line from illiterate users, for this to take off we have to take advantage of the automation JS provides to us and create tools your above average hacker can stand to use. Everything cannot be in some command-line window, you can't hack like that comfortably. It's like a bed of stone for a lot of people, so we most certainly need to apply a lot of effort towards graphics in this thing.
     

    Akiba

    [img]http://i.imgur.com/o3RYT4v.png[/img]
    4,262
    Posts
    13
    Years
  • I think y'all have misunderstood what I'm trying to do here. It's not a tool to create more tools, nor is it an entirely new programming language.

    It's simply a platform/design pattern, one of whose benefits is that it allows people who are making hacking tools to not worry about things like creating UIs. You're already using a programming language in the first place. If you're making a tool, you don't necessarily have to learn all the GUI stuff, since that's not what you should be worrying about.

    I also don't see why an API wouldn't work. The only other way would be to couple UI code with functionality, which we've already established is a terrible idea.

    Vantage could be used as a server for a visual application, or the application could just interface with the API directly. Since we need an API anyways, there's no point in avoiding it.
     

    Alexander Nicholi

    what do you know about computing?
    5,500
    Posts
    14
    Years
  • I think y'all have misunderstood what I'm trying to do here. It's not a tool to create more tools, nor is it an entirely new programming language.

    It's simply a platform/design pattern, one of whose benefits is that it allows people who are making hacking tools to not worry about things like creating UIs. You're already using a programming language in the first place. If you're making a tool, you don't necessarily have to learn all the GUI stuff, since that's not what you should be worrying about.

    I also don't see why an API wouldn't work. The only other way would be to couple UI code with functionality, which we've already established is a terrible idea.

    Vantage could be used as a server for a visual application, or the application could just interface with the API directly. Since we need an API anyways, there's no point in avoiding it.
    When I initially stated my support for this, that's what I thought it was. "Oh, we have this idea: Let's create a totally modular framework and build off of it together." It was in my head that script editors, map editors, and more could also be built on top of this framework and distributed as npm packages in the same way tools are now on PC or wherever else. Is that what you intend out of this?
     

    Akiba

    [img]http://i.imgur.com/o3RYT4v.png[/img]
    4,262
    Posts
    13
    Years
  • Yeah, that's basically it, but in a way that inverts control.

    Instead of writing a framework, we write a platform, in the same way that jQuery plugins can be written but do not run by themselves; they are applied to the jQuery prototype and thus express jQuery as an implicit dependency.

    For all the tool developers, that means instead of writing an application, you write a "plugin." Now, I don't really like the word plugin, since plugins are for frameworks and applications with core functionality. I'd prefer to use the word "Behavior," since the sole purpose of our platform is to bind together such behaviors.

    It would allow the end user (hacker) to specify what behaviors they'd need, and it would go and `npm install` them all. These would include map editing behaviors, free space finders, etc. Tool devs could also ship a GUI "Component" that pairs with the Behavior. We'd then place the Component on our GUI. This couples both to the platform, not to each other.
     
    Back
    Top