Tool: Patch Builder

Full Metal · Feb 2, 2014

Patch Builder

What is it?

Patch Builder is a completely new methodology to hacking that focuses on files and data structures rather than offsets, pointers, and bytes. In other words, when you're mapping, you save a map file -- not your ROM file. Later, this tool takes that map file, finds free space in a ROM, turns the map file into appropriate bytes, and then saves an IPS patch for it. This tool is designed to run with Python 3.3, but is 2.7 compatible ( to my knowledge ). You can get Python at the official site.

Why Use It?

As it stands, ROM Hacking involves a certain level of maintaining knowledge of where things are inserted, tracking changes, dealing with various bugs in tools, etc. This tool circumvents this by keeping the tools separate from the ROM. An other advantage to using this methodology is the ability to collaborate as a team on the same ROM Hack Project.

How Does It Work?

It's a lot more simple than you might think. First - this program depends on descriptions of in-rom structures. It looks for these in a few places:

A semicolon seperated list of paths in the environment variable "RomStructureSearch".
A subfolder "RomStructureSearch" in the location of the script.
A subfolder "RomStructureSearch" in the current directory.
The current directory.

.

Rom Structure definitions need to be named with the extension ".RomStructure.json". Since this software still has some engineering left to be done, the exact structure is not presently defined or set in stone. I plan to add support for specialization according to each game code. ( You can specify different structure definitions on a "per-game-code" basis, in other words ). These structures will also specify any enumerations ( "bulbasaur" is pokemon 0x00, for example), as well as any constants that need to be determined ( the map bank table, for example ). To notate a field as having a reference, prefix it with an "@" symbol in the fields list ( see the Git folders RomStructureSearch for an up-to-date example of a RomStructure file ). If it points to a table, then you can specify that like so "@[field_saying_how_large_the_table_is]@[field_name]". ( I need to add support for x-terminated tables, I just haven't yet ).

Entity's are what make up a project. Entities are named like so: entity_name.[entity_type].json. So if you made a map, the correct filename would be farmtown.pokemonmap.json. This would indicate to Patch Builder that the entity is named "farmtown" and that it is a "pokemonmap". Like Rom Structure's, you can also specialize on a per-game-code basis. These are the files that editors will work with. To refer to an internal rom structure in a referring field ( "@connection_header" ) then just set this field in the entity data area to be an integer set to the offset of the structure in-game. If you want to refer to an entity file, then you can simply enter "farmtown.connection_header" in the appropriate data field.

You'll notice I keep mentioning "game-code-specialization" -- for good reason! One of the huge benefits to this project is that you work on the meta-level. In other words, a map you make can be inserted to both Ruby, and Saphire, or even Fire Red. If you want to make two versions of your game ( like Ruby and Saphire ) then you can do that, too!

As it stands, there's not any existing tools for this type of hacking. But lo! Being as these are JSON documents, editing them manually is simple, and intuitive ( when compared to using a hex editor ).

Where Can I Find Out More?

This project is currently hosted on Github at this repository. Fee free to fork and pull away!

You can also clone it using Github For Windows or from this .git url using your console.

If you don't have git and don't feel like installing it, you can also download the latest revision at this url.

Who All Is Involved?

So far, this has mostly been a one-man-show. However, this is a big show and I'm looking for some help. Whether it's Rom Structure ideas, Rom Structure information, Table offsets, or anything, I'm interested in hearing what you have to contribute. If you're interested in helping, you can use Git, or Skype me through my signature.

Karate Kid - LZ77 Algorithms
Full Metal - Main Coder

Shiny Quagsire · Feb 3, 2014

Interesting idea. It seems like a bit of a reverse disassembly but with data only. Definately like the idea of it, and I'm actually interested in seeing this take off because this might be a good feature to add to MEH, that is, adding support to export maps to this specific patch format. And let me tell you, I can give a lot of into on the map format. Like, a lot. So if we get this thing going I could see this being a big thing in ROM Hacking. And it's nice to see that it's written in Python so that other users on different OS's can benefit as well. :)

Full Metal · Feb 3, 2014

Shiny Quagsire said:
Interesting idea. It seems like a bit of a reverse disassembly but with data only. Definately like the idea of it, and I'm actually interested in seeing this take off because this might be a good feature to add to MEH, that is, adding support to export maps to this specific patch format. And let me tell you, I can give a lot of into on the map format. Like, a lot. So if we get this thing going I could see this being a big thing in ROM Hacking. And it's nice to see that it's written in Python so that other users on different OS's can benefit as well. :)

> Bit of a reverse disassembly but with data only

Basically. Although, adding support for assembly wouldn't be outrageous or difficult, either. If I can ever make a proper EBNF parser, I'll probably design a specialized assembly language for use. This whole project very vaguely reminds me of this xkcd comic:

Seeing as we don't even need to have a ROM to do it anymore, theoretically.

And if you have information on map structure's, I would definitely appreciate it! Collaboration on our parts would definitely be a good idea, I think. However, if you really want this to work smoothly with your existing editor, you'll want to design it so that it primarily works with json documents and external images rather than from a ROM, but so that it asks for a ROM should it need one. Writing a hybrid tool could get complicated and confusing fairly quickly.

And yeah, Python is great. 8)

esperance · Feb 4, 2014

Well it certainly does sound cool.
In a way, it reminds me of how DS ROMs are already structured, with their internal file system and all. This way, GBA hacking would be similar to how DS hacking is done. Cool!

Full Metal · Feb 4, 2014

itari said:
Well it certainly does sound cool.
In a way, it reminds me of how DS ROMs are already structured, with their internal file system and all. This way, GBA hacking would be similar to how DS hacking is done. Cool!

Actually, that's what inspired this idea ages ago. But back then when I had the idea, I didn't have the practical knowledge or skillset required to implement it.

Astraea · Feb 13, 2014

Good idea! It can help many people like me!

Good luck bro! Complete it!

Full Metal · Feb 22, 2014

update:

I've finally determined a data structure that should encompass all the immediate needs I can think of for an Entity object:

Code:

{
  data:{ // default data. Consider this to be a "bottom layer", see comments below.
    field_name: field_value
  },
  specialization:{ // the "top layer" for data. Overrides any data in the root "data" node.
    game_code:{
      field_name: field_value
    }
  },
  meta:{ // not used by the compiler. simply used by the editor for convenience.
    field_name: field_value
  }
}

Then for data structures, my proposed final structure is as follows:

Code:

{
  fields:{
    name:{
     byte_code: struct.compile_code// see specification of struct's bytecode in the python docs.
     order: where_in_structure_should_be_placed
     description: optional_string
     refers_to: null_or_name_of_structure_type
    }
  },
  tables:{
    table_name:{
      index_name: null_or_name_of_field_in_structure // null means use meta information '$index',
      entry_structure: name_of_structure_for_entry // you can also pass in a structure object here instead of a reference string, but it's recommended to use a reference string.
      location: internal_rom_pointer_to_table
    }
  },
  constants:{
    name: value
  }
  specialization:{
    game_code:{
      fields: {},
      tables: {},
      constants: {}
    }
  }
}

There will also be a file used for maintaining a list of known free-space chunks given a game code. I'm undecided on whether this will be stored in a single file with a constant name, or if it will be spread over other files. It's not needed, but if you don't have this, then you do need a ROM so that you can process it for free-space information. Anyways, the structure for an individual game will be this:

Code:

[
  {
    location: internal_rom_pointer,
    size: how many bytes can be overwritten safely,
    value: 0xFF // optional.0xFF is assumed unless specified otherwise. Anything beyond the size of a byte will be interpreted as "any byte".
  }
]

Developers, do you have any questions or feedback for this proposal?

Tool: Patch Builder

More options

Full Metal

C(++) Developer.

Shiny Quagsire

I'm Still Alive, Elsewhere

Full Metal

C(++) Developer.

esperance

Full Metal

C(++) Developer.

Astraea

The Storm of Friendship

Full Metal

C(++) Developer.