• Our software update is now concluded. You will need to reset your password to log in. In order to do this, you will have to click "Log in" in the top right corner and then "Forgot your password?".
  • Staff applications for our PokéCommunity Daily and Social Media team are now open! Interested in joining staff? Then click here for more info!
  • Welcome to PokéCommunity! Register now and join one of the best fan communities on the 'net to talk Pokémon and more! We are not affiliated with The Pokémon Company or Nintendo.
    Register

The ROM Hacking Documentation Project

Akiba

Akiba

[img]http://i.imgur.com/o3RYT4v.png[/img]
    • 4,262
    Posts
    13
    Years
    rhdp: The ROM Hacking Documentation Project

    Introduction

    So, it's been almost a year and a half since I've hacked seriously. Most of you probably don't remember me, save a few old friends (shoutout to GoGo, shiny, Jambo, Team Fail, and karatekid). That being said, I've always yearned for a stable, definitive resource for ROM hacking information. I'm sure that many of you have come to realize that a bulletin board system isn't the best format for organizing in-depth research and development. I've been visualizing a web platform in which all that we know and ever will know about ROM hacking can be documented and well-organized. It's a grand undertaking, and may take months or years to become something really useful, but a step like this must eventually be taken if we are to wish for more productive hacking and tooling. I will consistently update this thread as more information becomes available and our goal becomes ever more clear.

    Context

    The community has created some great tools. However, those that were created earlier on (which we happen to rely on very often) employ many secret or obscure techniques, and are closed source. As a result, the creator of any given tool is the sole liability of his userbase. Bugfixes, optimizations, improvements, and implementations all depend on the creator, with few to no supporters critically eyeing the source. Furthermore, once the creator inevitably decides to leave the community, his project is halted indefinitely, and no further changes can be made. This is one characteristic of the mass fragmentation within the community. Tools developed cannot be integrated or improved, and any attempt at creating a general and sustainable development environment cannot succeed. Notable attempts include PGE and G3HS, but for many tasks, we rely on tools with no alternatives. Such tools are often simply standalone, and the furthest we can go to integrate them is to loosely string them together with embedded command line calls. One major disadvantage is that ROMs are loaded locally into the memory of a tool. The ROM can only be modified one task at a time, and should you require information of that ROM to be open in another tool, the entirety of it must be loaded into memory, resulting in redundant memory and processor usage. Another consequence of such tools not being open source is that they cannot be ported to other operating systems. Avid hackers such as shinyquagsire and myself use Linux as our operating system of choice. As some of you know, we work on a prospective successor to AdvanceMap -- the MEH: Map Editor of Happiness. This was one of the first steps in moving our ROM hacking community into the field of open source and progressive development. However, I feel that even MEH is deficient in many ways. For example, MEH depends on Java. While that may bring many advantages, I cannot see a tooling ecosystem that relies on Java becoming the future of the hacking community. This brings me to another point - the development processes for many our tools differs vastly. MEH is being created in Java. G3HS uses Python. PGE uses Visual Basic. AdvanceMap was created in some old Turbo language that I'm not even sure is supported anymore. The interactivity and parallelism between tools is remarkably low. A much better system would consist of all components maintained by multiple people, implemented mostly in the same language, and using a wide and well written subsystem. A single codebase, with libraries and modules contributed by multiple developers with expertise (creators of previous tools made for the same task). This way, ROM hacking can be done with more productivity, and by using tools with defined behavior, not risk corrupting a ROM or possibly even something worse. The fragmentation within the community extends past the tooling environment. New and old hackers alike often find themselves asking questions that could be answered by consulting a resource. There are some resources in the forums, but no definitive single resource, and sadly, threads suffer the same problem that closed source applications do. karatekid's introduction to Assembly is a great example of where improvement could be made. Instead of having a resource describing what Assembly is and how it works, then linking to several other threads (possibly containing links as well), all of the information could be combined into a single resource, and missing information can be supplemented and statements corroborated. Such a resource would contain an introduction, then specific and formal information, with each section followed by a succinct digest and one or more examples. One exemplary documentation base that I often both point to and utilize is the CanJS (a Javascript framework that I like to use) API documentation. There is a formal definition of each component of the framework, followed by explanation and usage scenarios, and concluded with live demos using some framework code. A similar comprehensive documentation system for the Pokemon ROM would be phenomenal in its extreme usefulness.

    tl;dr We are terribly disorganized and need to do something quick

    Purpose

    I am creating this documentation project for the purpose of aiding ROM hackers in their undertakings, as well as ROM researchers and hacking tool developers. I hope that this project will unify the community and bring about higher-level collaborative projects such as a ROM hacking IDE.

    Roadmap

    The project will be a dynamic site served by a to-be-determined server, and with code hosted on an already determined Github repository. Once the platform has reached a minimum viable product, it will be deemed ready for public use. At that point, most of the effort given to the project will be made by the community. Information from many threads previously written on the ROM Hacking Forum, as well as knizz's ROM decompilation will be organized and used as content. Five to fifteen content moderators will be selected or nominated, to review content submissions to the site's open submission system. They may also be responsible for updating the documentation content during early stages. Frequent contributors will be given privileges to more freely edit the content. Once the project reaches an agreeable level of maturity, I plan to initiate several projects that utilize the documentation. These include the ROM hacking IDE mentioned in the Purpose section of this manifest.

    Design

    We're currently working this out, but are more than happy to take suggestions. Drop one by at the IRC channel or on this thread. We've decided on a Single Page Application -- one where the page is loaded once and any further information is grabbed without impeding the user's experience. We've also decided on an Open Submission System, where edits and changes can be made by anyone. The approval process for a change can follow two paths: approval by moderator, or surpassing a number found through rough calculation of the ratio between amount of content in a change and number of upvotes for that change.

    Core Contributors

    Synchronous - Lead Platform Developer, Project Manager
    Team Fail - Secondary Project Manager
    knizz - Lead Researcher
    GoGoJJTech - Content Moderator, Researcher
    itari - Content Moderator
    AlexAhnon - Content Moderator
    Elsa - Content Moderator, Researcher
    Sisyphean - Resource Compiler

    Currently looking for volunteers.

    Communication

    Discussion relevant to rhdp can take place in this thread, in rhdp's IRC channel on Laz's website, or on the official discussion boards.
    IRC: https://chat.linkandzelda.com:9090/?channels=GoGo,rhdp
    BBS: https://systemic.io:8080/

    Be aware that the forum technology that we use is still under heavy development, so there will be some occasional issues.

    Site

    The project is still in its early stages, and volunteers for the project are welcome. Fork us and submit a pull request -- we'll need all the help we can get.

    Live Development: https://www.systemic.io/
    Incremental Release: https://rhdp.github.io/
    Git repository: https://github.com/rhdp/rhdp.git

    tl;dr We are terribly disorganized and need to do something quick
     
    Last edited:
    This seems promising. If it takes off and people start contributing to it, it'll be a wonderful reference resource that everyone can use to their disposal. Hopefully you'll be able to put up some sort of "demo" that everyone can take a look at and see how it'll function.
     
    Team Fail is writing a clarification post to explain some of the ideas that we've discussed on IRC.

    This is also actually a completely and utterly pointless thread bump that I made out of excitement
     
    Now, for some technical specifications of the project. (Rough outline, will be updated)

    Domain (pro tempore):
    https://rhdp.github.io/
    We'll buy the domain rhdp.io if this project is backed by high community participation (or donations).

    PaaS/Host:
    Heroku

    Github Repository:
    https://github.com/rhdp/rhdp

    Platform:
    Node.js
    MongoDB
    Express
    Socket.IO
    Stylus
    Jade
    CanJS
    jQuery

    Relevant Languages:
    Javascript/JSON
    Jade (HTML)
    Stylus (CSS)

    Paradigm:
    Single Page Application (SPA)
    Open Submission System

    Coding Style (for developers):
    4-spaced hard tab
    Single-quoted strings

    Javascript Sample
    Code: 
    var a = 1,
	b = {
		a: 'Hello',
		b: true
	},
	// synchronous
	fnSync = function(arg) {
		var result;
		// do something
		return result;
	},
	// asynchronous
	fn = function(arg, next) {
		var result;
		// do something
		next(err, result);
	};

doAsynchronousTask({
	size: 40
	encoding: 'utf-8'
}, function(err, data) {
	// do stuff with data
});
     
    Last edited:
    Well there isn't much I can contrubute right now(no stable home WiFi coupled with financial instability). But I can help by porting all of my tutorials and guides to the ROM Hack DB, if wanted ^_^
     
    DJTiKi said:
    Well there isn't much I can contrubute right now(no stable home WiFi coupled with financial instability). But I can help by porting all of my tutorials and guides to the ROM Hack DB, if wanted ^_^
    Click to expand...

    Definitely. We'd love to have as many contributors to this project as possible. Anybody with expertise our experience is more than welcome. Send me a message if you want to have write privileges or if you want to contribute in any other way.
     
    Last edited:
    I have talked to both Team Fail and Synchronous and I can tell you that this is a great idea and will (hopefully) revolutionize the hacking community forever!
     
    Last edited:
    Progress Report:

    An initial protoype has been developed and can be viewed at the incremental release.

    Sample content was copied from Wikipedia and placed on the server's database polyfill. We used special technologies to send this data from the server to the client with a short initial page load and negligible further latency.

    A screenshot:
    Spoiler:


    We've also fixed the bug that crashed the discussion forum last night.
     
    Last edited:
    Yeah, we'll probably implement subtopics and submissions by the next progress report.
     
    Team Fail said:
    Is there plans to add various formatting tags, as well as bars we can place on pages (For example, pages that need to be tagged as WIP, NSFW (Doubt that'll happen, but...), etc?
    Click to expand...

    I doubt there will be much NSFW content, and research is, of course, forever a WIP. Although we might add tags for different purposes.
     
    Team Fail said:
    Well, if we plan to target all sorts of game genres, there will certainly be some NSFW content. I'd list some examples, but...
    Click to expand...

    Ah, yeah. I understand. We'll probably add ESRB rating tags for each specific game.
     
    So as I have been combing through various threads finding data about ROM hacking, and I've come to the realization that a simple:
    Spoiler:

    method won't work as well. While I like the simplicity of a branched system, I feel like there is perhaps a better solution. It makes sense to organize the data like:
    Spoiler:

    But this leaves the Move Animations information only in the Graphics section of the site when it should also be in the Attacks section or in the Battle Effects section (if we choose to delineate it in such a way). We could always store the information in multiple places though which would work but it would increase the clutter as well. I feel like the solution is assigning each relevant topic a series of keywords and we could cross-reference between documents. This would make for an easier search feature as well.
    Just an idea as I've found it difficult to classify data.
    That being said, this is some condensed information that I have found and would classify as Battle Graphics (excluding sprites). I mainly copied and truncated tutorials and threads and then added in relevant additions made in the comment section.

    Replacing Old Battle Backgrounds:
    Spoiler:


    Expanding Number of Battle Backgrounds
    Spoiler:


    Creating Custom Animation Particles:
    Spoiler:


    Creating New Attack Backgrounds:
    Spoiler:


    Manipulating Attack Animations:
    Spoiler:


    To call this truncated might seem a little silly considering the length of each topic, but a lot has been removed. This is a link to the thread that contains the patch talked about in "Creating Custom Animation Particles": http :// www pokecommunity com/showthread php?t=302401(can't post full links).
     
    Sisyphean said:
    So as I have been combing through various threads finding data about ROM hacking, and I've come to the realization that a simple:
    Spoiler:

    method won't work as well. While I like the simplicity of a branched system, I feel like there is perhaps a better solution. It makes sense to organize the data like:
    Spoiler:

    But this leaves the Move Animations information only in the Graphics section of the site when it should also be in the Attacks section or in the Battle Effects section (if we choose to delineate it in such a way). We could always store the information in multiple places though which would work but it would increase the clutter as well. I feel like the solution is assigning each relevant topic a series of keywords and we could cross-reference between documents. This would make for an easier search feature as well.
    Just an idea as I've found it difficult to classify data.
    That being said, this is some condensed information that I have found and would classify as Battle Graphics (excluding sprites). I mainly copied and truncated tutorials and threads and then added in relevant additions made in the comment section.

    Replacing Old Battle Backgrounds:
    Spoiler:


    Expanding Number of Battle Backgrounds
    Spoiler:


    Creating Custom Animation Particles:
    Spoiler:


    Creating New Attack Backgrounds:
    Spoiler:


    Manipulating Attack Animations:
    Spoiler:


    To call this truncated might seem a little silly considering the length of each topic, but a lot has been removed. This is a link to the thread that contains the patch talked about in "Creating Custom Animation Particles": http :// www pokecommunity com/showthread php?t=302401(can't post full links).
    Click to expand...

    Great work, Sis. Your help is invaluable, and exactly what I was looking for.

    I also agree that we can't use a very flat tree model for containing all of our documentation. We'll have to think more about that.

    Maybe something like this? (example)
    Code: 
    GBA
	Overview
		BIOS
		Architecture
	FireRed
		Overview
			Binary
			Video
			Memory
			Interrupts
			Data Structures
		Graphics
			Overworld
				Sprites
					Frames
				Maps
					Blocks
					Movement Permissions
			Battle
				Move Animations
				Backgrounds
		Mechanics
			Battle
				AI
			Flags
     
    Back Again, I would describe these posts as being Miscellaneous In Battle Hacks I guess.

    Creating New Move Effects
    Spoiler:


    Creating New Moves
    Spoiler:


    Making an XY Inverse Battle:
    Spoiler:


    Trainer AI:
    Spoiler:


    Move Effects List:
    Spoiler:


    Updated Moves List
    Spoiler:
     
    Progress Report:

    Not quite what we promised yesterday, but we made some internal changes and user interface improvements.

    I plan on implementing submissions, database, and subtopics in that order.

    Again, it can be viewed at the incremental release.
     
    Another thing to mention, this project will come with mobile support baked in, so it will be more accessible to everyone.
     
    You must log in or register to reply here.
    Back
    Top