Development: Pokemon - GBA - IDA idb files

[Zilvereyes]

Hi you all,

I had a hope that some coorporation could be formed to to "map" data, pointers, data ect.

My original idea of a google code repository have been put to a rest by the community,
so references to it have now been removed, and I have stepped back and will now wait and see what the community decide to do.

[knizz]

IDB files are compressed binary files. So they aren't really suited to versioning systems. Also, you could run out of space since every commit means a 8-12MB copy of the file.
But even I worked on the db with other people. (Mainly Darthatron.) He just told me about his changes verbally and I applied these changes to my copy and distributed it every month or so. (We have an IRC Channel on irchighway.net #pokehack.) And at least for Firered we could still act as maintainers.

I hope you don't forget to write my name in huge letters on stuff you use that was made by me. (no seriously, proper credits)

[Zilvereyes]

@knizz, as you might have noticed you where the only one mentioned directly in my post, which in an indirect way was giving you credit, but to make it more clear I have now added a credits section to the 1st post, and to the google code SVN.

Anyway I never wanted credit for any of this, all I where looking for was people interested in joining forces to "map" the data, pointers ect. ect. ... personally I'm mapping my own ROM from scratch, due to different reasons, but I'm willing to un-credited share my information with those interested in it, and was hoping that I wasn't the only one, but np if people want credit, I'll not break from typing a little text, but maintaining a credit section might very well take time away from the original intention (a few minutes here and there will in the end make hours and so on).

[Bond697]

why just gba games? here's my white idb, there's quite a lot done:

http://hack.thundaga.com/arm9-white_decompressed_U.idb

[Zilvereyes]

Originally Posted by Bond697

why just gba games? here's my white idb, there's quite a lot done:

http://hack.thundaga.com/arm9-white_decompressed_U.idb

Truth be told, I aimed for GBA's since that what I'm doing atm, but thanks for sharing I'll add that one up in a few hours, RL needs my attention as i write this.

[Full Metal]

Wait, what about Wikipedia? Or a blog? Those would be much more suitable, just saying.

Game -> Category
Offset -> Custom Meta Field and / or tag.
Then of course, add appropriate tags.

One post = one routine.
With a description that links to other routines as references.

Now you see why wikipedia works great for this?

[ mods, if this is against the rules, I do apologize. Please just edit the link if it's against rules here ]

link.

Knizz has even posted a few pages on the website already.

[Zilvereyes]

Originally Posted by Full Metal

Wait, what about Wikipedia? Or a blog? Those would be much more suitable, just saying.

Game -> Category
Offset -> Custom Meta Field and / or tag.
Then of course, add appropriate tags.

One post = one routine.
With a description that links to other routines as references.

Now you see why wikipedia works great for this?

[ mods, if this is against the rules, I do apologize. Please just edit the link if it's against rules here ]

link.

Knizz has even posted a few pages on the website already.

I do get your point!

However if I personally where to use wiki, I would be spending more time due to how I personally work on stuff like this, and I would loose the ability to look at the bigger picture (grand scale ROM), but that is just me.

Anyway, no one is stopping anyone from also putting the info into the wiki.

[Full Metal]

When you upload or otherwise submit content to our Services, you give Google (and those we work with) a worldwide license to use, host, store, reproduce, modify, create derivative works (such as those resulting from translations, adaptations or other changes we make so that your content works better with our Services), communicate, publish, publicly perform, publicly display and distribute such content. The rights you grant in this license are for the limited purpose of operating, promoting, and improving our Services, and to develop new ones. This license continues even if you stop using our Services (for example, for a business listing you have added to Google Maps). Some Services may offer you ways to access and remove content that has been provided to that Service. Also, in some of our Services, there are terms or settings that narrow the scope of our use of the content submitted in those Services. Make sure you have the necessary rights to grant us this license for any content that you submit to our Services.

Also, this tidbit of TOS concerns me. Hacking may be legal ( and even that is questionable ). However, publishing the guts of the inner-workings of the game onto a HUGE code-publishing platform like Google Code? Nintendo dev's might be a little less apt to appreciate that.

[Team Fail]

Originally Posted by Full Metal

Also, this tidbit of TOS concerns me. Hacking may be legal ( and even that is questionable ). However, publishing the guts of the inner-workings of the game onto a HUGE code-publishing platform like Google Code? Nintendo dev's might be a little less apt to appreciate that.

Kinda have a point there. You'd be better off getting free hosting and running your own site with this information on it. Or host a site yourself, although you'd have to pay an arm and a leg for a server and to maintain it.

[Zilvereyes]

@Full Metal & Team Fail, have seen your messages, and to tell the truth I personally knew all that before your posts, but yeah google code is a HUGE code platform, which in my eyes makes it the PERFECT "hiding" place, google employees most likely will not understand what the repository is used for(*), and chances that Nintendo Devs ect. find the repository in the HUGE amount of other repositories are slim, unless alerted by a third party, these are my thoughts, and I in no way intend to alter anyone's mind on the matter, people should chose for them self what to feel and think.

(*) Even if they understand it, they can't even say that it's sharing of the ROM, since the .idb files will most likely never reach a point where they can be used to re-make the actual ROM.

Anyway feel free to contact me if you want access to commit to the repository (like first post says), or if you chose too don't contact me if you don't want access, it's a free choice ....

About hosting, I could host it on some server somewhere and pay a ton for the service, since billing info would need to be provided to the host, that would lead Nintendo straight to my door should they chose to do something ... My google account doesn't need to be supplied with this info, making it quite a bother to track down which door to knock down ? yes/no ?

Anyway do what you feel is the right thing for you, and so shall I ;-)

[Full Metal]

Proof of Concept.

I haven't quite finished the HTML/JS part yet, but here it is: a simple, self-maintaining server. It keeps track of offsets, and ROM's.

To Do:
Fix the duplicating-items problem in HTML.
Add forms for adding items through the web-interface.
Add mode for searching.
Wiki parsing via jQuery.

What's done already:
Web interface ( run 'offset_server.py', then go to this link. ).
Console Interface ( entirely done, for the most part )
You can tag, list, or add items.

For the console interface, run offset_organizer.py.
For the web interface, run offset_server.py.

I tested this with python 3.2, but it will work just as well in python 2.7 with the import from __future__ statement ( I can't remember what all needs to be imported. Probably just print. )

Ideas:
People can contribute by maintaining their own server [ cgi-based, since most web-hosts don't like this type of server ]. You can then add their server to your local-list of servers, and then sync your offsets and roms between servers. Voila! Lots of collaboration!

[ since you can't add things via web interface yet, add a few things in the console interface, then view them in the web interface. It's not pretty yet, but it's functional, except for duplicate items. Enjoy usage of the back button. Click on rows of a table to view further info ].

[Zilvereyes]

Wow!
Nice to see that there are different ideas and seeing initiative!

I'll have a look at it in the weekend, RL will keep me busy until Saturday.

[Full Metal]

Updated Release

Okay, the web front-end is nearly complete.
What's left to do?
○Searching
○Editing
○Deleting
What's done?
○Add ROMS
○Add OFFSETS
○View list of ROMS
○View OFFSETS associated with a ROM
○View details on an offset
○Wiki-markup-style-parsing for OFFSET descriptions. ( will add for ROM descriptions as well, don't worry ).

If anybody wants to make an editor for the wiki syntax, that would be terrific, but I'm not demanding. Otherwise, you'll just need to know how to use the markup I linked you to.

Tomorrow or this weekend I'll begin work on syncing this between a local copy and a server. Let's see how well that goes! ( Searching will take priority over this, of course )

Then I'll work on editing and deleting, as that is a fairly minor thing for me. If you must edit something, stop your server, open 'offsets.json' in a text editor and edit. save. exit the text editor, then re-open your server.

Don't know how to edit JSON?
Open the file in a text editor, then paste it on the left textbox found on this website, edit the stuff on the right, then push the button with a left arrow, copy the formatted text and replace the contents of your original offsets.json file.

[knizz]

Originally Posted by Zilvereyes

Even if they understand it, they can't even say that it's sharing of the ROM, since the .idb files will most likely never reach a point where they can be used to re-make the actual ROM.

In fact the idb contains a full copy of the ROM. ROM-Hacking is illegal anyway. Why still pretend to be in the gray-zone?

[Zilvereyes]

Originally Posted by knizz

In fact the idb contains a full copy of the ROM. ROM-Hacking is illegal anyway. Why still pretend to be in the gray-zone?

I have seen what you said, but note that this doesn't mean that I personally agree with you.

@knizz: I would like to take this opportunity to also ask you if you want me to remove the firered IDA DB from the repository?
Since I have no ownership of the current firered hosted on the repository the correct thing would be to ask, and this have with this post now been done.

@ all
I sense that my repository idea is being rejected in the by me thought out form and I will not be the one standing in the way of progress, so let me make it clear that I'm open for alternate solutions, I don't want to force anything on to someone, so if people can agree on some other form of doing something similar (like Full Metal already have posted) sure I will try my best to contribute, but I'll not be the one taking point, since I'll be dealing with different from my own ways of doing this.

I'll take a step back, and see where others might lead this, and hope I'll be able to contribute in some way.

[Darthatron]

I would prefer if you took it down.

[knizz]

Originally Posted by Zilvereyes

I have seen what you said, but note that this doesn't mean that I personally agree with you.

I'm a bit confused. I stated two facts (1. idb contains a full copy of the rom, 2. rom-hacking is illegal) and then asked a question. There's nothing to agree or disagree with.

Originally Posted by Zilvereyes

I would like to take this opportunity to also ask you if you want me to remove the firered IDA DB from the repository?

Originally Posted by Darthatron

I would prefer if you took it down.

Me too actually.

Please, don't get me wrong. I appreciate the effort. I'm not angry at you or anything but these repositories do not help us.

[Zilvereyes]

Originally Posted by Darthatron

I would prefer if you took it down.

Originally Posted by knizz

I'm a bit confused. I stated two facts (1. idb contains a full copy of the rom, 2. rom-hacking is illegal) and then asked a question. There's nothing to agree or disagree with.



Me too actually.

Please, don't get me wrong. I appreciate the effort. I'm not angry at you or anything but these repositories do not help us.

I have done as asked, and have removed the firered IDA DB by you guys from the repository, and ofc removed credits to you guys too.

I'll edit first post, and remove most of the contense in it, removing any references to the google code repository, there is little reason to share the information here, since it seems unwanted.

[Full Metal]

Originally Posted by Zilvereyes

I'll edit first post, and remove most of the contense in it, removing any references to the google code repository, there is little reason to share the information here, since it seems unwanted.

false. You're just going about it all wrong. If you'll give me a bit more time, I'll get the offset organizer to be more suited and geared towards collaborative efforts.

[Darthatron]

Originally Posted by Zilvereyes

I have done as asked, and have removed the firered IDA DB by you guys from the repository, and ofc removed credits to you guys too.

I'll edit first post, and remove most of the contense in it, removing any references to the google code repository, there is little reason to share the information here, since it seems unwanted.

Don't get us wrong, we do appreciate the effort, it's just that our names are plastered all over that and it contains a full ROM. That makes us 100% liable. Feel free to use code from it (and credit us,) but just don't host the actual .idb file.

[Zilvereyes]

Originally Posted by Full Metal

false. You're just going about it all wrong. If you'll give me a bit more time, I'll get the offset organizer to be more suited and geared towards collaborative efforts.

Think you misunderstood, what I was trying to say was that there no community support on the google code repository, which was why I removed info about it, I do see that there is interest in sharing and working together on the idea, just in another form.

@Darthatron
Thank you, I appreciated that.