Development: Standardization of PokéScript

Introduction

This standard was put together by A. Nicholi, cosarara97, Jambo51, Shiny Quagsire, and mewthree9000 on Sunday, the Fourteenth of June, 2015, and was set to be implemented in A. Nicholi's compiler Lapis, cosarara97's Red Alien, Shiny Quagsire's Script Editor Advance (SEA), and finally mewthree9000's PokéTronic; Jambo51 also provides influence for support of his ROM base project as well. It is our hope that others may follow this de facto standard, and that it be updated under the same lineage by appropriate authorities later in time as necessary. For that reason, its contents have been released under the GNU Free Documentation License.

Because of how large the Pokémon ROM hacking community was and the fact that it lacked any standards for scripting whatsoever, it was deemed necessary by the authors that there be designed a standard for the PokéScript assembly language, so as to unify developers' efforts and maintain a common denominator for compatibility amongst applications that used it. The authors thought it just to also invite others to derive from it, only because they were the ones writing applications that would utilize the standard at the time. Initially, A. Nicholi was the standard's sole writer, and the four others were specification consuls.

Why here?

I decided to publish this here and on the Cave of Origin so as to further the goal of this standard at large; I'm putting effort forth to unify the endeavours our community's developers make and put us all on the same sheet of music, for the betterance of our projects and products, and I'm doing that by putting it here in addition to other places of relevance/importance.

Why this?

If you're working with PokéScript it may be a wise idea to follow this standard – two new assemblers for the language are in development now, and it isn't very different at all from what XSE and PKSV-UI use. The standard will provide full compatibility with those two programs, actually.

Files

Download R6 as LaTeX or as a PDF
Download R5 as HTML or as a PDF
Download R4 as Markdown or as a PDF
Download R3 as Markdown or as a PDF
Download R2 as Markdown or as a PDF
Download R1 as Markdown or as a PDF

Out of curiousity, why? Why work to standardise PokeScript when XSE is so widely used and accepted. Although there may be a few undocumented scripting commands there (or areas that just aren't clarified enough) what would this actually add to that? PokeScript hasn't been commonly used in years now, why resuscitate it just to copy things over from existing, useful scripting tools?

Or am I completely missing the point here?

Mana said:

Out of curiousity, why? Why work to standardise PokeScript when XSE is so widely used and accepted. Although there may be a few undocumented scripting commands there (or areas that just aren't clarified enough) what would this actually add to that? PokeScript hasn't been commonly used in years now, why resuscitate it just to copy things over from existing, useful scripting tools?

Or am I completely missing the point here?

Click to expand...

People are finally realising that XSE is old and outdated. We need an alternative that is open source and maintainable. Since there are now a few competing editors trying to succeed it, a standard might be a good thing is people actually stick to it; massive fragmentation might become an issue. However, if only one tool remains supreme I doubt this will be much of an issue. This standard is an effort to remain faithful to the original XSE syntax and stuff while updating tools.

Also, I don't think he means PokeScript as in the tool. He just is using it as a name for the scripting language. IMO it's a poor name, as there are several scripting "languages" in the game. OWScript, worldscript or even mapscript might be better.

Touched said:

We need an alternative that is open source and maintainable.

Click to expand...

XSE is open source.

Spherical Ice said:

XSE is open source.

Click to expand...

So is PKSV, and you know what? They're both a mess. It's far more reasonable to redesign something of this low a magnitude than salvage old sources, and that's what cosarara and I are doing. The standard is, quite plainly, so we're all working together as opposed to separately, which is inherently better.

Try #define FACE and XSE'll throw a type mismatch with the faceplayer command. ;)

Mana said:

Out of curiousity, why? Why work to standardise PokeScript when XSE is so widely used and accepted. Although there may be a few undocumented scripting commands there (or areas that just aren't clarified enough) what would this actually add to that? PokeScript hasn't been commonly used in years now, why resuscitate it just to copy things over from existing, useful scripting tools?

Or am I completely missing the point here?

Click to expand...

Nick said:

Why here?

I decided to publish this here and on the Cave of Origin so as to further the goal of this standard at large; I'm putting effort forth to unify the endeavours our community's developers make and put us all on the same sheet of music, for the betterance of our projects and products, and I'm doing that by putting it here in addition to other places of relevance/importance.

If you're working with PokéScript it may be a wise idea to follow this standard – two new assemblers for the language are in development now, and it isn't very different at all from what XSE and PKSV-UI use. Scripts made for those programs are actually compatible with this standard, purposefully of course.

Click to expand...

Spherical Ice said:

XSE is open source.

Click to expand...

I said open source AND maintainable. Who writes in VB6 nowadays?

I didn't ask why did you make a thread here. Come on now.

As to the 'same sheet of music' comment, the majority of hackers (at least, in this community) use XSE. They're already on the same sheet. Honestly, it just sounds like work for the sake of work - which is fine, you can do whatever you want - I don't see how this will be helpful to the future of hacking.

I don't care what happens with this editor as long as I can keep using XSE's syntax (which it looks like I can). A super extra awesome bonus would be to include something like (I think it's) PKSV's movement planner feature. It makes writing movement scripts so much more intuitive, and would save me a bunch of time when writing them. As it is now, I'm constantly referring back to the movement #raw list and my map to determine, "Okay, so I want this person to run left (looks at list for correct movement) how many spaces (looks at map to count tiles)? Now it needs to walk two steps up and right (looks at list again)." PKSV's quality of life features were great. The syntax wasn't.

EDIT: A toggleable auto comment feature would also be good for noobs reading decompiled scripts, or people asking for help with them. For example:

These auto comments should be editable, so the user can switch the language or wording used to suit their own tastes.

The ability to automatically convert movement #raws from one version to another would be nice too, so that you could port those scripts easily from Firered to Emerald and vice versa.

A standard for pokéscript, or whatever you want to call the scripting language used to control events in third generation games of Pokémon is the first step into better tools and standardization (which, at least in reference to coding, is a good thing)

However I would like to make a few suggestions. One thing that always kind of annoyed me is the fact, that there is only one method directive "#org" which is then also used for text and "other" data. In the message command (which does not exist btw.) a pointer is loaded using "loadpointer" and then a standard script is executed to display the message (displaymessage, waitmessage, closeonkeypress etc. ) - If I have another data structure, a "custom" command (ASM routine etc.) that uses another data structure as parameter, I am forced to declare that one as a text or movement. In XSE the decompiler links the "message" command to text and the applymovement command to movement. If I do something like


my pointer, which is obviously trainer data is retranslated to text on decompilation.
Therefore, there should be other directives, like "#text", "#something" or even "#asm", they all should be kind of customizable (In the attempt I made once to write a code editor I build a class based plugin system, because an asm method cannot simply be described in an XML/INI file.

As for the rest I think thats all pretty much the same in XSE nowadays so switching to a new tool would not be too difficult for users.

~SBird

I'm cool with anything as long as it's format is the same with XSE not that I matter xD
Plus if it adds easier implementation of movements then switching with it would be very nice.. add a live error checker and your golden! :D

SBird said:

A standard for pokéscript, or whatever you want to call the scripting language used to control events in third generation games of Pokémon is the first step into better tools and standardization (which, at least in reference to coding, is a good thing)

However I would like to make a few suggestions. One thing that always kind of annoyed me is the fact, that there is only one method directive "#org" which is then also used for text and "other" data. In the message command (which does not exist btw.) a pointer is loaded using "loadpointer" and then a standard script is executed to display the message (displaymessage, waitmessage, closeonkeypress etc. ) - If I have another data structure, a "custom" command (ASM routine etc.) that uses another data structure as parameter, I am forced to declare that one as a text or movement. In XSE the decompiler links the "message" command to text and the applymovement command to movement. If I do something like

Code:

...
loadpointer @trainerdata
callasm @SPECIAL_BATTLE
...

my pointer, which is obviously trainer data is retranslated to text on decompilation.
Therefore, there should be other directives, like "#text", "#something" or even "#asm", they all should be kind of customizable (In the attempt I made once to write a code editor I build a class based plugin system, because an asm method cannot simply be described in an XML/INI file.

Click to expand...

This is a really good point. I'm going to go over it with my colleagues and get back to you.

Mana said:

I didn't ask why did you make a thread here. Come on now.

As to the 'same sheet of music' comment, the majority of hackers (at least, in this community) use XSE. They're already on the same sheet. Honestly, it just sounds like work for the sake of work - which is fine, you can do whatever you want - I don't see how this will be helpful to the future of hacking.

Click to expand...

Thinking about this, it seems that you're trying to say what XSE provides is good enough for now, and that we're wasting our time making a standard. What you don't get is that this is specification we're collaborating on and I'm writing up is designed to be uniform across tool boundaries – XSE is not a standard, and HackMew basically wrote up the semantics however he felt to with no concern for other tools. We're going above that and making something that is beyond our programs; we're emulating what a lot of the major league open source developers do, and are basically getting our shit together in the respect of scripting. :p

SBird said:

A standard for pokéscript, or whatever you want to call the scripting language used to control events in third generation games of Pokémon is the first step into better tools and standardization (which, at least in reference to coding, is a good thing)

However I would like to make a few suggestions. One thing that always kind of annoyed me is the fact, that there is only one method directive "#org" which is then also used for text and "other" data. In the message command (which does not exist btw.) a pointer is loaded using "loadpointer" and then a standard script is executed to display the message (displaymessage, waitmessage, closeonkeypress etc. ) - If I have another data structure, a "custom" command (ASM routine etc.) that uses another data structure as parameter, I am forced to declare that one as a text or movement. In XSE the decompiler links the "message" command to text and the applymovement command to movement. If I do something like

Code:

...
loadpointer @trainerdata
callasm @SPECIAL_BATTLE
...

my pointer, which is obviously trainer data is retranslated to text on decompilation.
Therefore, there should be other directives, like "#text", "#something" or even "#asm", they all should be kind of customizable (In the attempt I made once to write a code editor I build a class based plugin system, because an asm method cannot simply be described in an XML/INI file.

As for the rest I think thats all pretty much the same in XSE nowadays so switching to a new tool would not be too difficult for users.

~SBird

Click to expand...

When you decompile, the only information you have is what's in the ROM. How would the decompiler know your original source had an #asm in there?

EDIT: btw my tool has been around for a while.

cosarara97 said:

When you decompile, the only information you have is what's in the ROM. How would the decompiler know your original source had an #asm in there?

EDIT: btw my tool has been around for a while.

Click to expand...

Well from the command that called it, i.e. callasm (Yeah, problem for custom tags or the generic loadpointer, but in that case you might eighter use the general string / raw binary or ask the user if you have an UI. Definitely better than decompiling everything into a string. (If your encoding is not cautious enough you might even have problems with compiling, though a well enough designed encoding should use escape characters for fallback anyway)

~SBird

SBird said:

Well from the command that called it, i.e. callasm (Yeah, problem for custom tags or the generic loadpointer, but in that case you might eighter use the general string / raw binary or ask the user if you have an UI. Definitely better than decompiling everything into a string. (If your encoding is not cautious enough you might even have problems with compiling, though a well enough designed encoding should use escape characters for fallback anyway)

~SBird

Click to expand...

Yeah, we were talking, and we think some interactive disassembly would be a good fit for what you're talking about. Basically, there'd be a radio button group or something where you can choose data types or whathaveyou, because data types cannot be communicated about by the ROM, especially with ambiguous commands like loadpointer. :P

Team Fail said:

I don't know if you got it, but I did mention one feature I'd like to see in this editor would be PROPER Japanese character support. When I was decompiling the Mystery Gift script in BPRJ using XSE, the characters were shown in their English sound equivalents, ie: fushigi instead of ふしぎ (Which translates as mystery,) for instance. If that could be done for this editor, that would be simply amazing. I know this may not benefit English hackers much, but Japanese hacks are a thing and having a script editor that supports different languages (Perhaps in this case extend it for European support as well, such as the awkward characters used in German and French games) without any issues would definitely unify the community in a way.

Click to expand...

All of you guys' suggestions are really great for implementations, thank you for sharing them! I'm sure cosarara appreciates the guidance on where we should go with our tools, and I do for certain. :)

Speaking of the thing, I've added a second revision of the standard. There's more complete directive coverage, a standard header, and it's formatted much more similarly to LaTeX with the MS PMincho font.

Revision 3 has been published, and brings forth the #legacy and #thumb directives, differentiation between modern and legacy PokéScript and their semantics, allowances, etc., and lastly the command structures and their typings as well. Added as well were macro functions while (allowing looping) and text (allowing inline dynamic text).


What I would love to hear about from you, the users, is what directives and macro keywords you'd think would be a good idea to add, what your thoughts are on the divide between modern and legacy PokéScript with #legacy, and anything else you feel like sharing about the project. Thank you!

I'll admit that I just skimmed over it, so maybe I missed it, but I don't believe there's any kind of discussion on the supported number formats.

It would probably be good to specifically discuss them, as you do with strings - decimal integer literals, hex literals (both lower and upper-case) prefaced with 0x. And, of course, any other literals you feel useful / necessary, such as perhaps binary literals prefaced with 0b.

Revision Four is posted, guys. With this update comes the definitions for #freespace, #reserve (with an addition onto the XSE syntax), a specification for cascading preprocessor definitions (#defines having the values of other #defines by name), and also a specification for the macro functions XSE implements with things such as msgbox and giveitem.

In other news, I've finally got to start talking to two other scripting developers, namely Shiny Quagsire and also mewthree9000 (yes, as in the creator of Chaos Black). It's my goal that we can all come to a group consensus for what we'd like to see in this language and get something we're all relatively happy to implement in our tools. Collaboration is big with this, despite the apparence of me fronting the thing. :)

EDIT: I forgot to mention it, but the numerical specification Sagiri was talking about got added too. \o/

Did you consider that thumb code can switch to arm code?

Team Fail said:

It has absolutely nothing to do with this new standard whatsoever.

Click to expand...

#thumb
Syntax:
#org <offset>
#thumb
When the #thumb directive immediately follows an #org directive, the following bytecode will be assembled or disassembled as THUMB assembly code instead of PokéScript bytecode or other data.

Click to expand...

I could use a callasm command to run arm code or mixed thumb code.