The PokéCommunity Forums

The PokéCommunity Forums (https://www.pokecommunity.com/index.php)
-   Binary Hack Research & Development (https://www.pokecommunity.com/forumdisplay.php?f=195)
-   -   Development Standardization of PokéScript (https://www.pokecommunity.com/showthread.php?t=350668)

Alexander Nicholi June 16th, 2015 3:57 PM

Standardization of PokéScript
 
“PokéScript 2”: A Standard

Introduction


This standard was put together by A. Nicholi, cosarara97, Jambo51, Shiny Quagsire, and mewthree9000 on Sunday, the Fourteenth of June, 2015, and was set to be implemented in A. Nicholi’s compiler Lapis, cosarara97’s Red Alien, Shiny Quagsire’s Script Editor Advance (SEA), and finally mewthree9000’s PokéTronic; Jambo51 also provides influence for support of his ROM base project as well. It is our hope that others may follow this de facto standard, and that it be updated under the same lineage by appropriate authorities later in time as necessary. For that reason, its contents have been released under the GNU Free Documentation License.

Because of how large the Pokémon ROM hacking community was and the fact that it lacked any standards for scripting whatsoever, it was deemed necessary by the authors that there be designed a standard for the PokéScript assembly language, so as to unify developers’ efforts and maintain a common denominator for compatibility amongst applications that used it. The authors thought it just to also invite others to derive from it, only because they were the ones writing applications that would utilize the standard at the time. Initially, A. Nicholi was the standard’s sole writer, and the four others were specification consuls.

Why here?


I decided to publish this here and on the Cave of Origin so as to further the goal of this standard at large; I’m putting effort forth to unify the endeavours our community’s developers make and put us all on the same sheet of music, for the betterance of our projects and products, and I’m doing that by putting it here in addition to other places of relevance/importance.

Why this?


If you’re working with PokéScript it may be a wise idea to follow this standard – two new assemblers for the language are in development now, and it isn’t very different at all from what XSE and PKSV-UI use. The standard will provide full compatibility with those two programs, actually.

Files


Download R6 as LaTeX or as a PDF
Download R5 as HTML or as a PDF
Download R4 as Markdown or as a PDF
Download R3 as Markdown or as a PDF
Download R2 as Markdown or as a PDF
Download R1 as Markdown or as a PDF

Mana June 17th, 2015 10:46 AM

Out of curiousity, why? Why work to standardise PokeScript when XSE is so widely used and accepted. Although there may be a few undocumented scripting commands there (or areas that just aren't clarified enough) what would this actually add to that? PokeScript hasn't been commonly used in years now, why resuscitate it just to copy things over from existing, useful scripting tools?

Or am I completely missing the point here?

Touched June 17th, 2015 12:05 PM

Quote:

Originally Posted by Mana (Post 8802538)
Out of curiousity, why? Why work to standardise PokeScript when XSE is so widely used and accepted. Although there may be a few undocumented scripting commands there (or areas that just aren't clarified enough) what would this actually add to that? PokeScript hasn't been commonly used in years now, why resuscitate it just to copy things over from existing, useful scripting tools?

Or am I completely missing the point here?

People are finally realising that XSE is old and outdated. We need an alternative that is open source and maintainable. Since there are now a few competing editors trying to succeed it, a standard might be a good thing is people actually stick to it; massive fragmentation might become an issue. However, if only one tool remains supreme I doubt this will be much of an issue. This standard is an effort to remain faithful to the original XSE syntax and stuff while updating tools.

Also, I don't think he means PokeScript as in the tool. He just is using it as a name for the scripting language. IMO it's a poor name, as there are several scripting "languages" in the game. OWScript, worldscript or even mapscript might be better.

Spherical Ice June 17th, 2015 12:15 PM

Quote:

Originally Posted by Touched (Post 8802664)
We need an alternative that is open source and maintainable.

XSE is open source.

Alexander Nicholi June 17th, 2015 2:15 PM

Quote:

Originally Posted by Spherical Ice (Post 8802674)
XSE is open source.

So is PKSV, and you know what? They're both a mess. It's far more reasonable to redesign something of this low a magnitude than salvage old sources, and that's what cosarara and I are doing. The standard is, quite plainly, so we're all working together as opposed to separately, which is inherently better.

Try #define FACE and XSE'll throw a type mismatch with the faceplayer command. ;)

Quote:

Originally Posted by Mana (Post 8802538)
Out of curiousity, why? Why work to standardise PokeScript when XSE is so widely used and accepted. Although there may be a few undocumented scripting commands there (or areas that just aren't clarified enough) what would this actually add to that? PokeScript hasn't been commonly used in years now, why resuscitate it just to copy things over from existing, useful scripting tools?

Or am I completely missing the point here?

Quote:

Originally Posted by Nick (Post 8801266)

Why here?


I decided to publish this here and on the Cave of Origin so as to further the goal of this standard at large; I'm putting effort forth to unify the endeavours our community's developers make and put us all on the same sheet of music, for the betterance of our projects and products, and I'm doing that by putting it here in addition to other places of relevance/importance.

If you're working with PokéScript it may be a wise idea to follow this standard – two new assemblers for the language are in development now, and it isn't very different at all from what XSE and PKSV-UI use. Scripts made for those programs are actually compatible with this standard, purposefully of course.


Touched June 17th, 2015 2:17 PM

Quote:

Originally Posted by Spherical Ice (Post 8802674)
XSE is open source.

I said open source AND maintainable. Who writes in VB6 nowadays?

Mana June 17th, 2015 10:12 PM

I didn't ask why did you make a thread here. Come on now.

As to the 'same sheet of music' comment, the majority of hackers (at least, in this community) use XSE. They're already on the same sheet. Honestly, it just sounds like work for the sake of work - which is fine, you can do whatever you want - I don't see how this will be helpful to the future of hacking.

Deokishisu June 17th, 2015 10:53 PM

I don't care what happens with this editor as long as I can keep using XSE's syntax (which it looks like I can). A super extra awesome bonus would be to include something like (I think it's) PKSV's movement planner feature. It makes writing movement scripts so much more intuitive, and would save me a bunch of time when writing them. As it is now, I'm constantly referring back to the movement #raw list and my map to determine, "Okay, so I want this person to run left (looks at list for correct movement) how many spaces (looks at map to count tiles)? Now it needs to walk two steps up and right (looks at list again)." PKSV's quality of life features were great. The syntax wasn't.

EDIT: A toggleable auto comment feature would also be good for noobs reading decompiled scripts, or people asking for help with them. For example:

Code:

...
faceplayer 'causes the person event to face the player
msgbox @text 0xA 'opens a PokeNav call (emerald only) textbox with strings at @text
setvar 0x800D 0x1 'Sets LASTRESULT variable to 0x1
giveitem 0x0 0x1 MSG_OBTAIN 'gives 1 Master Ball with the obtain message
warp 0xA 0xB 0xC 0x0 0x0 'warps the player to Map 10.11 on warp number 12
...


These auto comments should be editable, so the user can switch the language or wording used to suit their own tastes.

The ability to automatically convert movement #raws from one version to another would be nice too, so that you could port those scripts easily from Firered to Emerald and vice versa.

SBird June 18th, 2015 6:01 AM

A standard for pokéscript, or whatever you want to call the scripting language used to control events in third generation games of Pokémon is the first step into better tools and standardization (which, at least in reference to coding, is a good thing)

However I would like to make a few suggestions. One thing that always kind of annoyed me is the fact, that there is only one method directive "#org" which is then also used for text and "other" data. In the message command (which does not exist btw.) a pointer is loaded using "loadpointer" and then a standard script is executed to display the message (displaymessage, waitmessage, closeonkeypress etc. ) - If I have another data structure, a "custom" command (ASM routine etc.) that uses another data structure as parameter, I am forced to declare that one as a text or movement. In XSE the decompiler links the "message" command to text and the applymovement command to movement. If I do something like

Code:

...
loadpointer @trainerdata
callasm @SPECIAL_BATTLE
...


my pointer, which is obviously trainer data is retranslated to text on decompilation.
Therefore, there should be other directives, like "#text", "#something" or even "#asm", they all should be kind of customizable (In the attempt I made once to write a code editor I build a class based plugin system, because an asm method cannot simply be described in an XML/INI file.

As for the rest I think thats all pretty much the same in XSE nowadays so switching to a new tool would not be too difficult for users.

~SBird

Splash June 18th, 2015 6:11 AM

I'm cool with anything as long as it's format is the same with XSE not that I matter xD
Plus if it adds easier implementation of movements then switching with it would be very nice.. add a live error checker and your golden! :D

Alexander Nicholi June 18th, 2015 7:41 AM

Quote:

Originally Posted by SBird (Post 8803911)
A standard for pokéscript, or whatever you want to call the scripting language used to control events in third generation games of Pokémon is the first step into better tools and standardization (which, at least in reference to coding, is a good thing)

However I would like to make a few suggestions. One thing that always kind of annoyed me is the fact, that there is only one method directive "#org" which is then also used for text and "other" data. In the message command (which does not exist btw.) a pointer is loaded using "loadpointer" and then a standard script is executed to display the message (displaymessage, waitmessage, closeonkeypress etc. ) - If I have another data structure, a "custom" command (ASM routine etc.) that uses another data structure as parameter, I am forced to declare that one as a text or movement. In XSE the decompiler links the "message" command to text and the applymovement command to movement. If I do something like

Code:

...
loadpointer @trainerdata
callasm @SPECIAL_BATTLE
...


my pointer, which is obviously trainer data is retranslated to text on decompilation.
Therefore, there should be other directives, like "#text", "#something" or even "#asm", they all should be kind of customizable (In the attempt I made once to write a code editor I build a class based plugin system, because an asm method cannot simply be described in an XML/INI file.

This is a really good point. I'm going to go over it with my colleagues and get back to you.

Quote:

Originally Posted by Mana (Post 8803531)
I didn't ask why did you make a thread here. Come on now.

As to the 'same sheet of music' comment, the majority of hackers (at least, in this community) use XSE. They're already on the same sheet. Honestly, it just sounds like work for the sake of work - which is fine, you can do whatever you want - I don't see how this will be helpful to the future of hacking.

Thinking about this, it seems that you're trying to say what XSE provides is good enough for now, and that we're wasting our time making a standard. What you don't get is that this is specification we're collaborating on and I'm writing up is designed to be uniform across tool boundaries – XSE is not a standard, and HackMew basically wrote up the semantics however he felt to with no concern for other tools. We're going above that and making something that is beyond our programs; we're emulating what a lot of the major league open source developers do, and are basically getting our shit together in the respect of scripting. :p

cosarara97 June 18th, 2015 8:04 AM

Quote:

Originally Posted by SBird (Post 8803911)
A standard for pokéscript, or whatever you want to call the scripting language used to control events in third generation games of Pokémon is the first step into better tools and standardization (which, at least in reference to coding, is a good thing)

However I would like to make a few suggestions. One thing that always kind of annoyed me is the fact, that there is only one method directive "#org" which is then also used for text and "other" data. In the message command (which does not exist btw.) a pointer is loaded using "loadpointer" and then a standard script is executed to display the message (displaymessage, waitmessage, closeonkeypress etc. ) - If I have another data structure, a "custom" command (ASM routine etc.) that uses another data structure as parameter, I am forced to declare that one as a text or movement. In XSE the decompiler links the "message" command to text and the applymovement command to movement. If I do something like

Code:

...
loadpointer @trainerdata
callasm @SPECIAL_BATTLE
...


my pointer, which is obviously trainer data is retranslated to text on decompilation.
Therefore, there should be other directives, like "#text", "#something" or even "#asm", they all should be kind of customizable (In the attempt I made once to write a code editor I build a class based plugin system, because an asm method cannot simply be described in an XML/INI file.

As for the rest I think thats all pretty much the same in XSE nowadays so switching to a new tool would not be too difficult for users.

~SBird

When you decompile, the only information you have is what's in the ROM. How would the decompiler know your original source had an #asm in there?

EDIT: btw my tool has been around for a while.

SBird June 18th, 2015 8:24 AM

Quote:

Originally Posted by cosarara97 (Post 8804059)
When you decompile, the only information you have is what's in the ROM. How would the decompiler know your original source had an #asm in there?

EDIT: btw my tool has been around for a while.

Well from the command that called it, i.e. callasm (Yeah, problem for custom tags or the generic loadpointer, but in that case you might eighter use the general string / raw binary or ask the user if you have an UI. Definitely better than decompiling everything into a string. (If your encoding is not cautious enough you might even have problems with compiling, though a well enough designed encoding should use escape characters for fallback anyway)

~SBird

Alexander Nicholi June 18th, 2015 8:33 AM

Quote:

Originally Posted by SBird (Post 8804077)
Well from the command that called it, i.e. callasm (Yeah, problem for custom tags or the generic loadpointer, but in that case you might eighter use the general string / raw binary or ask the user if you have an UI. Definitely better than decompiling everything into a string. (If your encoding is not cautious enough you might even have problems with compiling, though a well enough designed encoding should use escape characters for fallback anyway)

~SBird

Yeah, we were talking, and we think some interactive disassembly would be a good fit for what you're talking about. Basically, there'd be a radio button group or something where you can choose data types or whathaveyou, because data types cannot be communicated about by the ROM, especially with ambiguous commands like loadpointer. :P

Alexander Nicholi June 18th, 2015 6:42 PM

Quote:

Originally Posted by Team Fail (Post 8804629)
I don't know if you got it, but I did mention one feature I'd like to see in this editor would be PROPER Japanese character support. When I was decompiling the Mystery Gift script in BPRJ using XSE, the characters were shown in their English sound equivalents, ie: fushigi instead of ふしぎ (Which translates as mystery,) for instance. If that could be done for this editor, that would be simply amazing. I know this may not benefit English hackers much, but Japanese hacks are a thing and having a script editor that supports different languages (Perhaps in this case extend it for European support as well, such as the awkward characters used in German and French games) without any issues would definitely unify the community in a way.

All of you guys' suggestions are really great for implementations, thank you for sharing them! I'm sure cosarara appreciates the guidance on where we should go with our tools, and I do for certain. :)

Speaking of the thing, I've added a second revision of the standard. There's more complete directive coverage, a standard header, and it's formatted much more similarly to LaTeX with the MS PMincho font.

Alexander Nicholi June 20th, 2015 5:58 PM

Revision 3 has been published, and brings forth the #legacy and #thumb directives, differentiation between modern and legacy PokéScript and their semantics, allowances, etc., and lastly the command structures and their typings as well. Added as well were macro functions while (allowing looping) and text (allowing inline dynamic text).


What I would love to hear about from you, the users, is what directives and macro keywords you'd think would be a good idea to add, what your thoughts are on the divide between modern and legacy PokéScript with #legacy, and anything else you feel like sharing about the project. Thank you!

Zeturic June 20th, 2015 10:34 PM

I'll admit that I just skimmed over it, so maybe I missed it, but I don't believe there's any kind of discussion on the supported number formats.

It would probably be good to specifically discuss them, as you do with strings - decimal integer literals, hex literals (both lower and upper-case) prefaced with
0x
. And, of course, any other literals you feel useful / necessary, such as perhaps binary literals prefaced with
0b
.

Alexander Nicholi June 29th, 2015 11:52 PM

Revision Four is posted, guys. With this update comes the definitions for #freespace, #reserve (with an addition onto the XSE syntax), a specification for cascading preprocessor definitions (#defines having the values of other #defines by name), and also a specification for the macro functions XSE implements with things such as msgbox and giveitem.

In other news, I've finally got to start talking to two other scripting developers, namely Shiny Quagsire and also mewthree9000 (yes, as in the creator of Chaos Black). It's my goal that we can all come to a group consensus for what we'd like to see in this language and get something we're all relatively happy to implement in our tools. Collaboration is big with this, despite the apparence of me fronting the thing. :)

EDIT: I forgot to mention it, but the numerical specification Sagiri was talking about got added too. \o/

Mr.Pkmn July 1st, 2015 8:24 AM

Did you consider that thumb code can switch to arm code?

Mr.Pkmn July 2nd, 2015 5:32 AM

Quote:

Originally Posted by Team Fail (Post 8825795)
It has absolutely nothing to do with this new standard whatsoever.

Quote:

#thumb
Syntax:
#org <offset>
#thumb
When the #thumb directive immediately follows an #org directive, the following bytecode will be assembled or disassembled as THUMB assembly code instead of PokéScript bytecode or other data.
I could use a callasm command to run arm code or mixed thumb code.

Alexander Nicholi July 2nd, 2015 10:24 AM

Quote:

Originally Posted by Mr.Pkmn (Post 8826048)
I could use a callasm command to run arm code or mixed thumb code.

I do want to clarify a bit on why #thumb is even in there. PKSV was the first (and likely the only) tool to implement that feature, and to be honest having Thumb disassembly is a nice "wynaut" thing to have.

The deal with ARM code is that there is hardly a reason for it to be in the ROM because of how sluggishly it executes from it (being the pipeline to it is 16-bit). The only place ARM code makes sense is in EWRAM, because that RAM has a 32-bit bus; ARM code is full 32-bit, whereas Thumb is a "thumb of the arm" 16-bit subset.

With that said, I do think for completeness' sake we should add it, provided I'm not pelted with objections from the developers of the other SEs. I don't think it's the responsibility of the standard to go and cut people off from something the ROM puts in front of them regardless simply because they shouldn't use it – the standard shouldn't be a railing holding up someone who seriously wants to use ARM in the ROM :P

Mr.Pkmn July 2nd, 2015 11:39 AM

Quote:

Originally Posted by Nick (Post 8826407)
The deal with ARM code is that there is hardly a reason for it to be in the ROM because of how sluggishly it executes from it (being the pipeline to it is 16-bit). The only place ARM code makes sense is in EWRAM, because that RAM has a 32-bit bus; ARM code is full 32-bit, whereas Thumb is a "thumb of the arm" 16-bit subset.

Are you sure? I can compile ARM code in the ROM and copy the bytecode in the IWRAM (this is the standard way to use ARM code).
Everything depends on the behavior of the disassembler.

Touched July 2nd, 2015 12:33 PM

Quote:

Originally Posted by Mr.Pkmn (Post 8826508)
Are you sure? I can compile ARM code in the ROM and copy the bytecode in the IWRAM (this is the standard way to use ARM code).
Everything depends on the behavior of the disassembler.

Yes, but why would you want to use ARM. This is from inside the context of a script; it isn't some high-performance audio code where that approach make sense. It's 100% a quick once off action that improves the functionality of a script.
Having this feature just introduces more work for someone implementing the standard, and quite frankly I cannot see any benefit.

Mr.Pkmn July 2nd, 2015 1:21 PM

Quote:

Originally Posted by Touched (Post 8826564)
Yes, but why would you want to use ARM.

Mostly stuff like interrupts which requires critical timing. For example graphic transitions that requires scanline precision with affine matrices (imagine you want to implement bw sea "whoosh" triggered by a script).

Quote:

Originally Posted by Touched (Post 8826564)
Having this feature just introduces more work for someone implementing the standard, and quite frankly I cannot see any benefit.

Certainly not a priority but not something to discard.
I assume it would be nice to have pokescript and asm in a single integrated environment.

Touched July 3rd, 2015 6:37 AM

Quote:

Originally Posted by Mr.Pkmn (Post 8826613)
Mostly stuff like interrupts which requires critical timing. For example graphic transitions that requires scanline precision with affine matrices (imagine you want to implement bw sea "whoosh" triggered by a script).


Certainly not a priority but not something to discard.
I assume it would be nice to have pokescript and asm in a single integrated environment.

I didn't realise you were aiming for a unified environment for more complex routines. I was picturing tiny little routines, which I'm pretty sure is what the #thumb keyword was designed for. Tbh I would rather have an #import keyword that assembles, inserts and allows global symbols to be called with callasm and stuff. It makes sense to divide it hacks like this into multiple files.


All times are GMT -8. The time now is 8:40 AM.


Like our Facebook Page Follow us on Twitter © 2002 - 2018 The PokéCommunity™, pokecommunity.com.
Pokémon characters and images belong to The Pokémon Company International and Nintendo. This website is in no way affiliated with or endorsed by Nintendo, Creatures, GAMEFREAK, The Pokémon Company or The Pokémon Company International. We just love Pokémon.
All forum styles, their images (unless noted otherwise) and site designs are © 2002 - 2016 The PokéCommunity / PokéCommunity.com.
PokéCommunity™ is a trademark of The PokéCommunity. All rights reserved. Sponsor advertisements do not imply our endorsement of that product or service. User generated content remains the property of its creator.

Acknowledgements
Use of PokéCommunity Assets
vB Optimise by DragonByte Technologies Ltd © 2023.