Development: Standardization of PokéScript

[Alexander Nicholi]

I could use a callasm command to run arm code or mixed thumb code.

I do want to clarify a bit on why #thumb is even in there. PKSV was the first (and likely the only) tool to implement that feature, and to be honest having Thumb disassembly is a nice "wynaut" thing to have.

The deal with ARM code is that there is hardly a reason for it to be in the ROM because of how sluggishly it executes from it (being the pipeline to it is 16-bit). The only place ARM code makes sense is in EWRAM, because that RAM has a 32-bit bus; ARM code is full 32-bit, whereas Thumb is a "thumb of the arm" 16-bit subset.

With that said, I do think for completeness' sake we should add it, provided I'm not pelted with objections from the developers of the other SEs. I don't think it's the responsibility of the standard to go and cut people off from something the ROM puts in front of them regardless simply because they shouldn't use it – the standard shouldn't be a railing holding up someone who seriously wants to use ARM in the ROM :P

 

[Mr.Pkmn]

The deal with ARM code is that there is hardly a reason for it to be in the ROM because of how sluggishly it executes from it (being the pipeline to it is 16-bit). The only place ARM code makes sense is in EWRAM, because that RAM has a 32-bit bus; ARM code is full 32-bit, whereas Thumb is a "thumb of the arm" 16-bit subset.

Are you sure? I can compile ARM code in the ROM and copy the bytecode in the IWRAM (this is the standard way to use ARM code).
Everything depends on the behavior of the disassembler.

 

[Touched]

Are you sure? I can compile ARM code in the ROM and copy the bytecode in the IWRAM (this is the standard way to use ARM code).
Everything depends on the behavior of the disassembler.

Yes, but why would you want to use ARM. This is from inside the context of a script; it isn't some high-performance audio code where that approach make sense. It's 100% a quick once off action that improves the functionality of a script.
Having this feature just introduces more work for someone implementing the standard, and quite frankly I cannot see any benefit.

 

[Mr.Pkmn]

Yes, but why would you want to use ARM.

Mostly stuff like interrupts which requires critical timing. For example graphic transitions that requires scanline precision with affine matrices (imagine you want to implement bw sea "whoosh" triggered by a script).

Having this feature just introduces more work for someone implementing the standard, and quite frankly I cannot see any benefit.

Certainly not a priority but not something to discard.
I assume it would be nice to have pokescript and asm in a single integrated environment.

 

[Touched]

Mostly stuff like interrupts which requires critical timing. For example graphic transitions that requires scanline precision with affine matrices (imagine you want to implement bw sea "whoosh" triggered by a script).


Certainly not a priority but not something to discard.
I assume it would be nice to have pokescript and asm in a single integrated environment.

I didn't realise you were aiming for a unified environment for more complex routines. I was picturing tiny little routines, which I'm pretty sure is what the #thumb keyword was designed for. Tbh I would rather have an #import keyword that assembles, inserts and allows global symbols to be called with callasm and stuff. It makes sense to divide it hacks like this into multiple files.

 

[Zeturic]

Surprised at the lack of #undef to undefine macros. Was that intentional?

Also, you may want to set a maximum on conditional compilation depth (nested levels of #ifdef / #ifndef) and file inclusion depth (nested #include), to avoid the possibility of a file #include-ing itself, for example.

Can I also suggest adding predefined macros, based on the game code (e.g. BPRE) of the ROM the script is being compiled for? The values don't really matter; the idea would so that, if a script needs to be slightly different based on the game, you can use an #ifdef.

#ifdef BPRE
// insert FR specific code here
#endif
#ifdef BPEE
// insert Emerald specific code here
#endif

 

[Alexander Nicholi]

Surprised at the lack of #undef to undefine macros. Was that intentional?

Legitimately did not know that existed. o.o Adding it in the fifth revision.

Also, you may want to set a maximum on conditional compilation depth (nested levels of #ifdef / #ifndef) and file inclusion depth (nested #include), to avoid the possibility of a file #including itself, for example.

I'm not so sure about this - I'd like to hear what the other devs think. Infinite includes to me are one of those things implementations should explicitly catch; I don't think an arbitrary limit of sorts is the best solution.

Can I also suggest adding predefined macros, based on the game code (e.g. BPRE) of the ROM the script is being compiled for? The values don't really matter; the idea would so that, if a script needs to be slightly different based on the game, you can use an #ifdef.

#ifdef BPRE
// insert FR specific code here
#endif
#ifdef BPEE
// insert Emerald specific code here
#endif

This definitely deserves to be added. Will make it.

 

[esperance]

I've gotta say you're rather brave for using the underappreciated D language to write your compiler. I'd definitely like to help you, but my knowledge of D is minimal.

Now, I also have a question for you. In the specification, it says that #erase will allow dynamic offsets. How exactly would that work?

 

[Alexander Nicholi]

I've gotta say you're rather brave for using the underappreciated D language to write your compiler. I'd definitely like to help you, but my knowledge of D is minimal.

Now, I also have a question for you. In the specification, it says that #erase will allow dynamic offsets. How exactly would that work?

Like this, is how I had it planned out (in my head):

#dynamic 0x800000

#org @begin
// say this turns out to be 0x4E bytes long

#erase @begin 0x4E

It could also erase the offset gathered from @begin without an #org entirely, but that's kinda useless as it's erasing the already erased.

I don't really like it compared to #eorg. It's just carryover from XSE tbh

 

[esperance]

Like this, is how I had it planned out (in my head):

#dynamic 0x800000

#org @begin
// say this turns out to be 0x4E bytes long

#erase @begin 0x4E

It could also erase the offset gathered from @begin without an #org entirely, but that's kinda useless as it's erasing the already erased.

I don't really like it compared to #eorg. It's just carryover from XSE tbh

Oh, I see. So it would erase 0x4E bytes from the offset once it has been determined by the compiler. Why bother allowing dynamic offsets, though, if #eorg would be a better option? It might be a better idea to have #erase only allowed in legacy mode. In regular mode, it feels like #erase as it is now is mostly for allocating freespace, so why not have a command like #alloc instead?

 

[Alexander Nicholi]

Oh, I see. So it would erase 0x4E bytes from the offset once it has been determined by the compiler. Why bother allowing dynamic offsets, though, if #eorg would be a better option? It might be a better idea to have #erase only allowed in legacy mode. In regular mode, it feels like #erase as it is now is mostly for allocating freespace, so why not have a command like #alloc instead?

You know, I really like that idea. I dunno how terribly useful it may be, but I'll double check with the others to be sure.

 

[Mr.Pkmn]

A #copy command would be really useful for making batch scripts (eg: repoints)

 

[Alexander Nicholi]

The fifth revision of the standard is now published! Updates include environmental constants, the #undef directive, a new #mode directive that replaces the function of the #legacy directive, and assembly-style #raw replacements #byte, #hword, and #word.

The #mode directive sets the type of code to be read for the entirety of a particular file. It can be set to pks2 for standard, modern PokéScript, pksv or xse for legacy PokéScript, or battlescript/battleai for the scripts used with those engines.

 

[esperance]

Hmm, I definitely like the #byte, #hword, etc. directives. Much simpler than the #raw format.
But my word the number of directives is huge!

 

[Alexander Nicholi]

Hmm, I definitely like the #byte, #hword, etc. directives. Much simpler than the #raw format.
But my word the number of directives is huge!

I think the reason for that is because it's the area where the language can exercise the most freedom and truly fill its capabilities. The commands themselves are bound to the engine built into the ROMs so there isn't a whole lot to do with them, and macro functions have a rather limited sphere of practicality before you begin making higher abstractions – abstractions which are more well-suited in a language like subScript, not being as appropriate for the near-1:1 assembly language we're standardizing.

I already have a few things needing added for the sixth revision, such as the code that the currently defined macro functions boil down to, and also the XSE macro sugar setwildbattle and the code it corresponds to.

What I (and likely the others) would like to see for suggestions at this point—since we've about rounded out our directive collection for the time being—are ideas for more macro functions! The only guidelines I can really provide for coming up with them are that they (A) don't have overly complex assembly code correspondence (use reason with this), and (B) would prove useful and (at least somewhat) practical in the real world of PokéScript. :)

 

[Alexander Nicholi]

...Alright.

I've completed a new revision of the PokéScript Standard, and Revision 6 brings a boatload of changes, clarifications, and improvements over everything before it.

It is certainly more complete and concise; it separates the languages appropriately using #mode, clearly defines macro functions and pseudomacro "sugar," sets out a much more concise standard header, defines a character map for Pokétext, and boasts more complete descriptions and blurbs of everything. And best of all... it's written in LaTeX.

Download R6 as LaTeX or as a PDF!

 

[Blah]

I'd personally be happy if it supported expanding on the script commands. That's been the biggest fault of script writers in my opinion, you cannot make them support expanded commands. There are a few nops which you can use, but other than that there's nothing else.

 

[Alexander Nicholi]

I'd personally be happy if it supported expanding on the script commands. That's been the biggest fault of script writers in my opinion, you cannot make them support expanded commands. There are a few nops which you can use, but other than that there's nothing else.

How exactly do you propose we do this and still use the PokéScript engine? When you say "expanded commands," do you propose 16-bit LE command identifiers or something of that nature?

If you're imagining a custom scripting engine (which, if implemented, would be absolutely great), what sort of layout do you picture with that?

 

[Blah]

How exactly do you propose we do this and still use the PokéScript engine? When you say "expanded commands," do you propose 16-bit LE command identifiers or something of that nature?

If you're imagining a custom scripting engine (which, if implemented, would be absolutely great), what sort of layout do you picture with that?

There's a script commands table -> Repoint it to expand -> add your own cmds -> Oh no script editor is not supporting my new commands! This is the basic problem. When you're adding the command library, I believe it's a byte -> cmd relationship right? How do you determine which byte = which command? It's probably from a dictionary or some sort of data structure you've hard coded. Mind you, I haven't looked at this stuff in a bit so I'm a little foggy.

I believe this is the issue. A fix would be the make it read from the table given by the actual game code.

...
...
ROM:08069AF6                 LDR     R1, =0x815F9B4 @script cmd table location
ROM:08069AF8                 LDR     R2, =0x815FD08 @script cmd table end location (last command)
...
...

Here's a snip of code pointing to the start and end of the scripting table. When most scripts are called they execute this routine which has these 2 commands. Particularly these two offsets are at 08069B1C and 08069B20. Just make your script editor read from there or allow the user to edit a file which you'd read from (I think the former would be easier for most folks and the latter easier for you).

EDIT: If a new command is found without a cmd name just prompt the user to name it and save it in your database!

 

[Touched]

EDIT: If a new command is found without a cmd name just prompt the user to name it and save it in your database!

How does this work for argument lengths and variadic commands? IMO it would be better to just have a configuration file for an individual ROM or something in the standard (a directive maybe) that allows you define a command. Something expressive like:

#command INDEX "description"
    #arg LENGTH "name" "description"
    ...
#endcommand

That way you could #include them as distribute as a header if need be. No database necessary.

Also, with a proper macro system you could do this for free:

#macro mycommand arg1 arg2 arg3
#raw 0xFE // command index
#short arg1 // Do we even have these? GNU as has them
#byte arg2 
#long arg3
#endmacro

// Example usage
mycommand 1 2 3

// Hex Output:
// FE 01 00 02 03 00 00 00

Personally, I'd advocate the latter. The above snippet is also why I am against using the C preprocessor. The macro system is awful, and mixing and matching preprocessors feels awkward.