PokeMart as debugging number input / Recreating "Eval" in XSE

Eh, this is my first tutorial. It could be a bit... rough. Here, I'll demonstrate two things: how to create a number input in-game using a PokeMart, and how to dynamically compile and execute a script at run-time.

The two techniques are a bit off the beaten path when it comes to scripting, but they can be quite useful at times. I didn't see any similar tutorials or descriptions on the site (and I Googled extensively -- I did not want to have to figure this out for myself! XD), so I figured I'd write one.


There are probably lots of situations where you'd like the ability to input some number into a game at run-time for debugging purposes, to test something. For example, you may want...

• To be able to obtain any item at run-time, without needing to use cheat codes with your emulator
• To create a music-switcher, to see if a given track "fits" with your map without having to go back and forth between AdvanceMap and your emulator
• To warp to any map in the game, without having to create a test warp, etc..

Well, I came up with a script for the second one of those -- the ability to pick a track in the game and play it, using a short and concise XSE script. No branching paths for every single track. Just a simple, elegant number-input thing and some irresponsible and probably-dangerous scripting devilry.

Let's take a look at the techniques used in that script, and see how we can generalize them.

Problem: How do we input a number in-game?
This would seem to be a tricky one. Is there a quick, easy way to let a player input a number using one of the game's built-in GUI widgets? Not necessarily.

The very first thing my mind jumped to was using the PokeMart selector -- the thing that lets you choose how many of an item you want to buy. Unfortunately, no special allows you to use it directly, and JPAN's Hacked Engine, despite being epicawesome, does not contain any custom specials for the purpose.

So I figured, if we can't use part of the PokeMart functionality, why not just use the whole thing?

Solution: Use a debug item's quantity as our input.
We are going to reserve an item for use as a "debug selector". When you need to input a number, you call a PokeMart script in which only this item is available for purchase. The number you buy represents your numerical input, in decimal.

My ROM uses the item with index 0x36. It was originally an unused item with the name "????????"; I renamed it to "DebugSelector" and set the price to $1. Here is a rudimentary script for generating our selection box (in this tutorial, it will power a music-picker, hence the message shown in the script):


When that script runs, you will be shown a PokeMart in which only the "DebugSelector", priced at $1, is available for purchase. You select a number by simply purchasing that many DebugSelectors. Upon exiting the PokeMart (choosing "SEE YA!"), we move to the next part of the script, where we check how many of the item you purchased.


Problem: How do we check the item quantity?
Thing is, the "checkitem" function lets us check for a certain amount, but we can't check the amount. We can check, for example, if the player has 10 or more (or less) of an item... But we can't, for example, store the quantity of a given item in a variable directly. Which is a problem, because that's exactly what we need to do.

Solution: Recursively decrease the item quantity and increment a variable.
The inelegant way to do it would be to have a checkitem for every item quantity we would accept. checkitem 0x36 0x1, checkitem 0x36 0x2, checkitem 0x36 0x3... That could be a LOT of checkitem calls.

Alternatively, we can create a loop with a counter. Take a look at this:

If you were wondering what 0x8010 was in the first part of the script, wonder no more: it's our counter variable. When we're done, it will hold the amount of the item we (used to) have.

We start by checking if the player has ten or more of the item. If they do, we remove ten of that item, add ten to our counter, and run the check again. (We also reimburse them for the cost of ten of that item -- $10, in this case.) The routine runs itself over and over until the player has nine or fewer of the item.

After that, we do the same thing, but for ones instead of tens. Once this routine finishes, the player will have 0 of the item, and our counter will tell us how many they had purchased. We now have our numeric input. You can pass it to whatever function you wish.


Problem: USING the selected amount. In this case, to pick a music track.
But how do we put that amount into use? Most script functions only accept raw values, not variable references. How do we use the value of a variable to select a song ID for playsong, or an item ID for giveitem, or a Pokemon ID for givepokemon, or something else?

Simple: we are going to do something that is (comparatively speaking) overcomplicated and unconventional. We are going to recreate a tricky little function seen in some formal programming languages.

Solution: Enter "eval".
"Eval" is a function seen in some programming languages. It allows you to take a string of text and treat it as code. It's a lot like compiling a script at run-time, modifying the commands as necessary -- which is exactly what we're going to be doing here. FireRed has a "call" function that allows us to take any offset in RAM, treat it as if it were a script, and execute it. We are going to use the offset of the LASTRESULT (0x800D) variable, by writing our instructions to it and to other variables. We're pretty much recreating "eval" by hand, in Pokemon FireRed.

Here's the script for compiling an eval script that will play the song with the number we selected earlier (plus 0x100). Explanatory code comments are provided.

Note: The next post details a safer, but much more complicated, method. If you are very good at scripting, then I advise you adapt the technique described in the next post rather than using the method I am about to describe in this post. If, on the other hand, you're a novice or intermediate scripter, then feel free to use the below code -- just remember to keep your eval scripts small.


And here is the script that we actually compile at run-time, where XXXX is the value that we insert into it at run-time. In other words, this is what we're creating with all those writebytetooffset and copybyte lines:

You can use this to pass the value of any variable into any function that takes only values. Playsong, warp, additem, giveitem, givemoney, multichoice (for different numbers of options, if using JPAN's hacked engine)... Very useful for debugging-specific numeric inputs, and I can vaguely see a few uses for it in a finished ROM as part of a publicly-accessible script.

The only thing you have to remember is that when you create the eval script, you are overwriting variables' values with the script data. Make absolutely certain that you don't overwrite any variables that you need to use. (For the same reason, you don't want to use these variables in the eval script... Unless you want portions of the script to overwrite themselves. I'm not sure how that would turn out, but I don't think it would end well.)

Conclusion.
There you have it: a script for inputting a number via the PokeMart, and a script for dynamically compiling and executing a new script (allowing you to pass variables into functions that would not normally accept them). Piecing the two together is an exercise left up to the reader.

If you have any questions, feel free to post and I'll try to answer to the best of my ability. If there's anything I could have explained better, please let me know. (I'm aware that this tutorial is... dense. I just don't know how to improve it; I'm hoping that asked questions would give me an idea of where to revise things.)

Improved eval (advanced tutorial)

Improved Eval (Advanced Tutorial)
I've since improved the eval technique. Here is the revised procedure for creating an eval script. Bear in mind that this is quite complex. (At this point, I'm wondering whether documentation of my eval technique would be more fitting in Tutorials or in R&D...)

With these new improvements, the technique has considerably increased in complexity, but this increase allows for larger and more stable dynamic scripts.

Explanation
We start by writing our dynamic script -- the dynamic code that we are running with this technique. This is the script that we compile at run-time. We write hex bytes to a range of variables starting at 0x4100. You can use a different range of variables, but you will have to alter some of the math I do with RAM offsets (good luck figuring it out, I only managed to finish it with some shotgun debugging).

Here is where we run into the first implementation problem, which -- through lots of searching and sheer dumb luck -- I was able to solve quickly and elegantly. While "meta" variables like 0x800D are stored at fixed RAM locations, all other variables -- including the ones we will be using for our dynamic script -- are DMA-protected.

DMA, if memory serves, stands for "Dynamic Memory Access". It is a technique used to stop people from GameSharking OVER 9000 Master Balls into their Bag. DMA-protected information is moved to a random RAM offset, and the location of this offset is stored in a fixed RAM location. For FireRed, this relocation happens whenever the screen changes (i.e. opening a PokeMart, entering a battle, opening the bag, warping), and the fixed RAM location (which holds the offset currently in use by DMA) is at 0x3005008. DMA-protected information includes Bag data, all script flags, and all script variables (except for "meta" variables like 0x800D).

We need to take the four bytes at that location and reverse them. This gives us the current RAM offset of the start of all DMA-protected data; the base DMA offset. Adding 0x1000 to the base DMA offset gets us the location of variable 0x4000. Ergo adding 0x1200 to the base DMA offset gets us the location of variable 0x4100.

Therefore in order to call our script, we must take the base DMA offset, add 0x1200 to it, and then run a "call" command at that offset. Here, we run into our second implementation problem: "call" does not accept a dynamic value (i.e. two variables). A "call" command can only call the address specified when its containing script is compiled.

We work around this by compiling an auxiliary script at run-time into the RAM used to store variables 0x800D and onward. This is our dynamic loading script. The effect is that we compile a very small, and (through sheer dumb luck and good fortune) unlikely-to-be-corrupted caller script into the RAM used for 0x800D. We call that script, which in turn locates and calls the larger (or not, but it certainly can be) dynamic script that we have stored in variables 0x4100 and onward.

The execution of all code hence occurs in the following manner:

• Master script writes hex values to variable 0x4100 and onward; this is the compilation of the dynamic script. Master script then calls the loading helper.
• The loading helper finds the current RAM offset of 0x4100. It compiles the dynamic loading script, which consists of a "call" to that offset followed by a "return".
• The loading script calls the dynamic script.
• The dynamic script executes and returns. Execution returns to the loading script.
• The loading script returns. Execution returns to the master script.

Review of terminology
The master script is the total XSE script that you write. It compiles the dynamic script and calls the loading helper.

The dynamic script is a script that is compiled at run-time, by storing hex values in variables 0x4100 and onward.

The loading helper is a piece of script code that finds the current location of the frequently-moved variable RAM. It compiles the loading script, which calls the dynamic script.

The loading script is a script that is compiled at run-time, by storing hex values in variables 0x800D through 0x8010 (as well as the nonexistent variable 0x800E). We also do some math with variables 0x8011.

The loading helper and loading script
If you're running a FireRed BPRE v1 US ROM, then I've already done all the work here for you. Take a look at this:


That is the loading helper, which contains code that compiles and executes the loading script.

You'll notice that we use 0x8011 to do math on 0x800E indirectly. The reason that we cannot directly manipulate 0x800E is because it is not actually a variable. Setvar, addvar, subvar, and pals refuse to operate on it. If you try to copyvar the value of 0x800E into some other variable, the game will instead copy some random word from Darwin-only-knows-where.

Here is where we hit that stroke of dumb luck I mentioned earlier -- the one that prevents corruption of the loading script during execution of the dynamic script. The "return" byte in the loading script is contained in the RAM that would be used for 0x800E if 0x800E existed... But since 0x800E doesn't exist, there's no risk of [set|add|sub|copy]var statements in the dynamic script corrupting it! Unless the dynamic script deliberately uses writebytetooffset/copybyte/etc. to overwrite 0x800E's RAM, there's no risk of the loading script failing to return due to it being overwritten.

(Of course, overwriting 0x8010 -- the var containing the FF byte that terminates the script -- might be problematic. Unless XSE just adds that as padding, and it doesn't actually do anything. In any case, I haven't tested this, so do be careful with 0x8010.)

In any case, all you have to do is copy that routine into your script. You actually only need to compile it once; so long as you're careful to use #remove and not #removeall, you can manually reference that one copy of the routine from any and all scripts that use this technique (so long as you write down what offset the routine is written to).

A sample dynamic script
Here is an example script. It generates a random Pokemon index, and then shows the image of that Pokemon. As a bonus, it will also show a dialog box listing the name and native region of that Pokemon. The full code, spoiler'd for your convenience:


Let's look at this code piece-by-piece.

This is simple enough. We choose a random species number. If it is above a certain value, we add 0x1A to it (@adjustHoennNumber) to account for the several unused "?" Pokemon that lie between the Johto and Hoenn Pokemon indices.

Here, we compile our dynamic script. You'll see that I've translated the script into hex. Doing so is fairly simple:

• XSE's Command Help (F1) tells you the hex byte of the command itself.
• If a command takes a multibyte value, reverse the byte order when translating to hex.
• When translating from hex to vars, group bytes into words and then reverse the bytes within each word.
• Strategically-placed nops (0x00) before a command can save you some tricky math. Above, I used one nop so that XXYY remained as one whole variable, rather than being split across them.

Once we've translated the script into vars with hex values, we simply write them into our specially-designed eval variables, starting with 0x4100. It's a simple series of setvars, plus a copyvar or two for whichever parts of our script are actually dynamic. (In this case, the Pokemon species index, represented by "XXYY" in the code comments.)

Moving on:


And here's where stuff goes down. We show a simple message box, and then we call our loading helper. 0x800EC4 is not the offset you use in your ROM. You use the offset that @fExecuteDynamicScript was compiled to in the previous section ("The loading helper and loading script"). 0x800EC4 just happens to be the offset of that routine in my ROM hack.

Anyway, that call triggers the lo-- wait, I've already explained this, like, three times, haven't I? :P

The rest of the script is just cleanup and message displays. We've covered everything that's important.

Case study: simulating arrays using dynamic scripts
This case study requires JPAN's FireRed Hacked Engine.

Consider this code:

This master script powers an NPC that lets you choose from a menu of music tracks. Pick one, and he'll change the currently-playing song to it. Alternatively, you can have him pick one for you at random, or you can decline and not change the music at all.

Let's step through it to see some of the techniques it uses, which can of course be generalized and used in other applications.

Here, we are creating a makeshift array -- in this case, of song indices -- by writing to a sequence of consecutive variables. We choose 0x4000 for convenience sake, but any variables (except 0x4100 and onward!) could be used for the purpose.

That's all an array really is -- a list of things. In this case, we're just spreading that list across multiple variables. But how do we read from it dynamically?

That's how.

We compile a dynamic script that will take some variable XXYY, and copy its value into 0x4011. Instead of using separate code branches for each possible selection, we use a dynamic script and a "pseudo-array".

That script runs:

And then we use 0x4011 to pick the music track. We do that with another script, since (to the best of my knowledge) we can only pass raw values, not variables, into playsong.


And there we go.

This may seem like total overkill. Why not use code branches for each selection, instead? Well, if you thoroughly understand this technique, then you may find that these two benefits are worth the added complexity (it really depends on your tastes as a programmer):

• Ditch all the comments, and (in terms of line count) this code is shorter than an equivalent with branches would be. The more options you have, the bigger the size difference becomes.
• In this code, your array of songs is right at the top. You don't need to sift through instructions and branches to change the songs; they're right up in an obvious and prominent spot, and easily alterable.

Conclusion
This is how you can dynamically compile a script of (almost) any size at run-time, without the risk of that script overwriting itself when its instructions are processed.

As opposed to my first (prototypical) eval method, in the preceding post. That method limits the size of the script to the RAM block reserved for "meta" variables (a rather small area), and runs the risk that the script might overwrite itself (i.e. one of its commands sets 0x800D, nuking the first two bytes of the script) in a manner that could cause major problems (i.e. one of the later "meta" variables gets overwritten and boom, your script no longer returns or ends).

The only thing to keep in mind is that, as with the old method, you will nuke the "meta" variables when running dynamic scripts (specifically, when the loading script is compiled). If you need things like LASTTALKED or PLAYERFACING, copy them to standard variables before calling the loading helper.

Complicated? Yes. Overkill? Most definitely. But useful? Absolutely.

Just a heads-up for anyone using this: script variables above 0x40FF are demonstrably unsafe. The principles in this tutorial will still work, but you should limit yourself to the lower, safer variables.

On the bright side, far more commands accept variables as parameters than you might expect, so this technique isn't often necessary anyway.