View Single Post
  #2    
Old July 27th, 2011 (10:35 PM).
DavidJCobb DavidJCobb is offline
RESIDENT RAAAAAAAAAAAAGEMASTER
 
Join Date: Jul 2010
Gender: Male
Nature: Lonely
Posts: 275
Improved Eval (Advanced Tutorial)
I've since improved the eval technique. Here is the revised procedure for creating an eval script. Bear in mind that this is quite complex. (At this point, I'm wondering whether documentation of my eval technique would be more fitting in Tutorials or in R&D...)

With these new improvements, the technique has considerably increased in complexity, but this increase allows for larger and more stable dynamic scripts.

Explanation
We start by writing our dynamic script -- the dynamic code that we are running with this technique. This is the script that we compile at run-time. We write hex bytes to a range of variables starting at 0x4100. You can use a different range of variables, but you will have to alter some of the math I do with RAM offsets (good luck figuring it out, I only managed to finish it with some shotgun debugging).

Here is where we run into the first implementation problem, which -- through lots of searching and sheer dumb luck -- I was able to solve quickly and elegantly. While "meta" variables like 0x800D are stored at fixed RAM locations, all other variables -- including the ones we will be using for our dynamic script -- are DMA-protected.

DMA, if memory serves, stands for "Dynamic Memory Access". It is a technique used to stop people from GameSharking OVER 9000 Master Balls into their Bag. DMA-protected information is moved to a random RAM offset, and the location of this offset is stored in a fixed RAM location. For FireRed, this relocation happens whenever the screen changes (i.e. opening a PokeMart, entering a battle, opening the bag, warping), and the fixed RAM location (which holds the offset currently in use by DMA) is at 0x3005008. DMA-protected information includes Bag data, all script flags, and all script variables (except for "meta" variables like 0x800D).

We need to take the four bytes at that location and reverse them. This gives us the current RAM offset of the start of all DMA-protected data; the base DMA offset. Adding 0x1000 to the base DMA offset gets us the location of variable 0x4000. Ergo adding 0x1200 to the base DMA offset gets us the location of variable 0x4100.

Therefore in order to call our script, we must take the base DMA offset, add 0x1200 to it, and then run a "call" command at that offset. Here, we run into our second implementation problem: "call" does not accept a dynamic value (i.e. two variables). A "call" command can only call the address specified when its containing script is compiled.

We work around this by compiling an auxiliary script at run-time into the RAM used to store variables 0x800D and onward. This is our dynamic loading script. The effect is that we compile a very small, and (through sheer dumb luck and good fortune) unlikely-to-be-corrupted caller script into the RAM used for 0x800D. We call that script, which in turn locates and calls the larger (or not, but it certainly can be) dynamic script that we have stored in variables 0x4100 and onward.

The execution of all code hence occurs in the following manner:
  • Master script writes hex values to variable 0x4100 and onward; this is the compilation of the dynamic script. Master script then calls the loading helper.
  • The loading helper finds the current RAM offset of 0x4100. It compiles the dynamic loading script, which consists of a "call" to that offset followed by a "return".
  • The loading script calls the dynamic script.
  • The dynamic script executes and returns. Execution returns to the loading script.
  • The loading script returns. Execution returns to the master script.

Review of terminology
The master script is the total XSE script that you write. It compiles the dynamic script and calls the loading helper.

The dynamic script is a script that is compiled at run-time, by storing hex values in variables 0x4100 and onward.

The loading helper is a piece of script code that finds the current location of the frequently-moved variable RAM. It compiles the loading script, which calls the dynamic script.

The loading script is a script that is compiled at run-time, by storing hex values in variables 0x800D through 0x8010 (as well as the nonexistent variable 0x800E). We also do some math with variables 0x8011.

The loading helper and loading script
If you're running a FireRed BPRE v1 US ROM, then I've already done all the work here for you. Take a look at this:

Code:
#org @fExecuteDynamicScript
/*
 Compilation (loading script):
*/
writebytetooffset 0x04 0x020370D0 // 0x800D: "call", address byte
copybyte 0x020370D1 0x03005008
copybyte 0x020370D2 0x03005009 // 0x800E: address byte, address byte
copybyte 0x020370D3 0x0300500A
copybyte 0x020370D8 0x020370D2 // math on 0x800E (address += 0x1200)
copybyte 0x020370D9 0x020370D3
addvar 0x8011 0x0012
copybyte 0x020370D2 0x020370D8
copybyte 0x020370D3 0x020370D9
copybyte 0x020370D4 0x0300500B // 0x800F: address byte, "return"
writebytetooffset 0x03 0x020370D5
writebytetooffset 0xFF 0x020370D6 // 0x8010: script terminating byte, nop
writebytetooffset 0x00 0x020370D7
/*
 Execution (loading script -> dynamic script):
*/
call 0x020370D0
return
That is the loading helper, which contains code that compiles and executes the loading script.

You'll notice that we use 0x8011 to do math on 0x800E indirectly. The reason that we cannot directly manipulate 0x800E is because it is not actually a variable. Setvar, addvar, subvar, and pals refuse to operate on it. If you try to copyvar the value of 0x800E into some other variable, the game will instead copy some random word from Darwin-only-knows-where.

Here is where we hit that stroke of dumb luck I mentioned earlier -- the one that prevents corruption of the loading script during execution of the dynamic script. The "return" byte in the loading script is contained in the RAM that would be used for 0x800E if 0x800E existed... But since 0x800E doesn't exist, there's no risk of [set|add|sub|copy]var statements in the dynamic script corrupting it! Unless the dynamic script deliberately uses writebytetooffset/copybyte/etc. to overwrite 0x800E's RAM, there's no risk of the loading script failing to return due to it being overwritten.

(Of course, overwriting 0x8010 -- the var containing the FF byte that terminates the script -- might be problematic. Unless XSE just adds that as padding, and it doesn't actually do anything. In any case, I haven't tested this, so do be careful with 0x8010.)

In any case, all you have to do is copy that routine into your script. You actually only need to compile it once; so long as you're careful to use #remove and not #removeall, you can manually reference that one copy of the routine from any and all scripts that use this technique (so long as you write down what offset the routine is written to).

A sample dynamic script
Here is an example script. It generates a random Pokemon index, and then shows the image of that Pokemon. As a bonus, it will also show a dialog box listing the name and native region of that Pokemon. The full code, spoiler'd for your convenience:

Spoiler:
Code:
#dynamic 0x800000

#org @start
lock
faceplayer
/*
 Get a random species number.
*/
random 0x0181
copyvar 0x4000 0x800D
addvar 0x4000 0x0001
compare 0x4000 0x00FB
if 0x02 call @adjustHoennNumber

/*
 Compiling dynamic script.
 Script:
    showpokepic XXYY 0x00 0x00
    bufferpokemon 0x00 XXYY
    return
 Hex:
    00 75|YY XX|00 00|
    7D 00|YY XX|
    03 FF
 Vars:
    7500|XXYY|0000|007D|XXYY|FF03
*/
setvar 0x4100 0x7500
copyvar 0x4101 0x4000
setvar 0x4102 0x0000
setvar 0x4103 0x007D
copyvar 0x4104 0x4000
setvar 0x4105 0xFF03

msgbox @sAnnounce 0x02
call 0x800EC4 // dynamic script loader and executer
call @checkRegion
msgbox @sExplain 0x02
release
hidepokepic
end

/*
 There are 0x1A "?" Pokemon between the 
 last Johto and first Hoenn Pokemon. If 
 our random number selected a Hoenn 
 species, we need to adjust for this.
*/
#org @adjustHoennNumber
addvar 0x4000 0x001A
return

/*
 Check the Pokemon index number to find 
 what region it is from. Most of the 
 Pokemon are in Pokedex order already, 
 which makes this easy for us.
*/
#org @checkRegion
call @bZALGO
compare 0x4000 0x0001
if 0x4 call @bKanto
compare 0x4000 0x0098
if 0x4 call @bJohto
compare 0x4000 0x00FC
if 0x4 call @bZALGO
compare 0x4000 0x0115
if 0x4 call @bHoenn
compare 0x4000 0x019C
if 0x4 call @bZALGO
return

#org @bKanto
bufferstring 0x01 @sKanto
return
#org @bJohto
bufferstring 0x01 @sJohto
return
#org @bHoenn
bufferstring 0x01 @sHoenn
return
#org @bZALGO
bufferstring 0x01 @sZALGO
return

#org @sAnnounce
= Hey, look at this!
#org @sKanto
= Kanto
#org @sJohto
= Johto
#org @sHoenn
= Hoenn
#org @sZALGO
= ZALGO
#org @sExplain
= It's a [buffer1], from the [buffer2]\nregion.


Let's look at this code piece-by-piece.

Code:
#dynamic 0x800000

#org @start
lock
faceplayer
/*
 Get a random species number.
*/
random 0x0181
copyvar 0x4000 0x800D
addvar 0x4000 0x0001
compare 0x4000 0x00FB
if 0x02 call @adjustHoennNumber
This is simple enough. We choose a random species number. If it is above a certain value, we add 0x1A to it (@adjustHoennNumber) to account for the several unused "?" Pokemon that lie between the Johto and Hoenn Pokemon indices.

Code:
/*
 Compiling dynamic script.
 Script:
    showpokepic XXYY 0x00 0x00
    bufferpokemon 0x00 XXYY
    return
 Hex:
    00 75|YY XX|00 00|
    7D 00|YY XX|
    03 FF
 Vars:
    7500|XXYY|0000|007D|XXYY|FF03
*/
setvar 0x4100 0x7500
copyvar 0x4101 0x4000
setvar 0x4102 0x0000
setvar 0x4103 0x007D
copyvar 0x4104 0x4000
setvar 0x4105 0xFF03
Here, we compile our dynamic script. You'll see that I've translated the script into hex. Doing so is fairly simple:
  • XSE's Command Help (F1) tells you the hex byte of the command itself.
  • If a command takes a multibyte value, reverse the byte order when translating to hex.
  • When translating from hex to vars, group bytes into words and then reverse the bytes within each word.
  • Strategically-placed nops (0x00) before a command can save you some tricky math. Above, I used one nop so that XXYY remained as one whole variable, rather than being split across them.
Once we've translated the script into vars with hex values, we simply write them into our specially-designed eval variables, starting with 0x4100. It's a simple series of setvars, plus a copyvar or two for whichever parts of our script are actually dynamic. (In this case, the Pokemon species index, represented by "XXYY" in the code comments.)

Moving on:

Code:
msgbox @sAnnounce 0x02
call 0x800EC4 // dynamic script loader and executer
And here's where stuff goes down. We show a simple message box, and then we call our loading helper. 0x800EC4 is not the offset you use in your ROM. You use the offset that @fExecuteDynamicScript was compiled to in the previous section ("The loading helper and loading script"). 0x800EC4 just happens to be the offset of that routine in my ROM hack.

Anyway, that call triggers the lo-- wait, I've already explained this, like, three times, haven't I? :P

The rest of the script is just cleanup and message displays. We've covered everything that's important.

Case study: simulating arrays using dynamic scripts
This case study requires JPAN's FireRed Hacked Engine.

Consider this code:
Spoiler:
Code:
/*
 Demo:
  - Selection from arrays
     - Using user-entered multichoice value with 
       dynamic script
     - Using random value with dynamic script
*/

#dynamic 0x800000

#org @start
lock
faceplayer
msgbox @sIntro 0x02

/*
 Constructing our makeshift array:
*/
setvar 0x4000 0x0117 // Song: Cinnabar Island
setvar 0x4001 0x0120 // Song: Mt. Moon
setvar 0x4002 0x0132 // Song: Pokemon Tower
setvar 0x4003 0x0151 // Song: Sevii Islands 6 and 7 / Violet City
setvar 0x4004 0x0157 // Song: Deoxys Encounter

/*
 Asking the user to pick an option:
*/
setvar 0x8006 0x0 // Multichoice 0
loadpointer 0x0 @sSong0
special 0x25
setvar 0x8006 0x1 // Multichoice 1
loadpointer 0x0 @sSong1
special 0x25
setvar 0x8006 0x2 // Multichoice 2
loadpointer 0x0 @sSong2
special 0x25
setvar 0x8006 0x3 // Multichoice 3
loadpointer 0x0 @sSong3
special 0x25
setvar 0x8006 0x4 // Multichoice 4
loadpointer 0x0 @sSong4
special 0x25
setvar 0x8006 0x5 // Multichoice 5
loadpointer 0x0 @sRandom
special 0x25
setvar 0x8006 0x6 // Multichoice 6
loadpointer 0x0 @sCancel
special 0x25
multichoice 0x0 0x0 0x25 0x0 // Multichoice trigger
copyvar 0x4010 0x800D // save result
compare 0x4010 0x0005
if 0x1 call @randomize
compare 0x4010 0x0006
if 0x1 call @cancel

/*
 Compiling dynamic script.
 Description:
    Copies the value of a variable into 
    another variable. The variable to be 
    copied is dynamically selected at 
    run-time. Basically, we can pick an 
    element out of an "array" using a 
    variable index, and return that 
    element by copying it into variable 
    0x4011.
 Script:
    copyvar 0x4011 0xXXYY
    return
 Hex:
    00 19|11 40|YY XX|
    03 FF
 Vars:
    1900 4011 XXYY FF03
*/
setvar 0x4100 0x1900
setvar 0x4101 0x4011 // Destination variable for returning the element
copyvar 0x4102 0x4010 // Variable holding the index of the element we want
addvar 0x4102 0x4000 // The array starts at variable 0x4000, so add that amount to the zero-based index to get a variable ID.
setvar 0x4103 0xFF03

call 0x800EC4 // dynamic script loader and executer

/*
 Compiling dynamic script.
 Description:
    Selects a music track based on a 
    value we inject.
 Script:
    playsong 0xXXYY 0x00
    return
 Hex:
    00 33|YY XX|00 03|FF 00
 Vars:
    3300 XXYY 0300 00FF
*/
setvar 0x4100 0x3300
copyvar 0x4101 0x4011
setvar 0x4102 0x0300
setvar 0x4103 0x00FF

call 0x800EC4 // dynamic script loader and executer

msgbox @sSongChanged 0x02

release
end

#org @randomize
random 0x0005
copyvar 0x4010 0x800D
return

#org @cancel
release
end

#org @sIntro
= Hi, there! Would you like to change\nthe music?
#org @sSong0
= Cinnibar Theme
#org @sSong1
= Mt. Moon Theme
#org @sSong2
= Pokemon Tower Theme
#org @sSong3
= Violet Theme
#org @sSong4
= Deoxys OW Theme
#org @sRandom
= Surprise me
#org @sCancel
= I'd rather not
#org @sSongChanged
= Lemme know if you wanna change the\nmusic again.


This master script powers an NPC that lets you choose from a menu of music tracks. Pick one, and he'll change the currently-playing song to it. Alternatively, you can have him pick one for you at random, or you can decline and not change the music at all.

Let's step through it to see some of the techniques it uses, which can of course be generalized and used in other applications.

Code:
/*
 Constructing our makeshift array:
*/
setvar 0x4000 0x0117 // Song: Cinnabar Island
setvar 0x4001 0x0120 // Song: Mt. Moon
setvar 0x4002 0x0132 // Song: Pokemon Tower
setvar 0x4003 0x0151 // Song: Sevii Islands 6 and 7 / Violet City
setvar 0x4004 0x0157 // Song: Deoxys Encounter
Here, we are creating a makeshift array -- in this case, of song indices -- by writing to a sequence of consecutive variables. We choose 0x4000 for convenience sake, but any variables (except 0x4100 and onward!) could be used for the purpose.

That's all an array really is -- a list of things. In this case, we're just spreading that list across multiple variables. But how do we read from it dynamically?

Code:
/*
 Compiling dynamic script.
 Description:
    Copies the value of a variable into 
    another variable. The variable to be 
    copied is dynamically selected at 
    run-time. Basically, we can pick an 
    element out of an "array" using a 
    variable index, and return that 
    element by copying it into variable 
    0x4011.
 Script:
    copyvar 0x4011 0xXXYY
    return
 Hex:
    00 19|11 40|YY XX|
    03 FF
 Vars:
    1900 4011 XXYY FF03
*/
setvar 0x4100 0x1900
setvar 0x4101 0x4011 // Destination variable for returning the element
copyvar 0x4102 0x4010 // Variable holding the index of the element we want
addvar 0x4102 0x4000 // The array starts at variable 0x4000, so add that amount to the zero-based index to get a variable ID.
setvar 0x4103 0xFF03
That's how.

We compile a dynamic script that will take some variable XXYY, and copy its value into 0x4011. Instead of using separate code branches for each possible selection, we use a dynamic script and a "pseudo-array".

That script runs:
Code:
call 0x800EC4 // dynamic script loader and executer
And then we use 0x4011 to pick the music track. We do that with another script, since (to the best of my knowledge) we can only pass raw values, not variables, into playsong.

Code:
/*
 Compiling dynamic script.
 Description:
    Selects a music track based on a 
    value we inject.
 Script:
    playsong 0xXXYY 0x00
    return
 Hex:
    00 33|YY XX|00 03|FF 00
 Vars:
    3300 XXYY 0300 00FF
*/
setvar 0x4100 0x3300
copyvar 0x4101 0x4011
setvar 0x4102 0x0300
setvar 0x4103 0x00FF

call 0x800EC4 // dynamic script loader and executer
And there we go.

This may seem like total overkill. Why not use code branches for each selection, instead? Well, if you thoroughly understand this technique, then you may find that these two benefits are worth the added complexity (it really depends on your tastes as a programmer):
  • Ditch all the comments, and (in terms of line count) this code is shorter than an equivalent with branches would be. The more options you have, the bigger the size difference becomes.
  • In this code, your array of songs is right at the top. You don't need to sift through instructions and branches to change the songs; they're right up in an obvious and prominent spot, and easily alterable.

Conclusion
This is how you can dynamically compile a script of (almost) any size at run-time, without the risk of that script overwriting itself when its instructions are processed.

As opposed to my first (prototypical) eval method, in the preceding post. That method limits the size of the script to the RAM block reserved for "meta" variables (a rather small area), and runs the risk that the script might overwrite itself (i.e. one of its commands sets 0x800D, nuking the first two bytes of the script) in a manner that could cause major problems (i.e. one of the later "meta" variables gets overwritten and boom, your script no longer returns or ends).

The only thing to keep in mind is that, as with the old method, you will nuke the "meta" variables when running dynamic scripts (specifically, when the loading script is compiled). If you need things like LASTTALKED or PLAYERFACING, copy them to standard variables before calling the loading helper.

Complicated? Yes. Overkill? Most definitely. But useful? Absolutely.
__________________
Reply With Quote