how to recover wasted space in a ROM(and how to remove a batched script)

[jakerman999]

one of the less severe, but certainly more annoying problems in ROM hacking, is wasting space. this problem can be solved in several ways, such as expanding the ROM, but that could cause errors in some programs. a better solution, albeit one that takes more time, is to trim excess material out of the ROM and rewrite it with empty space.

this tutorial will cover:
fixing an incomplete/wrong script without wasting space (newb friendly and technical)
deleting a script that you don't want
recovering space after multiple compiles
using four Preprocessing Directives in XSE(#clean, #erase, #eraserange and #remove)


in this tutorial, I am using:
1 leaf green ROM
XSE
hex workshop
advance map

an explanation of the preprocessing directives(from the XSE guide)

Spoiler:

Clean the ROM from the last compiled script, as long as it was a dynamic one.

Usage:
#clean

Example:
#clean

which means, it takes the last script that was compiled to the ROM in question and undoes the change(useful when you notice an error just after you hit the compile button).

Overwrites a certain amount of bytes with the free space-byte value set, starting from the specified offset.

Usage:
#erase lOffset aValue

Example:
#erase 0x720000 0x64
0x64 (100) bytes from offset 0x720000 got overwritten with free space.

will erase a certain number of bytes after an offset and replace them with free space.

Overwrites a specified range of bytes with the free space-byte value set, starting from the first specified offset and ending at the second specified offset.

Usage:
#eraserange lOffset lOffset

Example:
#eraserange 0x720000 0x720500
0x1F4 (500) bytes from offset 0x7200000 to offset 0x720500 got overwritten with free space.

will replace all bytes between the two offsets with free space.

Removes the main part of a compiled script, if it's a valid one. Useful to gain free space and removing a spoilt script. By removing it means it's filled with the free space byte, either 0xFF/0x00.

Usage:
#remove lScriptOffset

Example:
#remove 0x16582F
The main part of the script at 0x16582F would be removed.

their are several variations on the #remove command(such as #removeall) but I will not discuss that here until some unforeseen reason presents itself. it can be used similar to the erase command but it is slightly harder use correctly.

before I begin the main section of this tutorial, I'd like to explain how I discovered this method, but its not really important so I'll put it in spoiler for all those interested.

Spoiler:

what lead me on to this was making a script and compiling it, then realizing i had missed a section. after fixing the script, compiling it again, and decompiling it, I was given the incomplete script again. after repeating the process several times I started to wonder what was stopping the script from being inserted. I opened up the hex viewer in XSE(found in the tools menu) and scrolled down to the space where the data was being written. when i compiled it again(with the viewer still open) it updated the free space, but not where I was writing to. this i quickly realized was because i was using dynamic scripts.

now dynamic scripts are great, but if they cause problems like this, heres how to fix it.

open the ROM in advance map, then use the "open script" button for where the script is compiled. click in XSE then go back to advance map and open the script again. this should open a second tab with the same script in it.

open the hex viewer from the tools menu and enter the offset where the script is written to. if you've been following good scripting format, there should be a couple blocks of space before where the script starts.if not, look for the first recognizable byte of the script. I'll give you an example script.

#dynamic 0x000001

'---------------
#org @start
checkflag 0x200
if 0x1 goto @snippet1
lock
msgbox @string1 MSG_NORMAL '"I've got something else t..."
applymovement MOVE_PLAYER @move1
release
end

'---------------
#org @snippet1
end


'---------
' Strings
'---------
#org @string1
= I've got something else to do


'-----------
' Movements
'-----------
#org @move1
#raw 0x10 'Step Down (Normal)
#raw 0xFE 'End of Movements

here the first byte I can see is the checkflag command. placing the cursor on the command and pressing F1 will bring up the command menu, which says that the hex value is 0x2B.

find the first instance of 2B in the hex viewer after the offset, and make sure it is followed by the flag number(remember that hex data is reversed) and it should something like {2B 0020} or {2B00 20} depending on where the script started.

close advance map and fire up hex workshop. scroll down to the rough area where the script should be(I haven't figured how to use the find function) and look for the same lines from the hexviewer. write this offset down, or store it in the notepad section of XSE(a very handy feature)


in XSE, flip to the other tab with the same script and edit it(be it fixing or destroying) and scroll the hexviewer down to the end of the free data. hit compile and watch where the script now ends. clicking on this block will display a number in the userbar(bar at the bottom of the window) that is the address(offset) of the last byte.

if you're going to take the easy route(you save time and lose experience) you can use the directives which I explained earlier.

you could calculate the number of bytes between the start offset and end offset to use the #erase function, or(because you already have the offsets handy), use #eraserange function instead

 #eraserange 0xstartoffset 0xendoffset

which will reenter FF or 00 bytes depending on your ROM.

for a more technical solution(or just to see what goes on behind the scenes in the erase function) I have provided this spoiler

Spoiler:

in hex workshop again, highlight from the first offset that we found(the beginning of the script) to the second offset that we found(where we edited the script). right click and press fill. it will bring up a dialog box. type in FF (or 00 depending on ROM) and press enter. when it asks you if you want to back up your ROM, hit yes.

you have now removed all of the data from the obsolete script.

if you want to replace the script, either write a new script or edit the decompiled(or leave it as it is if you are performing a recovery) one and recompile to the location you just freed. by doing this, you can recover space from multiple compiles and edit a script without wasting space, making a smaller patch file.


long live free space.

 

[Full Metal]

i know i'm not a mod but seriously though.
@gabe

Spoiler:

why can't it be serious? either way, that wasn't nescesary

anyways.
I thought it taught some good habits either way.

 

[HackMew]

Oh, my.... why do people like to NOT read the guide that comes with XSE? The newest version introduced many cleaning directives such as:

• #erase
• #eraserange
• #clean
• #remove

and more. The guide explains all of them.

 

[FoggyDoggy]

Yeah, the #erase directive is all you really ever need.

#erase 0x800000 0x64
That directive would erase 100 bytes from the offset 0x800000.

But like HackMew said...
# #eraserange 0x800000 0x800100
That would remove all bytes from 0x800100

#clean
Removes the last compiled script from the ROM, so long as it was a Dynamic Script. (Having the #dynamic directive at the header.)

#remove 0x800000
This directive would remove the main section of a script located at 0x800000 (Like how scripts have sections, this would delete the only one this points to...)

Please read the guide that comes with XSE, by loading XSE and pressing F2, OR, by navigating to your XSE folder, and double clicking 'Guide.chm'. If you are not able to see file extensions, (Like .chm, or .exe, or .txt, then just click the icon named 'Guide'.)
(LOL, not trying to stand you up HackMew, but I like giving gratuitous amounts of detail as often as possible...)

 

[jakerman999]

thiscovers a bit more than those functions. it includes instructions on how to remove scripts if they've been compiled multiple times, etc.

and while those functions serve they're purpose, this guide explains how to find what needs to be cleaned, then clean it. those four commands only clean.

 

[HackMew]

thiscovers a bit more than those functions. it includes instructions on how to remove scripts if they've been compiled multiple times, etc.

and while those functions serve they're purpose, this guide explains how to find what needs to be cleaned, then clean it. those four commands only clean.

Yet, those "four commands" aren't written anywhere, are they? Since you explain how to regain wasted free space, there's no reason not to include all the cleaning directives.

 

[jakerman999]

Yet, those "four commands" aren't written anywhere, are they? Since you explain how to regain wasted free space, there's no reason not to include all the cleaning directives.

alright, I updated the guide. have i appeased your sense of accomplishment yet or do have something else you wish for me to add?

 

[ME0W]

I have a question,
when I used the #eraserange command
on a script, like the one where you battle MAY
on ROUTE 103, the other scripts of some people went blank.
I mean, they have an offset but there are no commands
or anything.

So, when I talk to them in the game, they only show a blank
text box.

Like this:
EXAMPLE:
#eraserange 0x14EB92 0x14ED6B [MAY's SCRIPT]
Then I compiled it.
Then the one I mentioned above happened.
EXAMPLE:
Professor Birch's script
#org 0x800000 [I don't know the offset :)]








BLANK^
Did I do something wrong?
Or should have I used the #erase command instead?

Please help!
It seems that using this command affects other scripts too...

 

[HackMew]

I have a question,
when I used the #eraserange command
on a script, like the one where you battle MAY
on ROUTE 103, the other scripts of some people went blank.
I mean, they have an offset but there are no commands
or anything.

Did I do something wrong?
Or should have I used the #erase command instead?

Please help!
It seems that using this command affects other scripts too...

You better use #removeall instead of #eraserange if you don't know where the script starts and ends, exactly. Check the XSE guide.

 

[ME0W]

Originally posted by HackMew
You better use #removeall instead of #eraserange if you don't know where the script starts and ends, exactly. Check the XSE guide.

Thanks! It helped a lot! Now I feel a lot better...