A smart(-ish) C1 scriptorium browser.

One of the hard aspects of trying to understand the C1 internals is trying to figure out which scripts exist for a given object.

C2 and C3, both have "Scriptorium browsers" available, that make it easy to visually browse ALL existing scripts for given classifiers.

Surprisingly, C1 seems to lack such a tool, and leaves us with no easy way to be sure we found all scripts governing a given object or behaviour.

Why is that so ? On what mechanisms do those scriptorium browsers operate ? And what can we do about it ?

Follow me for some technical exploration of the mysterious Shees CAOS library, and if you bear with me to the end of the article, a proper, exhaustive C1 scriptorium extractor will be the takeaway :)

 

The scripts basics:

As you already now, all "things" within a Creatures world, are arranged into broader families that represent their appartenance.
Those classifiers are written as a series of 3 values, representing  the object's or creatures family,genus, and specie.

For each of the classifiers, there are 255 slots that can contain a script.
Some of those slots numbers have special meanings, and are triggered by the game itself to produce effects of the various creatures or cobs.

For example the "2 6 3"  classifier corresponds to a C1 carrot (and translates as the "Simple"->"Food"->"Carrot" branch).

I use a small CAOS command I call the "Show me one" to easily identify which object corresponds to a given classifier (X Y Z being the classifier you're interested in) :


rtar X Y Z,sys: camt,sys: edit posl post posr posb

The command picks a random object of the given classifier, centers cam on it, and circles it in pink so you now where to look at.

The "1" script number is the universal script for "Activate1".

Therefore, if we retrieve the script number "2 6 3 1" we can now know what happens when a carrot gets activated.

This is cool and all, but it only works if you are targeting a known event for a known object.
How would we know if there were undocumented events available ?
What if you don't know the exact object you are looking for ?
The various games contain numerous scenery, or background or invisible objects. Some of them do stuff you can't easily guess about while some seem to but don't.
How do we find out?

For this, we need a way to EXHAUSTIVELY browse all scripts for given classifiers.

 

 

How a C2/C3 Scriptorium browser works:

As most of Creatures related tools, a scriptorium browser can be implemented by sending the game the appropriate CAOS commands.

Here are the relevant Creatures 2 commands that allow implementing a scriptorium browser.

TOTL family genus species : Returns the number of objects in the world that fit this specifier

DDE: GIDS ROOT: Returns a list of all the Family numbers that exist, separated with a space.

DDE: GIDS FMLY number: Returns a list of all genuses that exist from within a certain family number.

DDE: GIDS GNUS family number: Returns a list of all species that exist from within a certain family and genus number

DDE: GIDS SPCS family genus number: Returns a list of all events that exist from within a certain family genus and species number

DDE: SCRP family genus species event: Fetches a script from the scriptorium matching this specifier.

SCRP family genus species event : Install the rest of the Macro as the script for the given event

SCRX family genus species event: Removes the script matching this classifier from the scriptorium.

Armed with all those commands, implementing an exhaustive and fast scriptorium browser is trivial:

- First, use the "DDE: GIDS ROOT" command to get a list of all used family numbers

- Then, for each number of this list, use "DDE: GIDS FMLY" to get the list of all existing genuses for all existing families.

- Then again, use "DDE: GIDS GNUS" on all items of the list to get the list of all species that exist for all genuses, for all families.

- You get the drill, then it's "DDE: GIDS SPCS" ran for all known species, to get a list of all events that are associated with those known species.

You now have a complete, list of all scripts loaded inside the game, and only those.

From there it's just a matter of iterating through your list of classifiers and retrieving them with the "DDE: SCRP" command.

Easy enough, right ?

The Creatures 1 problem:

As of today, tools have been made that exploit most of exploitable commands available through the CAOS language.
Implementing a C2 scriptorium browser is trivial, and a logic thing to do to be able to more comfortably explore the game inner workings.

Then why hasn't anybody ever bothered making one for Creatures 1 ?

The answer is simple.

The Creatures 1 CAOS interpreter lacks all the "DDE: GIDS * " commands.
The game just doesn't provide you with any means of obtaining a list of all installed families, genuses, species or events.

Since you can't obtain a sure list of what exists and what does not, how could you know the list of scripts on which to run the "DDE:SCRP" to retrieve them ?

Are we bound to use "dumb" C1 scriptorium browsers that always list 255 of everything and let you manually sift through 90% emptiness ?

Implementing a C1 Scriptorium browser anyway:

Since the Creatures 1 engine doesn't provide us any tools to build a precise list of existing scripts, how could we establish it anyway ?

The first thought, quick and dirty approach would be to :

What if we just tried to query all possible events for all possible species for all possible genuses, for all possible families ?
The length of this last sentence should be the giveaway.

There are 255 potential families, within which are 255 potential genuses, each potentially containing 255 species, each of them potentially having 255 event scripts.

To query them all would require 4228250625 successive calls to "dde: scrp".
I'll spare you the calculations: it would take a couple days, assuming an optimised and quick way of sending the commands to the game.

Ok, I will admit it.At some point I thought I could get away with that approach and just let the extraction run for a couple of days.
The result is that even if you chose that option, the Creatures dde engine just doesn't support such an aggressive and long querying sequence and silently just stops answering after a couple hundred thousand calls before you even get to the most interesting scripts.

Being smarter about it

What now?
We will need to be slightly smarter about it.

How could we cut down that huge number of calls ?
Despite not providing the "DDE: GIDS *" commands, the Creatures 1 engine still provides the "TOTL" one.

-This command allows us to count how many objects of a given classifier exist in the world.
-We can use 0's as wildcards to specify "all of this branch".
-But unfortunately this doesn't allow us to count how many scripts are installed for a given specie.

Although it doesn't give us an exhaustive list right away, it will allow us to significantly cut down our number of tries.

4228250625 potential scripts might seem like a huge number of potential scripts, fact is that this library is veeeeeery sparsely populated, and that most of the game scripts are concentrated in a few "regions". Most, if not all scripts are in the Creatures, foods, toys and transporters branches, as well as some scenery stuff.

By simply identifying empty families/genuses/species, we already can avoid needlessly scanning huge chunks of the library.

The approach isn't perfect by any means, as for example, if you tried this on an empty world with no creatures, the "Totl" for the Norn specie would be 0, and we would therefore not obtain any Norn related script.

We will have to accept this fact, and consider that any realistic C1 scriptorium browser will only allow us to show scripts for all items that do exist in game at a given moment.But that's still better than nothing.

The procedure to extract the C1 scriptorium then becomes:

• Pause the game (we really don't want the number of existing objects to vary during our extraction.)
• Iterate through the 255 possible level 1 families with the "TOTL" CAOS command (0 0 0, to 255 0 0)
• Discard all families that reported 0 members, keep the other ones.
• For each Family having members , iterate through the 255 possible level 2 genuses with the "TOTL" command until we find enough to match the number of expected members for this family. Again, discarding all branches that announced having 0 children.
• Once again, running the "TOTL" command on all 255 possible species, on the established list of genuses that have more than 0 members.Always stopping as soon as we found as much child nodes as the parent category is expected to have.
• We end up with a list of all species that exist in our current game.No more shortcuts, we will now need to iterate over all 225 possible scripts for each item of this list and try to fetch them with the "DDE: SCRP" command to see if they exist.That is because we really want to be sure no "hidden" action slots exist.For a consumer grade application we could make some more assumptions as to script number ranges that just aren't ever used.

This should give us a pretty reasonable list of most of the game scripts (keeping in mind the mentioned limitations).One that is as good as it gets without further human tuning that is.

Closing thoughts

As always, you can find the python script used to illustrate this concept on the site's github.
I also put up a compiled version of the script here. So you can use to extract all scripts from your game yourself.

Expect at least 1 hour operation for it to complete.
It seems that the release version works in under 2 minutes in a real environnement :)

It will dump all scripts inside a directory as separate text files named after the script classifier.

Let me know if you run into any trouble trying the executable versions of the scripts.