Parsing the Creatures 1 Graveyard file

 
One more victim of my file format reversing spree : the "The Graveyard" file containing all entries of C1 cemetery.

Parsing it will allow us to retrieve/edit/add all entries from this game file.

Understanding in game stores (Doctor's page and Aprodisiacs). aka "Health" and "Aphro" files.

There are two "stores" in Creatures 1, at least that's how I call them.

Those are the panels allowing you to inject additional items to the world through the Doctor's page and the aphrodisiacs page from the reproduction kit.

The C1 "Stores"

In this post I will show you reversing of those stores file formats, so we can replenish, parse, and edit them.

Parsing the Creatures 1 "TheRegister" file. (extracting Norn birthdates and owner information).

This will be a real quick article about parsing the Creatures1 "TheRegister" file.
This is like a primitive version of the C2 history files. It's the file that holds all creature information saved when you register a creature's birth.

They contain information about the Norn such as name, parents, date of birth and all owner info.

There's nothing really fancy about reversing and understanding the file format, so let's get this done.

Investigating Creatures 1 Teleporters

Alright, despite having loads of stuff in store, maybe I'll take a break from gory reversing stuff for this post, before I loose all of my readers (yeah, something like "both of them") :)

So why not take this opportunity to study something more generic? what about investigating how C1 teleporters work?

A happy Norn family, ready for exploration of time and space.

It only requires a bit of CAOS reading and your favorite C1 CAOS tool.

Extracting information from Creatures 1 PhotoAlbum files (And uncovering a 20 years old 2 pixel bug :)

In the last article, we've seen how to manually use the "dde: pict" CAOS command to take picture of our Norns the way the Owner's kit does it.

When doing it the usual way through the Owner's kit, all pictures are saved in individual photo album files for each Norn.
In this article I will show you all you need to know to extract all information from those file : timestamps, comments, and obviously, the picture data.

Repairing and understanding the lost "dde: pict" command (Taking photos of C1 Norns)

One promising nugget in the Creatures 1 CAOS reference, is the "dde: pict" command.

According to the documentation,

 Unfortunately, as many other things this command not only lacks critical bits of explanation but is also inherently broken (at least considering the above definition) and when used produces less than stellar results:

It's a shame we can't use this functionality in our own tools to take pictures as the Owner's kit does.
Or can't we ?

One of the main motives behind the site being uncovering such lost knowledge and abilities of the ancient Shee, come with me and find out how to understand and ultimately take advantage of this lost command.

Reversing the C1 .exp file format part 1 (General layout and extracting the genome)

It's been all those years, and despite a lot of talented persons having contributed to making Creatures tools or documenting the game various features, there still isn't any available description of the .exp file formant anywhere as we're nearing 2015.

The .exp files are the format in which creatures are exported or imported from and to the game.

Obviously, they contain about anything one could need to know about a given Norn, and exploiting them for writing custom tools might allow for a lot of interesting options such as exporting a critically ill norn, performing offline surgery on the file, and reimporting it into the game.

This would allow curing a couple otherwise fatal issues since the ingame mechanics don't provide any means of removing a given chemical from a creature bloodstream or reinvigorating degraded C2 organs.
(See the case of C2 grendels being oversensitive to cyanide poisoning because they lack the chemical reaction genes that would allow them to lower their cyanide levels)

In this series of articles, I will describe the process of reverse engineering the C1 (and later C2) .exp file format so any programmer out there can use the information to implement .exp file manipulating programs.

The first articles of the series will show the process followed so more people can learn "how it's done" and maybe later take on reversing other undocumented game file formats.
Hopefully, if enough information can be gathered about the format, a final article will sum everything up into the cleanest and most complete description possible.

For this first article, we will analyse the general file layout, and learn how to extract a working genome file from an .exp file.

Let's dive into binary DNA !

Parsing Creatures 1 .spr files (Extracting/Manipulating any of the game's graphics)

In a previous article I showed how to read Creatures 2 .s16 files to extract any of the game image data.
In this obvious follow up I will show you how to read out Creatures 1.spr files for the same result.

The Creatures 1 .spr file format is only marginally harder to parse than the .s16 format.
This is because the .spr files do not directly contain color information for each pixel, but rather an index number, to be used as a lookup index inside an external palette file to get the actual color data.
 
Thankfully, Python magic can once again save us most of the hard work and makes extracting an manipulating C1 images files a breeze.

Connecting to the Creatures DDE interface from python

We've recently seen how easy it is to connect to the C1 and C2 DDE interfaces using VB.

This time we will show how easy it is to connect to those interfaces using python, which will prove useful in upcoming articles.

Connecting to a DDE server from inside python is can be a quick and easy process.
The hardest part is in acquiring the correct modules to do so, (which can be a real pain if you take it from the wrong side), there are also a couple specifics that make debugging the first DDE connections unreliable.
Fortunately I've sorted those perks out for you,and will show you the single quickest and easy way to connect to the Creatures 1 and 2 DDE interface from python.

Parsing Creatures 2 .S16 files (Extracting/Manipulating any of the game's graphics).

"Back then" few easy image manipulation libraries were available, making the extraction of game image data an intricate process (that most creature fans/developers tackled in an easy but not very user friendly way as we will expose).Nowadays things got easier.

In this post I will explain the creatures 2 .s16 file format, and show you how you can now very easily extract any picture from the game using python.
Not only that but we can also harness the power of modern image manipulation libraries to convert the game sprites to any file format we like contrary to most legacy tools only proposing to export images as BMP's and letting us do the conversion work manually.

Connecting to the Game: accessing the DDE interface with 3 lines of code. (Tutorial)

We've been studying a lot of stuff lately, most of it involving running CAOS commands against C1 and C2 games.
C3 has a convenient built-in CAOS console, and various third party tools also propose that feature for C1 and C2.

But how would you go about implementing this kind of connectivity inside your own programs?
The simplest way to connect to a running C1 or C2 game is through the DDE interface (Dynamic Data Exchange). Also it REALLY is simple even if you barely know anything about programming, so this might be worth checking out.If you never programmed anything and don't have any development tools handy, you could initiate a DDE connection even from inside an Excel macro using VBA.Also the C1 and C2 DDE interfaces being strictly identical, you'll be writing code only once, and be able to use it in both games right now, without any change.

By sending the game some basic CAOS commands and reading their results you can reproduce all of the game functionalities and then some more: reading a Norn's needs and chemicals levels in realtime, selecting Norns and Grendels, carrying them, monitoring their brains,etc...

This time I show you a very quick overview to get you started in pumping CAOS in and out the C1 and C2 games.More advanced use cases or details will come in futher posts.
 

Investigating C1 climate (for a huge fail)

We all now C2 has an complex ecology model, where temperature,wind light, radiation and others such parameters dynamically shape the worlds face.

But did you know that basic ecology elements are also present in C1 to some extent?
Let's find out about those together.

Finally, a Creatures 2 Norn autopsy tool

So, we've just learnt how to extract various information from C2 History files, among which a list of concentrations for a Norn's bloodstream chemicals at time of death.

We've also learnt how to map a chemical's number to a readable chemical name.

We can finally get to perform autopsies of dead C2 Norns!

Getting C2 chemicals descriptions (chemicals.def file format)

What a sneaky bastard I am.
This was supposed to be a C2 Norn autopsy article! (I swear I'm not doing that to avoid awarding anyone the promised carrot.)

I just realised something was still missing from the last article.
In the C1 chemicals information article we were able to pull out both chemical names and descriptions from the chemicals.txt file.

So where's that information in C2 ?

Reversing and Parsing C2 allchemicals.str file format

Aright, we already covered how to obtain an up to date list of Chemical numbers and names inside our programs by extracting it from C1 game files.

Obviously this is an ability of much practical value for C2 too, so let's see how we could do that now.
I haven't found the allchemicals.str file format described anywhere so far, but the format is simple enough so reversing it should be Norn's play for us at this point :)

Back to more traditionnal automated Norn genealogy using python and graphviz ( Intro )

We've already had a glimpse at using advanced tools for experiments in Albian genealogy.
Gephi is a nice tool for interactively browsing decently sized datasets on the macro scale, but what about more conventional genealogy tools ?

In this post I will show you basic use of graphviz in python to make some more traditional but far more detailed family trees automatically.This will once again be only a short glimpse at graphviz possibilities so we can use it as an useful tool further down the road.
Hopefully it will make you want to add this to your Albian explorer toolbox.

Parsing C2 histfiles

Once again, a quick filler post about parsing a random creatures file format.

This time we will be investigating the Creatures 2 "History file" format.

 This one is also already documented online, but being able to parse it can allow us to extract some interesting data.

I guess this post needed at least one picture

Designing farting norns (Tutorial)

Many colored alternative breeds exist for all of the creatures games.

In this tutorial I will show you how to design a new breed for C1.
But not one that involves making new sprites for new colored or punk Norns, but rather one that implements new Norn behaviours and possibilities.
Using those techniques one could add loads of interesting behaviours to Norns such as :

• Adding a kicking stance and associated behaviours to fight off Grendel more efficiently
• Making Norns that can reinvigorate and heal by resorting to meditation
• Enabling actual telekinesis in Norns ( yes, this IS possible with some work )
• Making jumping Norns
• Mimicking more realistic life behaviour ( digestion, elimination ... )
• ...

But this time we will be:

Making Farting Norns:

Norns caesarean (saving the egg from a dead pregnant Norn)

I read a post today on (the awesome) Discover Albia blog and got slightly depressed by a random statement about the grief of loosing a pregnant Norn along with her egg.

Not that it happened that much to me, maybe once at worst.Yet this was the wake up call I needed to be ready should this ever happen.
So here's some CAOS black magic that will help you saving the egg from a pregnant Norn that passed away before getting to term.

Generating arbitray In-game eggs from CAOS commands, while also investigating the Norn reproductive system.

Okay, so in a previous article we've seen how to generate .egg files from genomes with a custom tool.
Even if it may be a nice skill to add to your toolbox, it might not be the most practical everyday tool to inject genomes in your game.

Wouldn't it be more practical if we could directly create ingame eggs from arbitrary genomes/couples as if they were just laid ? This is not only possible but also quite easy.

Some available tools such as the Genome injector on the awesome CDR website already perform that feat. ( Be sure to check out the whole site, as it is definitely the single most complete and comprehensive resource on writing C1 and C2 custom programs )

From there we could go the easy way, as Chris conveniently included a detailed description of the CAOS magic involved in spawning C2 eggs from genomes in the readme file the genome injector ( thanks for that Chris ! ).

But I'll rather show you how you're supposed do find out that information by yourself when starting from scratch, as it will serve us better in the long run when investigating other undocumented game features.
We will be doing this for C1, as Chris' tool only seems to work on Creatures 2.

We will use the CAOS language,it's documentation and some external tools to try to understand what exactly happens during the whole Norn reproductive process, so we can mimic it with a custom and more practical tool.

Parsing C2 GameLog files with python.

In this post we will see how to precisely parse a C2 Gamelog file.
This file is an ordered collection of all main events happening in Albia : Birth, Ageing, Dieing, getting a Name, Exports or Imports.
Parsing it can be useful to redraw a full timeline of stuff happening in your world, or to see which Norns are still alive or were ever used, if you need to perform some cleanup in the game folder ( by default, genomes and sprites files accumulate in the game folder for each Norn and are never removed )

Whatever your needs are, I hope yo will find this quick reference useful.

The Creatures 1 chemicals.txt file format.

For some odd reason, one of the most commonly usable file in the Creatures series, the Chemicals.txt file,  format is still totally undocumented in 2014.

Maybe somebody did document it and his work was lost in time ? Maybe I'm stupid and this is considered such common knowledge that nobody ever thought writing about it would be worth it ?
Anyway, I take the opportunity to publish the information here so nobody has to find it out by himself again if ever needed.

The Chemicals.txt file contains the list of all chemicals in game, their associated name, as well as their description (as used in the genetics kit)
It's a valuable resource when writing your own programs for dynamically establishing the various chemicals names without worrying about the various countries variations, or custom chemicals installed by various players.

26/01/2014 Update: Seems I've committed a small mistake (sorry, but I'm mostly writing all of that as I do it, so some things might need tuning afterwards)
Although The Chemicals.txt holds chemical description data, it's the chemicals.str file that holds the current actively used ingame chemicals list.
The chemicals.str file has exactly the same file format, except it doesn't include the chemicals descriptions and that each entry length prefix is only one byte long instead of 2.
This doesn't prevent you from using the Chemicals.txt file as a chemical description source, but the most up to date data synchronised with your game status is to be extracted from the chemicals.str file.

Generating Creatures 2 Egg files with Python

The Creatures 2 .egg file format is one of the few well documented aspects of the game series.

Why bother writing about it on a blog about investigating undocumented game aspects then ?
Well, the ability to generate .eeg files is a pretty simple skill that might prove an interesting addition for anybody writing Creatures tools.

The aim here is just to show how quick and easy it is

Introduction to advanced Norn genealogy using Social Network Analysis (SNA)

This post will be a quick taste of things to come.

I will show you where to look for Norn genealogy information in the game files, how to extract it using python, and then use it with more powerful tools than the built-in "family tree" of the original games to fully map out all of your Norn population.

We will only be scratching the surface of all those vast topics here, and each aspect of it will be the topic for a dedicated article further down the road. For now,I'm trying to get as much new and varied information out as I can. I hope this will bring some new topics, tools and knowledge to the Creatures community.

Investigating the effects of cyanide poisonning in Norns

While I was messing around C2 the other day, I was following a Grendel around in an unpopulated test world, and it suddenly dropped comatose with no warning signs.

All his organs had brutally stopped for no apparent reason,he had various food at hand and a full life bar minutes ago.A quick overview of it's bloodstream showed an obvious and fatal cyanide poisoning.So I took the opportunity to fully investigate the mechanisms of cyanide poisoning and the appropriate cure.

So what exactly happens during Cyanide poisoning in C2, and how do you counteract that really severe condition ?