The Creatures 1 chemicals.txt file format.

For some odd reason, one of the most commonly usable file in the Creatures series, the Chemicals.txt file,  format is still totally undocumented in 2014.

Maybe somebody did document it and his work was lost in time ? Maybe I'm stupid and this is considered such common knowledge that nobody ever thought writing about it would be worth it ?
Anyway, I take the opportunity to publish the information here so nobody has to find it out by himself again if ever needed.

The Chemicals.txt file contains the list of all chemicals in game, their associated name, as well as their description (as used in the genetics kit)
It's a valuable resource when writing your own programs for dynamically establishing the various chemicals names without worrying about the various countries variations, or custom chemicals installed by various players.

26/01/2014 Update: Seems I've committed a small mistake (sorry, but I'm mostly writing all of that as I do it, so some things might need tuning afterwards)
Although The Chemicals.txt holds chemical description data, it's the chemicals.str file that holds the current actively used ingame chemicals list.
The chemicals.str file has exactly the same file format, except it doesn't include the chemicals descriptions and that each entry length prefix is only one byte long instead of 2.
This doesn't prevent you from using the Chemicals.txt file as a chemical description source, but the most up to date data synchronised with your game status is to be extracted from the chemicals.str file.


Fortunately for us the file is of quite simple structure and can be easily reversed and parsed.

The C1 Chemicals.txt file in all it's glory

By just opening it in a Hex editor, it's structure is pretty obvious:

It contains no header whatsoever, but a series of entries of chemical names.
Each entry is of the format

<Entrylength ( 2 bytes ) ><Chemical Name (Entrylength long)>

There's no numbering scheme or indexation in the file whatsoever, entries are just read sequentially, and crammed into an array ( starting at 0, ending at 255 ) so you can later reference them by number in your program.Their number is defined as the position they appear at in the file.

The "Hunger" entry: a length of  6, coded on 2 bytes, followed by 6 bytes string containing the chemical name

Once the 256th entry is reached, there is an empty 4 byte string "\x00\x00\x00\xx00", serving as a delimiter to signal further entries are chemical descriptions.

Antigen 7 is the last chemical entry, followed by the "\x00\x00\x00\x00" marker, from now on all further entries are chemical descriptions

Chemical descriptions share the same format as the chemical names entries.

<Entrylength (2 bytes)><Chemical Description (Entrylength long)> 

They are also read sequentially, the first chemical description mapping to the first chemical, etc...




The final file format therefore is divided in three main blocs, chemical names, transition marker, chemical descritptions :


<1stEntrylength (2 bytes)><1stChemical Name (1stEntrylength long)>
<2ndEntrylength (2 bytes)><2ndChemical Name (2ndEntrylength long)>
...
<NthEntrylength (2 bytes)><NthChemical Name (NthEntrylength long)>
...
<256thEntrylength (2 bytes)><256thChemical Name (256thEntrylength long)

<\x00\x00\x00\x00 (4 bytes) marker signialing further entries are chemical descriptions > 

<1stEntrylength (2 bytes)><1stChemical Description(1stEntrylength long)>
<2ndEntrylength (2 bytes)><2ndChemical Description(2ndEntrylength long)>
...
<NthEntrylength (2 bytes)><NthChemical Description(NthEntrylength long)
...
<256thEntrylength (2 bytes)><256thChemical Description (256thEntrylength long)


Now we know that, we can simply parse the file within our programs to get an updated list of all ingame chemicals.

The following python script illustrates loading the Chemicals.txt file into a list, so each Chemical can be accessed by number; and his name and description be easily recovered:
( The example python file can be found here : ParseC1chemicalsfile_example.py )


import struct

number=0
Chemicals={}

f = open("Chemicals.txt", "rb")
try:
    byte = struct.unpack("H",f.read(2))[0]
    while byte != "\x00\x00":
         # We are reading a chemical entry
         if number < 256:
            Chemname = f.read(byte)
            Chemicals[number] = {"number":number,"name":Chemname}
            #print Chemicals[number]["number"],":",Chemicals[number]["name"]
            number+=1
            byte = struct.unpack("H",f.read(2))[0]

         # We are skipping the \x00\x00\x00\x00 marker
         elif number == 256:
            byte = struct.unpack("H",f.read(2))[0]
            number+=1

         # We are reading a chem description entry
         elif number < 513:
                Chemdesc=f.read(byte)
                Chemicals[number-257]["desc"]=Chemdesc
                number+=1
                byte = struct.unpack("H",f.read(2))[0]
        else:
                break
finally:
    f.close()

# printout chemicals recovered :
for chemical in Chemicals:
    print Chemicals[chemical]["number"],":",Chemicals[chemical]["name"],"(",Chemicals[chemical]["desc"],")"

Sure enough we get the expected result:

0 : <NONE> (  )
1 : Pain ( Drive levels... )
2 : Need for Pleasure (  )
3 : Hunger (  )
4 : Coldness (  )
5 : Hotness (  )
6 : Tiredness (  )
7 : Sleepiness (  )
8 : Loneliness (  )
9 : Crowded (  )
10 : Fear (  )
11 : Boredom (  )
12 : Anger (  )
13 : Sex Drive (  )
14 : not_allocated2 ( Un-used drive levels )
15 : not_allocated3 (  )
16 : not_allocated4 (  )...

Hope some of you will find that useful !