REZ FileFormat Description by Stefan Bolder

Version 1.13     Date: 02nd November 1999     Email: S.Bolder@BlackAngel-Software.com



Contents


1. Intro
2. Overview Rez Fileformat.
3. Structures


1. Intro


In this document I will try to describe the REZ Fileformat used
by Shogo. I spent a couple of hours behind a Hex-Editor to get
all this info, so some mistakes could've creeped in there. If
you find an error, please let me know so I can update this description.
I will assume that people reading this will know how to program because
of the 'abstract' level of this document (just some offsets & structs).

Any remarks are more than welcome and if you have any questions feel
free to E-Mail me.

2. Overview of the Rez Fileformat.


REZ files consist of 3 distinct section, being:

This gives us the following view of the file:

Header
Data
Index


Header

 As said before the header holds a ID-string to identify itself as being a valid REZ File. This string reads something like this:

  "RezMgr Version 1 Copyright (C) 1995 MONOLITH INC."
   "LithTech Resource File "

The easiest way to write this string to a REZ file from within your own application would be to have the whole string as binary data which can then be directly written to your file. I Suggest using a hex editor to cut out the string from an existing rez file, and then including the string in your application resources as binary data. This because there are some special Hex characters in there, which you just can't type yourself.

Right after this ID-String comes what I call the 'variable part' of the header. In it we find offsets to the IndexBlock, Creation Date, and some other 'Variables'.
So lets take a look at the total overview of the header.

OFFSETDESCRIPTION
00-7EHEADER STRING (RezMgr .....)
7F-82Start Variable Section (0x01 00 00 00;int 1;)
83-86Offset of Root/Start-Folder in IndexBlock
87-8ALength of Root IndexBlock
8B-8E0x00 00 00 00
8F-92Offset to IndexBlock
93-96Date/Time
97-9A0x00 00 00 00
9B-9ELength of Longest Foldername
9F-A2Length of Longest Filename
A3-A70x00 00 00 00 01

NOTE:Please remember that the integer values are stored in 'reverse' (the correct term slipped my mind). If we have a value of 0x1A 2B 3C 4D, it will be written as:4D 3C 2B 1A in the file.
People who work with C or C++ will probaply have no problems just reading the integer value straight from the file, but I'm not sure if other programming languages will need to do a conversion.
I shall use the '0x00 00 00 00 notation when i mean the bytes in the file, and use an 'int' value (such as: int 1) to indicate the 'true' value.

At 0x83-86 we find the offset to the 'Root/Start-Folder' in the Index-Block of this rez file. This is where the Index-Block begins. The files & folders listed here are the ones that were in the folder that you specified when creating a Rez file: e.g 'd:\shogosrc\MyProject', with Lithrez.

After some other data and some 'fillers' (the 0x00 00 00 00's) we come across 2 interesting entries:

  The Longest Foldername Including the terminating zero.
  The Longest Filename Including terminating zero.

That this data is located in the rez file STRONGLY suggests, that the original programmers use only 2 buffers; char *Foldername; and char *Filename; which they allocate dynamically when Lithrez starts.
Offcourse you can make a whole character array of names if it works better for you, and if you want to remember which names are in the rez file you will probaply have to use an array of some sort.

NOTE:For people who don't understand the trailing zero principal:
it's actually quite easy: Every strings ends with a 0x00 (also written as '\0', which is 1 byte) value so the program knows how long the string is. So suppose we have a folder called: 'NEWTEXTURES' this would mean it's 11 + 1 = 12 (0xC) bytes long. Same goes for the Filename: if 'Ordog.abc' is the longest filename, it would be '6'! ('Ordog\0')

After all this we find more fillers and a 0x01 which marks the end of the header at offset 0xA7.

Data Block

 The Datablock basicly consists of all the files in the rez file, concatenated to eachother (like TAR on Unix) creating one big Data storage area. Without the Index-Block it would be impossible to tell where one file stops and another begins.

So make sure that when you create your own REZ files, that you keep track of which file you put at which offset. An easy method would be to loop through all the files, adding each one to the end of the DataBlock, and then storing the offsets for each file in memory. After this it's just a matter of writing the whole list of offsets & Filenames into the 'IndexBlock'.

Index Block

 Understanding what happens in the Index block can be a bit tricky at first, but after a bit of practice you'll see how it works. In the index we have offcourse the file & folder names, sizes, ID's and basicly everything else which we haven't written elsewhere.

There is basicly a 'Index Entry' for every Folder. In it is written which subfolders and files it has (and their offsets & sizes offcourse). To make clear to what sort of offset we are pointing (file or folder) there are 2 types of 'Index Entry Lines':
  1. File Entry Lines
      File Entries will point to the offset of a file in the Datablock, and has the length, ID, extension & filename.
  2. Folder Entry Lines
      A Folder entry points to the offset of another Index entry which is the index for that (Sub)Folder.

The problem now is: How do we tell one from the other?
The answer is offcourse quite simple. Every Index-Entry Line starts with a flag/an Identifier which tells us wether its a file or folder index.

If we find the value of: 0x00 0x00 0x00 0x00 it's a File (integer value 0)
If we find the value of: 0x01 0x00 0x00 0x00 it's a Folder (integer value 1)

The offsets(relative from beginning of the Index Entry) for a File Entry Line would like this:
OFFSETDESCRIPTION
00-03Identifier (File:0x00000000)
04-07Offset of the File
08-0BLength of File
0C-0FDate/Time stamp
10-14Numeric ID
15-18Filename Extension(REVERSED!) & trailing '\0'
19-1C0x00 00 00 00
1D-xxFilename with 2(TWO!) trailing '\0's

For a Folder entry the offsets look like this:

OFFSETDESCRIPTION
00-03Identifier (Folder:01 00 00 00; int value 1)
04-07Offset of Folder(SubIndex)
08-0BLength of SubIndex
0C-0FDate/Time stamp
10-xxFoldername with Ending '0x00'

NOTE: When Lithrez creates an entry for an empty folder, it can (read: will) happen that the offset is the same as the previous folder. The only way to check if it's empty is checking the Length. If the Length == 0, then the Folder is empty, so we don't have to recursivly jump to the offset given. If you forget this check, you will get lots of weird errors & stack overflows as we here have already encountered.

As you can see, all the strings also write their trailing '0', but with the Filename we actually write an extra one.
Also note that for some reason the Extension of a file is written BACKWARDS into the file.
If for instance you have a DLL file, you should write: 'LLD\0'

Following the above data we get the following picture for each individual Index & Sub Index Entry:
FolderEntry-1
FolderEntry-...
FolderEntry-n
FileEntry-1
FileEntry-...
FileEntry-n

As you can imagine, a recursive function would probaply be the best solution in a case like this.

Sub Index Filler Struct

This part must be the trickiest/weirdest part of the Index-Block. For every 'Sub-Index'(subfolder;so that's every index BELOW the root index) a filler is used, that is the total size of the 'Sub-Index' itself. If we find a reference to a SubIndex with a length of let's say 0xE6, we will find a filler of 0x00's with a length of 0xE6 right after this subindex! Thus the actual amount of used space would be 2*0xE6 = 0x1CC. It is somewhat hard to explain, but if you use these offsets and browse through one of the smaller shogo patches for instance you'll see what happens.
xx-..Filler 0x00's with the Length of the Subindex

3. STRUCTURES


There are offcourse numerous ways to design the structures needed to deal with REZ Files. The structures here are just a guide to get you going in the right direction. I start by 'skipping' the ID-String cause we don't need to keep that in memory.

typedef struct{
    long Start_Vars;
    long Root_Offset;
    long Offset_Length;
    long Blank1;
    long Index_Offset;
    long Date;
    long Blank2;
    long Max_Foldername;
    long Max_Filename;
    long Blank;
    BYTE Start;
} HEADER, *LPHEADER;

When we start reading from the index, I first read a long with the Identifier, and based on that
I used either a FILEINDEX Structure or a FOLDERINDEX structure:

typedef struct {
    long Offset;
    long Length;
    long Date;
    long ID;
    char Extension[4]; // it's in REVERSE! So DLL would be: LLD
    long Blank;
    char *Name; //2 trailing '\0's!
} FILEINDEX, *LPFILEINDEX;

typedef struct {
    long Offset;
    long Length;
    long Date;
    char *Name; //1 Trailing '\0'
} FOLDERINDEX, *LPFOLDERINDEX;

Here the data from the Header comes in handy: Longest Filename and Longest Foldername. We can safely define the length of the *Name variables with these values, without having to fear that they'll overflow.

These structures are not the only solution to reading the index, and I have used several other ideas in my own WinReZ program. It is however a good point to begin with and to start understanding the structure of the REZ files.


Well that was it. I hope this helps some of the people out there who want to develop their own Rez-Managers.
If you have any comments, suggestions or find any errors in this document feel free to Email me.

Stefan "Black Angel" Bolder
S.Bolder@BlackAngel-Software.com