Mird reference manual

Contents

Functions

mird_initialize - initialize a database structure

MIRD_RES mird_initialize(char *base_filename,
			 struct mird **dest);
This allocates a new structure for a database object, and initializes it with the default values as well as the filename to use. You can change some values before you call mird_open, see struct mird.

mird_open - open the database

MIRD_RES mird_open(struct mird *db);
This opens the database. The structure to use in this call is created by mird_initialize.

Note that a reopening of a database not closed correctly can take some time, since it's cleaned before usage can start.

mird_sync - sync the database and journal

MIRD_RES mird_sync(struct mird *db);
This syncs the database, ie flushes any buffers, syncs against disk, and then starts to reuse any block marked as free. The last operation reads through the journal file for blocks that might be reused, so it can take some time depending on how big your journal file is. The journal file is recreated after this operation.

The sync operation fails if there is ongoing transactions, since no block can be reused and therefor the journal file can't be recreated. Call mird_sync_please if you don't know if you have transactions still in progress.

mird_sync_please - sync the database and journal

MIRD_RES mird_sync_please(struct mird *db);
This calls mird_sync immediately, if no transactions in progress. If there is, it sets a flag in the database which will call mird_sync when the last transaction is closed or cancelled. This is useful to call once in a while, for instance once per minute.

mird_close - close the database

MIRD_RES mird_close(struct mird *db);
This syncs the database (see mird_sync) and closes the database, marked as clean. It also frees the database structure.

Note that close cancels and unlinks any transaction objects still in progress - the db pointer in the structure sets to NULL - but does not free them. If you might have transactions left after a close, you need to call mird_tr_free to free the transaction structure.

mird_free_structure - frees the database structure

void mird_free_structure(struct mird *db);
This frees the database structure. This function is only to be used only in emergency situations, since the database isn't cleaned before close.

mird_free - frees a database buffer

void mird_free(unsigned char *buffer);
This frees any buffers that is the result of operations like
mird_key_lookup and mird_s_key_lookup.

mird_transaction_new - starts a new transaction

MIRD_RES mird_transaction_new(struct mird *db,
			      struct mird_transaction **mtr);
This creates a new transaction, within changes to the database are possible. A transaction can either succeed in total or fail in total.

mird_transaction_close - completes the transaction

MIRD_RES mird_transaction_close(struct mird_transaction *mtr);
This finishes of a transaction. This will return conflict exceptions if this transaction conflicts with any other. It also frees the transaction object (upon success).

mird_transaction_cancel - cancels the transaction

MIRD_RES mird_transaction_cancel(struct mird_transaction *mtr);
This cancels a transaction and frees the transaction object.

mird_tr_resolve - tries to resolve the transaction

MIRD_RES mird_tr_resolve(struct mird_transaction *mtr);
This can be called to synchronize the transaction with the current state of the database. It complains in the same manner as mird_transaction_close, but may be called more then once.

mird_tr_free - frees the transaction structure

void mird_tr_free(struct mird_transaction *mtr);
This frees a transaction object. This is only for those cases when the database is closed but you still have transaction structures. (For instance, when libmird is used in an object oriented environment.)

mird_key_store - store a key:value pair in a table

MIRD_RES mird_key_store(struct mird_transaction *mtr,
			mird_key_t table_id,
			mird_key_t key,
			unsigned char *value,
			mird_size_t len);
This writes down a key:value pair (where key is a 32 bit integer) in the database, in the given table.

If value is NULL, the key:value pair is removed from the table.

The table must preexist (mird_key_new_table) and be of the correct type for the operation to succeed.

mird_key_lookup - lookup a key in a table

MIRD_RES mird_key_lookup(struct mird *db,
			 mird_key_t table,
			 mird_key_t key,
			 unsigned char **data,
			 mird_size_t *len);
This looks up a key (integer) in a table. (*data) is set to NULL if the key didn't exist in the table.

The resulting data must be freed by a call to mird_free, if it isn't NULL.

The table must preexist (mird_key_new_table) and be of the correct type for the operation to succeed. (MIRDE_NO_TABLE and MIRDE_WRONG_TABLE)

See also mird_transaction_key_lookup and mird_s_key_lookup.

mird_transaction_key_lookup - lookup a key in a table

MIRD_RES mird_transaction_key_lookup(struct mird_transaction *mtr,
				     mird_key_t table_id,
				     mird_key_t key,
				     unsigned char **data,
				     mird_size_t *len);
This looks up a key (integer) in a table. (*data) is set to NULL if the key didn't exist in the table.

The resulting data must be freed by a call to mird_free, if it isn't NULL.

The table must preexist (mird_key_new_table) and be of the correct type for the operation to succeed. (MIRDE_NO_TABLE and MIRDE_WRONG_TABLE)

The difference against mird_key_lookup is that the lookup operation is transaction-local. This means the state when the transaction is created is kept, and unless mird_tr_resolve is called, no other transaction can change the state of the database from this transaction's point of view. This might be useful to do atomic operations, or to read back data in a transaction not yet closed (or to create a temporary data storage, to later cancel the transaction).

mird_key_new_table - create a new table

MIRD_RES mird_key_new_table(struct mird_transaction *mtr,
			    mird_key_t table_id);
This creates a new table in the database, for use with integer (32-bit) keys; mird_key_lookup and mird_key_store.

This fails if the table preexists (MIRDE_TABLE_EXISTS).

mird_delete_table - completely delete a table

MIRD_RES mird_delete_table(struct mird_transaction *mtr,
			   mird_key_t table_id);
This deletes a table and it's contents from the database.

The operation fails if the table doesn't exist (MIRDE_NO_TABLE).

mird_depend_table - add a table dependency

MIRD_RES mird_depend_table(struct mird_transaction *mtr,
			   mird_key_t table_id);
This adds a dependency on a table; no other transaction may change this table in any way. This method might be useful if the atomic result is depending on the contents of a whole table, but does not change it. It can be seen as a write-lock on a table.

mird_table_scan - reads entries from a table

MIRD_RES mird_table_scan(struct mird *db,
			 mird_key_t table_id,
			 mird_size_t n,
			 struct mird_scan_result *prev,
			 struct mird_scan_result **dest);
This is used to read all contents from an integer key table, n elements at a time.

If prev is NULL, the reading starts from the beginning of the table. Next time around, reuse the old dest as prev to continue reading.

(*dest) is set to NULL if there is no more to read. (*dest)->n contains the number of elements actually read, which may differ for string tables.

The table must exist and be of the correct type for the operation to succeed. (MIRDE_NO_TABLE and MIRDE_WRONG_TABLE)

The prev structure is freed upon call, otherwise it must be freed by calling mird_free_scan_result.

Example:

      struct mird_scan_result *scanner;
      MIRD_RES res;
      
      for (;;)
      {
   	 if ( (res=mird_table_scan(my_database,some_table,100,scanner,&scanner)) )
   	    [...do something with the error...]
      
   	 if (!scanner) break;  /* no more to read */
      
   	 [...do something with the scan result...]
      }
   

The key:value pairs read are not in any particular order.

mird_transaction_table_scan - reads entries from a table

MIRD_RES mird_transaction_table_scan(struct mird_transaction *mtr,
				     mird_key_t table_id,
				     mird_size_t n,
				     struct mird_scan_result *prev,
				     struct mird_scan_result **dest);
This is used to read all contents from an integer key table, n elements at a time.

If prev is NULL, the reading starts from the beginning of the table. Next time around, reuse the old dest as prev to continue reading.

(*dest) is set to NULL if there is no more to read. (*dest)->n contains the number of elements actually read.

The table must exist and be of the correct type for the operation to succeed. (MIRDE_NO_TABLE and MIRDE_WRONG_TABLE)

The prev structure is freed upon call, otherwise it must be freed by calling mird_free_scan_result.

The difference against mird_table_scan is that this reads at the transaction-local point of view, see mird_transaction_key_lookup.

mird_free_scan_result - frees scan result

void mird_free_scan_result(struct mird_scan_result *msr);
This frees the scan structure result from the mird_table_scan functions. See also mird_free_s_scan_result.

mird_s_key_store - store a string key:value pair in a table

MIRD_RES mird_s_key_store(struct mird_transaction *mtr,
			  mird_key_t table_id,
			  unsigned char *skey,
			  mird_size_t skey_len,
			  unsigned char *value,
			  mird_size_t len);
This writes down a key:value pair (where the key is a string of any size) in the database, in the given table.

If value is NULL, the key:value pair is removed from the table.

The table must preexist (mird_s_key_new_table) and be of the correct type for the operation to succeed.

mird_s_key_lookup - lookup a string key in a table

MIRD_RES mird_s_key_lookup(struct mird *db,
			   mird_key_t table_id,
			   unsigned char *key,
			   mird_size_t key_len,
			   unsigned char **data,
			   mird_size_t *len);
This looks up a key (string) in a table. (*data) is set to NULL if the key didn't exist in the table.

The resulting data must be freed by a call to mird_free, if it isn't NULL.

The table must preexist (mird_s_key_new_table) and be of the correct type for the operation to succeed. (MIRDE_NO_TABLE and MIRDE_WRONG_TABLE)

See also mird_transaction_s_key_lookup and mird_key_lookup.

mird_transaction_s_key_lookup - lookup a string key in a table

MIRD_RES mird_transaction_s_key_lookup(struct mird_transaction *mtr,
				       mird_key_t table_id,
				       unsigned char *key,
				       mird_size_t key_len,
				       unsigned char **data,
				       mird_size_t *len);
This looks up a key (string) in a table. (*data) is set to NULL if the key didn't exist in the table.

The resulting data must be freed by a call to mird_free, if it isn't NULL.

The table must preexist (mird_s_key_new_table) and be of the correct type for the operation to succeed. (MIRDE_NO_TABLE and MIRDE_WRONG_TABLE)

The difference against mird_key_lookup is that the lookup operation is transaction-local. This means the state when the transaction is created is kept, and unless mird_tr_resolve is called, no other transaction can change the state of the database from this transaction's point of view. This might be useful to do atomic operations, or to read back data in a transaction not yet closed (or to create a temporary data storage, to later cancel the transaction).

mird_s_key_new_table - create a new string key table

MIRD_RES mird_s_key_new_table(struct mird_transaction *mtr,
			      mird_key_t table_id);
This creates a new table in the database, for use with string keys; mird_s_key_lookup and mird_s_key_store.

This fails if the table preexists (MIRDE_TABLE_EXISTS).

mird_s_table_scan - read entries from a table

MIRD_RES mird_s_table_scan(struct mird *db,
			   mird_key_t table_id,
			   mird_size_t n,
			   struct mird_s_scan_result *prev,
			   struct mird_s_scan_result **dest);
This is used to read all contents from an string key table, approximately n elements at a time.

If prev is NULL, the reading starts from the beginning of the table. Next time around, reuse the old dest as prev to continue reading.

(*dest) is set to NULL if there is no more to read. (*dest)->n contains the number of elements actually read, which may differ for string tables.

The table must exist and be of the correct type for the operation to succeed. (MIRDE_NO_TABLE and MIRDE_WRONG_TABLE)

The prev structure is freed upon call, otherwise it must be freed by calling mird_free_s_scan_result.

Example:

      struct mird_scan_result *scanner;
      MIRD_RES res;
      
      for (;;)
      {
   	 if ( (res=mird_s_table_scan(my_database,some_table,100,scanner,&scanner)) )
   	    [...do something with the error...]
      
   	 if (!scanner) break;  /* no more to read */
      
   	 [...do something with the scan result...]
      }
   

The key:value pairs read is not in any particular order.

mird_transaction_s_table_scan - read entries from a table

MIRD_RES mird_transaction_s_table_scan(struct mird_transaction *mtr,
				       mird_key_t table_id,
				       mird_size_t n,
				       struct mird_s_scan_result *prev,
				       struct mird_s_scan_result **dest);
This is used to read all contents from an string key table, approximately n elements at a time.

If prev is NULL, the reading starts from the beginning of the table. Next time around, reuse the old dest as prev to continue reading.

(*dest) is set to NULL if there is no more to read. (*dest)->n contains the number of elements actually read, which may differ.

The table must exist and be of the correct type for the operation to succeed. (MIRDE_NO_TABLE and MIRDE_WRONG_TABLE)

The prev structure is freed upon call, otherwise it must be freed by calling mird_free_s_scan_result.

The difference against mird_s_table_scan is that this reads at the transaction-local point of view, see mird_transaction_s_key_lookup.

mird_free_s_scan_result - frees scan result

void mird_free_s_scan_result(struct mird_s_scan_result *mssr);
This frees the scan structure result from the mird_s_table_scan functions. See also mird_free_scan_result.

mird_free_error - frees MIRD_RES error structure

void mird_free_error(MIRD_RES error);
This frees the error structure contained in MIRD_RES, and must be called if the result from any call is non-NULL.

mird_describe_error - describes a MIRD_RES structure

void mird_describe_error(MIRD_RES error,char **dest);
This creates a human-readable (kinda) message from the error structure. The resulting string may be freed by calling mird_free.

mird_perror - prints error messages

void mird_perror(char *prefix,MIRD_RES error);
This prints out a human-readable message (same as from mird_describe_error) on stderr, preceeded by the given prefix string and a colon (":").

MIRD_IS_CONFLICT - test if an exception means conflict

int MIRD_IS_CONFLICT(MIRD_RES res);
This macro reports true (1) if the given errors means a conflict, as reported from mird_close.

Mird data types:

struct mird - the database object

This is the main database object. It is created by calling
mird_initialize.

Some parameters can be modified before calling mird_open, to either set up the behavior or the physical database parameters.

These are the physical database parameters, they may not be changed once the database is created: And some parameters that exists for optimization and to tune memory usage. Experts only: There really isn't any other user friendly contents of this structure.

See also: mird_initialize, mird_open, mird_close and mird_free_structure.

struct mird_transaction - the transaction object

This structure doesn't contain any user friendly information, although the element db may be checked to see if the transaction object is connected to a database (ie, use mird_transaction_cancel) or not (ie, use mird_free_structure. In the later case, it is set to NULL.

mird_key_t and mird_size_t - the database integer types

These two are typedef'ed to be 32 bit unsigned integers, always. They exist as special types and not as for instance "int", since the size of these types may vary between systems and compilers.

The 32 bit size is used in the physical database.

MIRD_RES - the error or result structure

MIRD_RES #defined as pointer to a struct mird_error, which contains information about what has happened. A value of NULL means that everything went as expected.
struct mird_error
{
   int error_no;
   char *s;
   unsigned x;
   unsigned y;
   unsigned z;
};

s, x, y and z elements contains different information depending on the given error number (in error_no). (See the error list.)

The mird_describe_error and mird_perror exists to get some readable text from this structure.

If not NULL, it must always be freed by calling mird_free_error.

struct mird_scan_result - result from mird_table_scan

struct mird_scan_result
{
   mird_size_t n;              - number of actual elements
   struct mird_scan_tupel      
   {
      mird_key_t key;          - tupel key
      unsigned char *value;    - value data 
      mird_size_t value_len;   - value data size
   } tupel[...];
};
Note that the value data may be unaligned in memory.

struct mird_s_scan_result - result from mird_s_table_scan

struct mird_s_scan_result
{
   mird_size_t n;              - number of actual elements
   struct mird_s_scan_tupel 
   {
      unsigned char *key;      - key data
      mird_size_t key_len;     - key data size
      unsigned char *value;    - value data
      mird_size_t value_len;   - value data size
   } tupel[...];
};
Note that both the key or the value data may be unaligned in memory.

Mird errors and exceptions:

Invalid arguments

MIRDE_INVALID_SETUP - invalid database parameters
MIRDE_TR_CLOSED - transaction is closed
MIRDE_TR_RUNNING - there is ongoing transactions
MIRDE_READONLY - database is readonly, s=func
MIRDE_NO_TABLE - no such table, x=table id
MIRDE_TABLE_EXISTS - table exists already, x=table id
MIRDE_WRONG_TABLE - wrong table type, x=table id

file open/create/lock general errors

s=filename, x=errno
MIRDE_OPEN - open failed
MIRDE_OPEN_CREATE - open/create failed
MIRDE_LOCK_FILE - didn't get lock on file
MIRDE_RM - failed to remove file
MIRDE_CREATED_FILE - special case returned from mird_open_file, * meaning the file was created (no other info)

database file errors

x=block no
MIRDE_DB_LSEEK - lseek failure, y=errno
MIRDE_DB_READ - read failure, y=errno
MIRDE_DB_READ_SHORT - read too few bytes, y=bytes, z=exp.
MIRDE_DB_WRITE - read failure, y=errno
MIRDE_DB_WRITE_SHORT - read too few bytes, y=bytes, z=exp.
MIRDE_DB_READ_CHECKSUM - checksum error in block read
MIRDE_DB_STAT - call to fstat failed, y=errno
MIRDE_DB_SYNC - call to fsync failed, y=errno
MIRDE_DB_CLOSE - call to close failed, y=errno

Fatal database errors

x=block no
MIRDE_WRONG_BLOCK - wrong block type; y=exp, z=got
MIRDE_ILLEGAL_FRAG - try to read an illegal frag; y=frag
MIRDE_ILLEGAL_FREE - link to a non-freelist block
MIRDE_WRONG_CHUNK - x=chunk id, wrong type; y=exp, z=got
MIRDE_CELL_SHORT - x=chunk id, cell chain broken early
MIRDE_SMALL_CHUNK - x=chunk id, y=needed bytes, z=type
MIRDE_DATA_MISSING - skey block doesn't contain the data it's supposed to

hashtrie errors; x=hashtrie root y=key z=current
MIRDE_HASHTRIE_RECURSIVE - too deep recursion (infinite?)
MIRDE_HASHTRIE_AWAY - led to key that doesn't match

opening an old database

while reading header, no other info
MIRDE_INVAL_HEADER - given file has an invalid header
MIRDE_UNKNOWN_VERSION - can't read that version of database
MIRDE_GARBLED - database garbled; need resurrection

journal file errors

MIRDE_JO_LSEEK - lseek failure, y=errno
MIRDE_JO_WRITE - write failure, y=errno
MIRDE_JO_WRITE_SHORT - read too few bytes, y=bytes, z=exp.
MIRDE_JO_TOO_BIG - journal too big, sync it
MIRDE_JO_READ - read failure, x=pos, y=errno
MIRDE_JO_MISSING_CANCEL - missing cancel marker just written
MIRDE_JO_SYNC - fsync/fdatasync failed, y=errno

reading back journal errors

MIRDE_MISSING_JOURNAL - dirty and missing journal file
MIRDE_VERIFY_FAILED - transaction verification failed

resource errors

MIRDE_RESOURCE_MEM - out of memory; x=bytes needed
MIRDE_RESOURCE_ERROR - failed to allocate error; x=b needed

resolving conflicts, x=table_id y=key

MIRDE_CONFLICT - conflict
MIRDE_DEPEND_BROKEN - broken dependency
MIRDE_CONFLICT_S - conflict, s=key, z=len
MIRDE_DEPEND_BROKEN_S - broken dependency, s=key, z=len
MIRDE_TABLE_DEPEND_BROKEN - broken table dependency, y=table

Index