osl logo

The lightweight storage library


[README] [INSTALL] [Quick Start] [API] [man page]

README

libosl, the object storage layer, is a library for storing small to medium-sized data in relational tables. It is easy to use, lightweight, fast and portable. libosl is suitable for applications that need only a small fraction of the features a full database management system provides.

Libosl is simple:

Libosl is fast:

Libosl is portable:

Libosl is open source:

Apart from the library itself, the package also contains simple examples and an fsck program, oslfsck, which can be used to recover from corrupted tables due to system crashes or bugs in the application or the library.

Resources:

Comments and bug reports are welcome. Please provide enough info such as the version of osl you are using and relevant parts of the logs.


INSTALL

Besides a working C compiler, gnu m4, gnu make and the lopsub library must be installed to to build and install this package.

If everything mentioned above is available on your system, type

    make

to build libosl. Then type

    sudo make install

to install. Finally, update the library cache by running

    ldconfig

The default installation prefix is /usr/local. Use

    make install PREFIX=/somewhere/else

to install in /somewhere/else instead.

The make targets in the web/ directory are used for web page generation and are not needed to compile, install or use libosl. In order to make these targets the following additional tools are needed:

The osl logo was created with Adobe Illustrator by Sebastian Schultheiss.


QUICK START

This document describes the steps which have to be performed in order to create and use an osl table. The code sniplets in this section are taken from the file osltar.c in the source distribution.

The complete API reference is contained in the file osl.h.

Define an enum that assigns descriptive names to the columns of your table. Example:

    enum osltar_columns {
        OTC_NAME,
        OTC_DATA,
        NUM_OT_COLUMNS
    };

The last element is is useful because the number of columns of your table must be specified later, see below.

Define an array of struct osl_column_description, one array member per column:

    struct osl_column_description tar_table_cols[] = {
        [OTC_NAME] = {
            .storage_type = OSL_MAPPED_STORAGE,
            .storage_flags = OSL_RBTREE | OSL_UNIQUE,
            .name = "filename",
            .compare_function = string_compare,
        },
        [OTC_DATA] = {
            .storage_type = OSL_MAPPED_STORAGE,
            .name = "data",
        },
    };

Three different storage types are available which may be selected on a per-column basis: OSL_DISK_STORAGE, OSL_MAPPED_STORAGE, and OSL_NO_STORAGE.

For columns of type OSL_MAPPED_STORAGE and OSL_NO_STORAGE an optional rbtree is maintained by the osl library which allows to quickly lookup rows by cell content. Whether or not an rbtree should be used must be specified in the storage_flags field which should contain the bitwise or of suitable osl_storage_flags.

If a column has an associated rbtree, i.e. if the OSL_RBTREE flag is set in the storage flags for the column, the compare_function field must be initialized to point to a function of type osl_compare_func. In this example, string_compare() is used, which is just a wrapper for strcmp() that interprets osl objects as C-strings and calls strcmp() on the object data.

Define a struct osl_table_description and initialize it with the number of columns of your table and the column descriptions:

    struct osl_table_description tar_table_desc = {
        .name = "tar_table",
        .num_columns = NUM_OT_COLUMNS,
        .column_descriptions = tar_table_cols,
        .dir = "/tmp/osltest"
    };

Create the table by calling osl_create_table():

    ret = osl_create_table(&tar_table_desc);

Open the newly created table by calling osl_open_table():

    struct osl_table *table;
    ret = osl_open_table(&tar_table_desc, &table);

To add a new row to the table, you must define an array of struct osl_object of length NUM_OT_COLUMNS which holds the contents of the new row. Note that an osl object is just a blob: It consists of a data pointer and a size value. Once the array has been initialized, pass it to osl_add_row() together with the table handle obtained from osl_open_table():

    struct osl_object objs[NUM_OT_COLUMNS];
    /* ...init the array... */
    ret = osl_add_row(table, objs);

Close the table with osl_close_table().

    osl_close_table(table, OSL_MARK_CLEAN);

The storage type of both columns of the table in this example is OSL_MAPPED_STORAGE, so you can later open the table again and retrieve its contents:

    ret = osl_get_row(table, OTC_NAME, &obj, &row);
    if (ret < 0) {
        fprintf(stderr, "osl_get_row(%s): %s\n", name,
            osl_strerror(-ret));
        return ret;
    }
    ret = osl_get_object(table, row, OTC_DATA, &obj);

The call to osl_get_row() uses the rbtree of the OTC_NAME column to find the row whose object in the OTC_NAME column matches the given object obj. If a row was found, it is passed to osl_get_object() which returns the object of the OTC_DATA column of this row.

This concludes the quick start document. Of course, libosl contains many more public functions than those used above. For details on the C-API, look at the file osl.h which contains the declarations of all public functions and the complete documentation of the public part of the library.

The “examples” subdirectory of the source distribution contains the full code of the above example and another small program which illustrates the use of columns of type OSL_NO_STORAGE. Larger applications using libosl are paraslash, a network audio streaming system, and adu, the advanced disk usage tool.


Man page

oslfsck


NAME

oslfsck - check and repair the tables of an osl database

SYNOPSIS

oslfsck [--help] [--detailed-help] [--version] [--loglevel=<severity>] [--database-dir=<path>] [--dump-dir=<path>] [--no-fsck] [--force] [--dry-run] [--] [table]...

DESCRIPTION

oslfsck tries to recover the tables of an osl database after a crash which potentially leaves the database in an inconsistent state.

OPTIONS

-h, --help

print help and exit

--detailed-help

print help, including all details, and exit

-V, --version

print version and exit

-l, --loglevel=<severity>

set loglevel (0-6)

default: 3

Log messages are always written to stderr while normal output goes to stdout. Lower values mean more verbose logging.

-d, --database-dir=<path>

full path to the database directory

Unless non-option arguments are given, all subdirectories of <path> are considered osl tables which oslfsck will try to fix.

-D, --dump-dir=<path>

enable dump mode

If path is non-empty, oslfsck will write a dump of all given tables to the specified path.

-n, --no-fsck

disable fsck mode

This is mainly useful in conjunction with the --dump-dir option.

-f, --force

enable force mode

Ignore the dirty bit when opening osl tables.

--dry-run

only report problems, don’t try to fix them