rice.persistence
Class PersistentStorage

java.lang.Object
  rice.persistence.PersistentStorage

All Implemented Interfaces:

Catalog, Storage

public class PersistentStorage

extends java.lang.Object

implements Storage

This class is an implementation of Storage which provides persistent storage to disk. This class also guarentees that the data will be consistent, even after a crash. This class also provides these services is a non-blocking fashion by launching a seperate thread which is tasked with actaully writing the data to disk. This class was initially designed to support only Ids whose toString() method returns a String of constant length. It has been exteneded to support variable-length toString()s, but we have the caveat that not toString() can be a substring of another toString() - this will cause undefined behavior. The serialized objects are stored on-disk in a GZIPed XML format, which provides extensibility with reasonable storage and processing costs. Additionally, any metadata, if provided, is also stored in the on-disk file. The format of the file is [Key, Object, Version, Gzipped XML] [Metadata, Gzipped XML] [persistence magic number, long] [persistence version, long] [persistence revision, long] [metadata length, long] The persistence package is set up to automatically upgrade older versions of the on-disk format as new data is written under the key. Persistence also supports the metadata interface specified in the Catalog interface. All metadata is guaranteeded to be stored in memory, so fetching the metadata of a given key is an efficient operation.

Version:

$Id: pretty.settings,v 1.3 2005/03/11 20:22:33 jeffh Exp $

Author:

jeffh

Field Summary

static java.lang.String	BACKUP_DIRECTORY Static variables which define the location of the storage root
static java.lang.String	LOST_AND_FOUND_DIRECTORY DESCRIBE THE FIELD
static int	MAX_DIRECTORIES The maximum number of subdirectories in a directory before splitting
static int	MAX_FILES The splitting factor, or the number of files in one directory
static java.lang.String	METADATA_FILENAME DESCRIBE THE FIELD
static int	METADATA_SYNC_TIME The amount of time before re-writing the metadata file
static long	PERSISTENCE_MAGIC_NUMBER Static variables defining the layout of the on-disk storage
static long	PERSISTENCE_REVISION_2_0 DESCRIBE THE FIELD
static long	PERSISTENCE_REVISION_2_1 DESCRIBE THE FIELD
static long	PERSISTENCE_VERSION_2 DESCRIBE THE FIELD

Constructor Summary

PersistentStorage(IdFactory factory, java.lang.String rootDir, int size, Environment env)
Builds a PersistentStorage given a root directory in which to persist the data.

PersistentStorage(IdFactory factory, java.lang.String name, java.lang.String rootDir, int size, boolean index, Environment env)
Builds a PersistentStorage given and an instance name and a root directoy in which to persist the data.

PersistentStorage(IdFactory factory, java.lang.String name, java.lang.String rootDir, int size, Environment env)
Builds a PersistentStorage given and an instance name and a root directoy in which to persist the data.

Method Summary

boolean	exists(Id id) Returns whether or not an object is present in the location id .
void	flush(Continuation c) Method which is used to erase all data stored in the Catalog.
java.io.Serializable	getMetadata(Id id) Returns the metadata associated with the provided object, or null if no metadata exists.
java.lang.String	getName() Gets the name of this instance
void	getObject(Id id, Continuation c) Returns the object identified by the given id.
protected IdRange	getRangeForDirectory(java.io.File dir) Utility function which returns the range of keys which a directory corresponds to.
java.lang.String	getRoot() gets the root directory that the persistence Manager uses
int	getSize() Returns the number of Ids currently stored in the catalog
long	getStorageSize() gets the amount of storage that the persistence Manager uses
long	getTotalSize() Returns the total size of the stored data in bytes.The result is returned via the receiveResult method on the provided Continuation with an Integer representing the size.
void	rename(Id oldId, Id newId, Continuation c) Renames the given object to the new id.
IdSet	scan() Return the objects identified by the given range of ids.
IdSet	scan(IdRange range) Return the objects identified by the given range of ids.
java.util.SortedMap	scanMetadata() Returns a map which contains keys mapping ids to the associated metadata.
java.util.SortedMap	scanMetadata(IdRange range) Returns a map which contains keys mapping ids to the associated metadata.
java.util.SortedMap	scanMetadataValuesHead(java.lang.Object value) Returns the submapping of ids which have metadata less than the provided value.
java.util.SortedMap	scanMetadataValuesNull() Returns the submapping of ids which have metadata null
void	setMetadata(Id id, java.io.Serializable metadata, Continuation c) Updates the metadata stored under the given key to be the provided value.
boolean	setRoot(java.lang.String dir) Sets the root directory that the persistence Manager uses
boolean	setStorageSize(int size) Sets the amount of storage that the persistence Manager uses
void	setTimer(Timer timer) Method which allows the persistence root to schedle an event which will tell it to sync the metadata cached.
void	store(Id id, java.io.Serializable metadata, java.io.Serializable obj, Continuation c) Makes the object persistent to disk and stored permanantly If the object is already persistent, this method will simply update the object's serialized image.
void	unstore(Id id, Continuation c) Request to remove the object from the list of persistend objects.
protected void	writeDirty() Function which writes out all of the dirty metadata files and marks them as clean.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PERSISTENCE_MAGIC_NUMBER

public static final long PERSISTENCE_MAGIC_NUMBER

Static variables defining the layout of the on-disk storage

See Also:

Constant Field Values

PERSISTENCE_VERSION_2

public static final long PERSISTENCE_VERSION_2

DESCRIBE THE FIELD

See Also:

Constant Field Values

PERSISTENCE_REVISION_2_0

public static final long PERSISTENCE_REVISION_2_0

DESCRIBE THE FIELD

See Also:

Constant Field Values

PERSISTENCE_REVISION_2_1

public static final long PERSISTENCE_REVISION_2_1

DESCRIBE THE FIELD

See Also:

Constant Field Values

BACKUP_DIRECTORY

public static final java.lang.String BACKUP_DIRECTORY

Static variables which define the location of the storage root

See Also:

Constant Field Values

LOST_AND_FOUND_DIRECTORY

public static final java.lang.String LOST_AND_FOUND_DIRECTORY

DESCRIBE THE FIELD

See Also:

Constant Field Values

METADATA_FILENAME

public static final java.lang.String METADATA_FILENAME

DESCRIBE THE FIELD

See Also:

Constant Field Values

MAX_FILES

public static final int MAX_FILES

The splitting factor, or the number of files in one directory

See Also:

Constant Field Values

MAX_DIRECTORIES

public static final int MAX_DIRECTORIES

The maximum number of subdirectories in a directory before splitting

See Also:

Constant Field Values

METADATA_SYNC_TIME

public static final int METADATA_SYNC_TIME

The amount of time before re-writing the metadata file

See Also:

Constant Field Values

Constructor Detail

PersistentStorage

public PersistentStorage(IdFactory factory,
                         java.lang.String rootDir,
                         int size,
                         Environment env)
                  throws java.io.IOException

Builds a PersistentStorage given a root directory in which to persist the data. Uses a default instance name.

Parameters:

factory - The factory to use for creating Ids.

rootDir - The root directory of the persisted disk.

size - the size of the storage in bytes

env - DESCRIBE THE PARAMETER

Throws:

java.io.IOException - DESCRIBE THE EXCEPTION

PersistentStorage

public PersistentStorage(IdFactory factory,
                         java.lang.String name,
                         java.lang.String rootDir,
                         int size,
                         Environment env)
                  throws java.io.IOException

Builds a PersistentStorage given and an instance name and a root directoy in which to persist the data.

Parameters:

factory - The factory to use for creating Ids.

name - the name of this instance

rootDir - The root directory of the persisted disk.

size - the size of the storage in bytes

env - DESCRIBE THE PARAMETER

Throws:

java.io.IOException - DESCRIBE THE EXCEPTION

PersistentStorage

public PersistentStorage(IdFactory factory,
                         java.lang.String name,
                         java.lang.String rootDir,
                         int size,
                         boolean index,
                         Environment env)
                  throws java.io.IOException

Builds a PersistentStorage given and an instance name and a root directoy in which to persist the data.

Parameters:

factory - The factory to use for creating Ids.

name - the name of this instance

rootDir - The root directory of the persisted disk.

size - the size of the storage in bytes

index - Whether or not to index the objects

env - DESCRIBE THE PARAMETER

Throws:

java.io.IOException - DESCRIBE THE EXCEPTION

Method Detail

getMetadata

public java.io.Serializable getMetadata(Id id)

Returns the metadata associated with the provided object, or null if no metadata exists. The metadata must be stored in memory, so this operation is guaranteed to be fast and non-blocking.

Specified by:

getMetadata in interface Catalog

Parameters:

id - The id for which the metadata is needed

Returns:

The metadata, or null of non exists

getObject

public void getObject(Id id,
                      Continuation c)

Returns the object identified by the given id.

Specified by:

getObject in interface Catalog

Parameters:

id - The id of the object in question.

c - The command to run once the operation is complete

getTotalSize

public long getTotalSize()

Returns the total size of the stored data in bytes.The result is returned via the receiveResult method on the provided Continuation with an Integer representing the size.

Specified by:

getTotalSize in interface Catalog

Returns:

The total size, in bytes, of data stored.

getSize

public int getSize()

Returns the number of Ids currently stored in the catalog

Specified by:

getSize in interface Catalog

Returns:

The number of ids in the catalog

getRangeForDirectory

protected IdRange getRangeForDirectory(java.io.File dir)

Utility function which returns the range of keys which a directory corresponds to.

Parameters:

dir - The directory

Returns:

The RangeForDirectory value

getRoot

public java.lang.String getRoot()

gets the root directory that the persistence Manager uses

Returns:

String the directory for the root

getStorageSize

public long getStorageSize()

gets the amount of storage that the persistence Manager uses

Returns:

int the amount of storage in MB allocated for use

getName

public java.lang.String getName()

Gets the name of this instance

Returns:

String the name of the instance

setTimer

public void setTimer(Timer timer)

Method which allows the persistence root to schedle an event which will tell it to sync the metadata cached. Should be called exactly once after the persistence root is created.

Parameters:

timer - The timer to use to schedule the events

setMetadata

public void setMetadata(Id id,
                        java.io.Serializable metadata,
                        Continuation c)

Updates the metadata stored under the given key to be the provided value. As this may require a disk access, the requestor must also provide a continuation to return the result to.

Specified by:

setMetadata in interface Catalog

Parameters:

id - The id for the metadata

metadata - The metadata to store

c - The command to run once the operation is complete

setRoot

public boolean setRoot(java.lang.String dir)

Sets the root directory that the persistence Manager uses

Parameters:

dir - the String representing the directory to use

Returns:

boolean, true if the operation suceeds false if it doesn't

setStorageSize

public boolean setStorageSize(int size)

Sets the amount of storage that the persistence Manager uses

Parameters:

size - the amount of storage available to use in MB

Returns:

boolean, true if the operation suceeds false if it doesn't

rename

public void rename(Id oldId,
                   Id newId,
                   Continuation c)

Renames the given object to the new id. This method is potentially faster than store/cache and unstore/uncache.

Specified by:

rename in interface Catalog

Parameters:

oldId - The id of the object in question.

newId - The new id of the object in question.

c - The command to run once the operation is complete

store

public void store(Id id,
                  java.io.Serializable metadata,
                  java.io.Serializable obj,
                  Continuation c)

Makes the object persistent to disk and stored permanantly If the object is already persistent, this method will simply update the object's serialized image. This is implemented atomically so that this may succeed and store the new object, or fail and leave the previous object intact. This method completes by calling recieveResult() of the provided continuation with the success or failure of the operation.

Specified by:

store in interface Storage

Parameters:

obj - The object to be made persistent.

id - The object's id.

metadata - The object's metadata

c - The command to run once the operation is complete

unstore

public void unstore(Id id,
                    Continuation c)

Request to remove the object from the list of persistend objects. Delete the serialized image of the object from stable storage. If necessary. If the object was not in the cached list in the first place, nothing happens and false is returned. This method also guarantees that the data on disk will remain consistent, even after a crash by performing the delete atomically. This method completes by calling recieveResult() of the provided continuation with the success or failure of the operation.

Specified by:

unstore in interface Storage

Parameters:

id - The object's persistence id

c - The command to run once the operation is complete

exists

public boolean exists(Id id)

Returns whether or not an object is present in the location id .

Specified by:

exists in interface Catalog

Parameters:

id - The id of the object in question.

Returns:

Whether or not an object is present at id.

scan

public IdSet scan(IdRange range)

Return the objects identified by the given range of ids. The IdSet returned contains the Ids of the stored objects. The range is partially inclusive, the lower range is inclusive, and the upper exclusive. NOTE: This method blocks so if the behavior of this method changes and uses the disk, this method may be deprecated.

Specified by:

scan in interface Catalog

Parameters:

range - The range to query

Returns:

The idset containg the keys

scan

public IdSet scan()

Return the objects identified by the given range of ids. The IdSet returned contains the Ids of the stored objects. The range is partially inclusive, the lower range is inclusive, and the upper exclusive. NOTE: This method blocks so if the behavior of this method changes and uses the disk, this method may be deprecated.

Specified by:

scan in interface Catalog

Returns:

The idset containg the keys

scanMetadata

public java.util.SortedMap scanMetadata(IdRange range)

Returns a map which contains keys mapping ids to the associated metadata.

Specified by:

scanMetadata in interface Catalog

Parameters:

range - The range to query

Returns:

The map containg the keys

scanMetadata

public java.util.SortedMap scanMetadata()

Returns a map which contains keys mapping ids to the associated metadata.

Specified by:

scanMetadata in interface Catalog

Returns:

The treemap mapping ids to metadata

scanMetadataValuesHead

public java.util.SortedMap scanMetadataValuesHead(java.lang.Object value)

Returns the submapping of ids which have metadata less than the provided value.

Specified by:

scanMetadataValuesHead in interface Catalog

Parameters:

value - The maximal metadata value

Returns:

The submapping

scanMetadataValuesNull

public java.util.SortedMap scanMetadataValuesNull()

Returns the submapping of ids which have metadata null

Specified by:

scanMetadataValuesNull in interface Catalog

Returns:

The submapping

flush

public void flush(Continuation c)

Method which is used to erase all data stored in the Catalog. Use this method with care!

Specified by:

flush in interface Catalog

Parameters:

c - The command to run once done

writeDirty

protected void writeDirty()

Function which writes out all of the dirty metadata files and marks them as clean.