# The FreePastry Tutorial.

This tutorial is designed to get you cooking quickly with the FreePastry API and software toolkit.

#### Version @tutorial_version@; @tutorial_date@. For FreePastry version @freepastry_version@. Maintained by @maintainer@.

# Lesson 7

## Past &mdash FreePastry's DHT.

### Download the tutorial files: MyPastContent.java, PastTutorial.java into a directory called rice/tutorial/lesson7/.

Past is FreePastry's Distributed Hash Table (DHT). This tutorial will show you how to get scribe up and running. You will learn how to do the following:

- Create a PastContent.
- Create a Past instance.
- Put an object in Past.
- Get an object from Past.
- A failed Get.

## Terms:

- Past—A large-scale, peer-to-peer archival storage facility. Past is FreePastry's DHT. It has insert() and lookup() operations.
- Content—A possible value of the DHT.
- IdFactory—A hash function. An IdFactory implements a hash function to build Ids that are compatable with pastry.
- PastryIdFactory—A commonly used IdFactory in Pastry. The PastryIdFactory uses SHA1 as it's underlying hash function.
- Storage—provides a local storage service for example, a persistence storage service or an in-memory storage service
- MemoryStorage—an implementation of Storage which provides in-memory storage. This class is specifically not designed to provide persistent storage.
- PersistentStorage—an implementation of Storage which provides persistent storage to disk.
- Cache—finite temporary storage for rapid access.
- LRUCache—implemetation of a least-recently-used (LRU) cache.
- StorageManager—a storage attached to a cache. Objects inserted and retrieved from the storage are auto-matically cached, to speed up future accesses.

### Create a PastContent.

Unlike Scribe, you do not need to write a client to utilize Past. However, you do need to create a content object to put/get. This is called a PastContent. The easiest way to create a PastContent is to extend rice.p2p.past.ContentHashPastContent. Here is the code for MyPastContent.

    public class MyPastContent extends ContentHashPastContent {
      /**
       * Store the content.
       *
       * Note that this class is Serializable, so any non-transient field will
       * automatically be stored to to disk.
       */
      String content;

      /**
       * Takes an environment for the timestamp
       * An IdFactory to generate the hash
       * The content to be stored.
       *
       * @param idf to generate a hash of the content
       * @param content to be stored
       */
      public MyPastContent(Id id, String content) {
        super(id);
        this.content = content;
      }

      /**
       * A descriptive toString()
       */
      public String toString() {
        return "MyPastContent ["+content+"]";
      }
    }

Note that the super constructor needs an Id. Other than that the class must be serializable.

### Create a Past instance.

Now on to the PastTutorial. The first part of PastTutorial looks identical to ScribeTutorial. The interesting part is creating the Past application.

          // used for generating PastContent object Ids.
          // this implements the "hash function" for our DHT
          PastryIdFactory idf = new rice.pastry.commonapi.PastryIdFactory(env);

          // create a different storage root for each node
          String storageDirectory = "./storage"+node.getId().hashCode();

          // create the persistent part
          Storage stor = new PersistentStorage(idf, storageDirectory, 4 * 1024 * 1024, node
              .getEnvironment());
          Past app = new PastImpl(node, new StorageManagerImpl(idf, stor, new LRUCache(
              new MemoryStorage(idf), 512 * 1024, node.getEnvironment())), 3, "");

Recall the PastryIdFactory is basically a hash funciton that produces Ids. The rest of the code sets up a typical PastImpl. It uses PersistentStorage for the primary store, and an LRUCache backed by a MemoryStorage as the cache.
Here is an overview of what is going on.
PastImpl needs the following:

- The Node
- StorageManager
- The number of Replicas to maintain — this is a parameter that you can tune to adjust overhead vs. chance of data-loss
- An instance name

StorageManagerImpl (the StorageManager) needs the following:

- An IdFactory
- A storage — we'll use the PersistentStorage
- A cache — we'll use the LRUCache

The PersistentStorage needs the following:

- An IdFactory
- The directory to put the storage root. — Note that we need a different storage root for each node, otherwise they will use the same one. This is only necessary because we are running multiple nodes in the same JVM.
- The max size (in bytes) — 1 Meg
- The environment

The LRUCache needs the following:

- A storage — we'll use a MemoryStorage
- The max size (in bytes) — 0.5 Meg
- The environment

### Put an object in Past.

Now lets store some data. The loop around this code is so we can later look up the data that was stored.

          // these variables are final so that the continuation can access them
          final String s = "test" + env.getRandomSource().nextInt();

          // build the past content
          final PastContent myContent = new MyPastContent(localFactory.buildId(s), s);

          storedKey[ctr] = myContent.getId();

          // pick a random past appl on a random node
          Past p = (Past)apps.get(env.getRandomSource().nextInt(numNodes));
          System.out.println("Inserting " + myContent + " at node "+p.getLocalNodeHandle());

          // insert the data
          p.insert(myContent, new Continuation() {
            // the result is an Array of Booleans for each insert
            public void receiveResult(Object result) {
              Boolean[] results = ((Boolean[]) result);
              int numSuccessfulStores = 0;
              for (int ctr = 0; ctr < results.length; ctr++) {
                if (results[ctr].booleanValue())
                  numSuccessfulStores++;
              }
              System.out.println(myContent + " successfully stored at " +
                  numSuccessfulStores + " locations.");
            }

            public void receiveException(Exception result) {
              System.out.println("Error storing "+myContent);
              result.printStackTrace();
            }
          });

Note that we call `Past.insert()` with our newly created MyPastContent object, and a Continuation. `Past.insert()` calls `receiveResult()` on the continuation with a Boolean[] that represents the success/failure of the individual stores for this object in the ring. Our continuation prints how many times this node was stored. Because our replication factor is 3, it should be stored 4 times (primary+3 replicas). If an error occurs it will call `receiveException()`.

### Get an object from Past.

We choose a random application and call Past.lookup()

          final Id lookupKey = storedKey[ctr];

          // pick a random past appl on a random node
          Past p = (Past)apps.get(env.getRandomSource().nextInt(numNodes));

          System.out.println("Looking up " + lookupKey + " at node "+p.getLocalNodeHandle());
          p.lookup(lookupKey, new Continuation() {
            public void receiveResult(Object result) {
              System.out.println("Successfully looked up " + result + " for key "+lookupKey+".");
            }

            public void receiveException(Exception result) {
              System.out.println("Error looking up "+lookupKey);
              result.printStackTrace();
            }
          });

### A failed Get.

Now let's see what happen if the Past doesn't have the requested content. Here's a hint, it calls `receiveResult()` with a null result.

        final Id bogusKey = localFactory.buildId("bogus");

    ...

        System.out.println("Looking up bogus key " + bogusKey + " at node "+p.getLocalNodeHandle());
        p.lookup(bogusKey, new Continuation() {
          public void receiveResult(Object result) {
            System.out.println("Successfully looked up " + result + " for key "+bogusKey+".  Notice that the result is null.");
          }

          public void receiveException(Exception result) {
            System.out.println("Error looking up "+bogusKey);
            result.printStackTrace();
          }
        });

When you run the code, your output will resemble:

    java -cp .:FreePastry-@freepastry_version@.jar rice.tutorial.lesson7.PastTutorial 9001 10.9.8.7 9001 5
    :1122487192406:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
    :1122487192421:No bootstrap node provided, starting a new ring...
    Finished creating new node SocketNodeHandle (<0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116])
    Finished creating new node SocketNodeHandle (<0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790])
    Finished creating new node SocketNodeHandle (<0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501])
    Finished creating new node SocketNodeHandle (<0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085])
    Finished creating new node SocketNodeHandle (<0x4AE14A..>/FOO/10.9.8.7:9005 [-7392240638718374041])
    Storing 5 keys
    Inserting MyPastContent [test1382830848] at node [SNH: <0x22D2E1..> -> <0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116]]
    Inserting MyPastContent [test-618483902] at node [SNH: <0xED50E1..> -> <0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085]]
    Inserting MyPastContent [test482928483] at node [SNH: <0x22D2E1..> -> <0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116]]
    Inserting MyPastContent [test-919775810] at node [SNH: <0x4AE14A..> -> <0x4AE14A..>/FOO/10.9.8.7:9005 [-7392240638718374041]]
    Inserting MyPastContent [test1099145466] at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
    MyPastContent [test1382830848] successfully stored at 4 locations.
    MyPastContent [test-618483902] successfully stored at 4 locations.
    MyPastContent [test482928483] successfully stored at 4 locations.
    MyPastContent [test-919775810] successfully stored at 4 locations.
    MyPastContent [test1099145466] successfully stored at 4 locations.
    Looking up the 5 keys
    Looking up <0x9515FC..> at node [SNH: <0x16C2C6..> -> <0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501]]
    Looking up <0xB987B7..> at node [SNH: <0x16C2C6..> -> <0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501]]
    Looking up <0x999A32..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
    Looking up <0xC9B06E..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
    Looking up <0xC2EF27..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
    Successfully looked up MyPastContent [test1099145466] for key <0xC2EF27..>.
    Successfully looked up MyPastContent [test-618483902] for key <0xB987B7..>.
    Successfully looked up MyPastContent [test482928483] for key <0x999A32..>.
    Successfully looked up MyPastContent [test-919775810] for key <0xC9B06E..>.
    Successfully looked up MyPastContent [test1382830848] for key <0x9515FC..>.
    Looking up a bogus key
    Looking up bogus key <0x216ACE..> at node [SNH: <0xED50E1..> -> <0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085]]
    Successfully looked up null for key <0x216ACE..>.  Notice that the result is null.

Also 5 directories were created, each containing a FreePastry-Storage-Root subfolder:

    ls storage*
    storage-1404778543:
    FreePastry-Storage-Root

    storage-1501429183:
    FreePastry-Storage-Root

    storage-1862004272:
    FreePastry-Storage-Root

    storage2042841361:
    FreePastry-Storage-Root

    storage848913672:
    FreePastry-Storage-Root