# The FreePastry Tutorial.

This tutorial is designed to get you cooking quickly with the FreePastry API and software toolkit.

#### Version 1.3; August 1, 2005. For FreePastry version 1.4.2. Maintained by Jeff Hoye.

## Contents

### Getting Started

- What You'll Need

### Lessons

- Lesson 0.a: Continuations.
- Lesson 0.b: Environment.
- Lesson 1: Minimal code to create/join a pastry ring.
- Lesson 2: Execute the code to launch your new ring.
- Lesson 3: Write a simple application using the commonAPI. (this will show you how to send and receive messages)
- Lesson 4: Running multiple nodes in the same JVM.
- Lesson 5: Scheduling tasks on the FreePastry timer.
- Lesson 6: Introducing Scribe.
- Lesson 7: Past &mdash FreePastry's DHT.

### Coming soon:

- Lesson 8: Splitstream.
- Lesson 9: Advanced Scribe: Policies.
- Lesson 10: Advanced Environment.

## What You'll Need

Before you begin, you should meet the following requirements.

1. An intermediate level of knowledge in the Java programming language. The Java Tutorial is a good place to start.
2. A copy of a current JDK, 1.4.2_06 or later. Sun's J2SE.
3. A basic understanding of Pastry overlay structure. Read the short overview [pdf][ps] of FreePastry.
4. Have a working copy of FreePastry-1.4_01 or later. You can simply download the current jar, or download and build the source.

# Lesson 0.a

## Continuations.

FreePastry and its applications use Continuations in various places, so it is important that you understand what they do and how to use them.
A continuation is similar to a callback, or in java, a Listener, but it is typically used only once. A google search for continuations will show you a variety of uses in computer science, but the primary use for them in FreePastry is to handle network latency, or other types of IO that may block or take a significant amount of time.
To understand why we use continuations, let's look at an alternative style of calls. RMI/RPC style calls are generally blocking calls. Blocking calls work similar to a regular function call. The lookup command in Past (FreePastry's DHT) could look like:

      public Result lookup(Id id) {
        // blocking network call
      }

But if the network call takes a while to execute due to latency, then your program will block until it completes. In the meantime you could be doing other lookups, but because of this style of call, you are forced to wait, or use a lot of threads. Continuations allow you to "continue" processing when the result arrives, and in the meantime issue other requests.
Let's look at the `rice.Continuation` interface.

    /**
     * Asynchronously receives the result to a given method call, using
     * the command pattern.
     *
     * Implementations of this class contain the remainder of a computation
     * which included an asynchronous method call.  When the result to the
     * call becomes available, the receiveResult method on this command
     * is called.
     *
     * @version $Id: index.html 2716 2005-08-10 15:42:38Z jeffh $
     *
     * @author Alan Mislove
     * @author Andreas Haeberlen
     */
    public interface Continuation {

      /**
       * Called when a previously requested result is now availble.
       *
       * @param result The result of the command.
       */
      public void receiveResult(Object result);

      /**
       * Called when an execption occured as a result of the
       * previous command.
       *
       * @param result The exception which was caused.
       */
      public void receiveException(Exception result);
    }

In Past (the DHT that runs on top of FreePastry) the code to do a lookup looks like this:

      public void lookup(Id id, Continuation command) {
        // non-blocking network call
      }

Note the differences between this call and the alternative approach mentioned before:

1. The method takes a Continuation.
2. There is no return value.

When the result arrives, Past will call `command.receiveResult()` with the response. Inside the continuation, you can complete the task that required the lookup. If instead, an error occurs, Past will call `command.receiveException()`. Note that because `receiveResult()` is called with an Object, you must cast the result to what you are expecting from the lookup.
Here is an example implementation of a continuation MyContinuation.java:
First, create the continuation :

    class MyContinuation implements Continuation {
      /**
       * Called when the result arrives.
       */
      public void receiveResult(Object result) {
        PastContent pc = (PastContent)result;
        System.out.println("Received a "+pc);
      }

      /**
       * Called if there is an error.
       */
      public void receiveException(Exception result) {
        System.out.println("There was an error: "+result);
      }
    }

Note that the continuation has 2 methods it must implement: `receiveResult()` and `receiveException()`. This continuation will print "Received a blah" when the result comes in. Note that we need to cast the result to what we should be expecting. In the case of Past, we know that the result will be a PastContent. If there is an error, we will print "There was an error: Description of the error"
Here is the code to use our continuation. Note that this code will not run, because Past and Id have not been initialized. (You will learn how to do that in Lesson 7).
Download the code here: TestContinuation.java

    public class TestContinuation {
      public static void main(String[] args) {
        Past past = null; // generated elsewhere
        Id id = null; // generated elsewhere

        // create the continuation
        Continuation command = new MyContinuation();

        // make the call with the continuation
        past.lookup(id, command);
      }
    }

This code calls lookup with the continuation. When the result arrives, `command.receiveResult()` will be called.

### Anonymous inner classes

Java hackers love anonymous inner classes, so here is the same code as an anonymous inner class, rather than a seperate class. This is handy because it flows a bit more like a typical blocking program. First do a `past.lookup()` then `System.out.println()`.
Download the code here: TestContinuationAnon.java

    public class TestContinuationAnon {

      public static void main(String[] args) {
        Past past = null; // generated elsewhere
        Id id = null; // generated elsewhere

        // same code as TestContinuation and MyContinuation combined
        past.lookup(id, new Continuation() {
          // will be called if success in the lookup
          public void receiveResult(Object result) {
            PastContent pc = (PastContent)result;
            System.out.println("Received a "+pc);
          }

          // will be called if failure in the lookup
          public void receiveException(Exception result) {
            System.out.println("There was an error: "+result);
          }
        });
      }
    }

Click here for more information on Java Anonymous Inner Classes.

# Lesson 0.b

## Environment

The environment allows us to more easily virtualize pastry inside the same Virtual machine. This is the main new feature of FreePastry 1.4.2. The environment serves FreePastry in 6 key ways:

- Logging &mdash We have standardized on a logging system throughout FreePastry. This is a simpler logging framework than the one that ships with java. However it is compatible. (In other words, you can implement our LogManager and Logger with the Java one.) This logging replaces the previous "-verbose" etc. logging.
- Parameters &mdash Formerly hard-coded (`public static final`) parameters are now able to be specified at startup, or even changed during runtime rather than requiring a recompile. It allows parameters to be changed from code, and to be persistent. In other words, you could have a gui that lets you change the parameters, then store them to disk (By calling Parameters.store()), so next time the code is run, it retains the parameters.
- SelectorManager &mdash You can control which nodes use which SelectorManager. The SelectorManager is a single thread that handles all network IO and Timer events. This model usually gives improved performance, and simpler synchronization than several threads. Note that the SelectorManager is a Daemon thread that will cause the JVM to not exit when the main method ends. See below to find out how to destroy the SelectorManager.
- Processor &mdash It is important that tasks done on the SelectorThread don't take long, as this is the network IO thread, and can cause other nodes to suspect you are faulty. However, sometimes you need to do a CPU intensive task (like calculate a manifest for a bloom filter) that will cause problems if done on the SelectorThread. For these tasks, you can use the "Processor. This is simply a different thread that will be context switched automatically by the system. Typically, you will access this by using the Node.process() or Endpoint.process() function call. These calls result in a call to the Processor. Future implementations could use more Threads for computers with several processors, or hyper-threading.
- TimeSource &mdash FreePastry and its apps now call TimeSource.currentTimeMillis() rather than System.currentTimeMillis(). This will (for example) allow you to run a FreePastry node on a clock that is offset from your computers clock. This can be particularly helpful in a situation like Planetlab where some nodes have incorrect system clocks due to misconfigured NTP servers.
- RandomSource &mdash Has the same interface as java.util.Random, but Allows for easier reproducability of some errors. You can seed the RandomSource with the same parameter so you can reproduce conditions.

Pastry applications access the Environment from the PastryNode or the commonapi Node (PastryNode implements Node). Generally you can use the same Environment for everything inside your JVM. The two Objects that you will likely construct with an Environment are the PastryNodeFactory and the IdFactory.

### How should I get an environment?

- In your main method simply call `new Environment()`.
- In your application, call `node.getEnvironment(), or endpoint.getEnvironment()`.

The default constructor for an environment looks in the classpath for a params file called freepastry.params. If you compiled the jar with the included ant task, or used the jar from the web, this should work automagically. If you are compiling FreePastry from an IDE (such as eclipse) or from the command line (javac) then you will need to make sure to put the freepastry.params file located in jars/freepastry in your classpath, or use a different contstructor for the Environment.

### My JVM won't exit, or How should I get rid of an environment?

The SelectorManager and Processor both have daemon threads that are used to handle communication and processing for the PastryNode. For the JVM to be able to exit, you will need to properly destroy the environment, by calling environment.destroy(). This will call store on the parameters, and destroy the selector/processor threads. Of course, you can also call System.exit(), but that is generally considered bad form.
This should be sufficient information for you to continue with the tutorial. You can learn more in Lesson 10 Advanced Environment.

# Lesson 1

## Minimal code to create/join a pastry ring.

### Download the tutorial file DistTutorial.java.

In this lesson we are going to use Socket transport layer to communicate. The Socket transport layer is a package within the pastry source tree that is used to communicate between nodes using Internet Protocol (IP). It is located in rice.pastry.socket. Socket uses TCP for all messages except liveness which it uses UDP to accomplish. Before we dive into the code lets start with a short vocabulary lesson:

## Terms:

- NodeId — The randomly assigned unique identifier of a Node in the FreePastry. Typically a NodeId is 160bit represented by 20 Hex digits. We usually only display the first 6 digits as this is enough to uniquely identify nodes for a pretty large ring.
- NodeIdFactory — This generates your local NodeId. Why do we need a NodeIdFactory? In a real deployment of Pastry, it is critical that one cannot choose their NodeId. To accomplish this you may want to use a Certificate Authority to centrally assign NodeIds. However, for your initial purposes one doesn't need to be able to secure the choice of NodeId. The factory pattern gives us the flexibility we need to change this behavior in the future. Thus we begin with RandomNodeIdFactory which generates a Random node id.
- PastryNode — This is a Node in the network. Your application will send messages through the node and the node will deliver messages to your application.
- NodeHandle — This is a "reference" to a PastryNode. It is how you refer to a specific node in the network. A NodeHandle consists of a NodeId and whatever information the transport layer needs to find the node in the underlieing protocol. In this case an IP address and port. In this lesson, you will use a NodeHandle to bootstrap your node into the ring. You can get the NodeHandle to your local node by calling PastryNode.getLocalHandle(). In the distributed environment, you will need to get a NodeHandle to your bootstrap node(s) by asking the transport layer. In this case you are interested in acquiring a node handle from an IP address and port. The the Socket transport uses this information to opens a socket to this address:port and requests a copy of the NodeHandle from the remote node. If it doesn't find a node at this location it returns null.
- Bootstrap Node — The node you use to join the ring. When a node starts up, it can either join an existing ring, or start a new one. If you don't have a ring yet, then your choice is narrowed down to starting your own. But once your first node has started a ring, you probably want all new nodes join that one. To join a ring, you need to bootstrap off of any one node in the existing ring. This is the bootstrap node. In the example below, we use the first node that we construct as the bootstrap node. In a real deployment, you may want to cache several nodes so that if you are unable to boot off of the first one, you can try others.
- PastryNodeFactory — Constructs and initializes the Pastry Node. This sets up the PastryNode including the transport layer, the Leafset Maintenance protocol and, the RoutingTable Maintenance protocol. These protocols are necessary for a FreePastry deployment to properly maintain the overlay structure. In this example we will use the SocketPastryNodeFactory which as its name implies sets up the pastry node to use the Socket transport layer to communicate with other nodes.
- Daemon thread — Prevents the JVM from exiting after the main method is complete. See java.lang.Thread for more information.

Now that you have the proper vocabulary, here is the constructor for the DistTutorial. You can see most of the elements described in the vocabulary.

      /**
       * This constructor sets up a PastryNode.  It will bootstrap to an
       * existing ring if it can find one at the specified location, otherwise
       * it will start a new ring.
       *
       * @param bindport the local port to bind to
       * @param bootaddress the IP:port of the node to boot from
       * @param env the environment for these nodes
       */
      public DistTutorial(int bindport, InetSocketAddress bootaddress, Environment env) throws Exception {

        // Generate the NodeIds Randomly
        NodeIdFactory nidFactory = new RandomNodeIdFactory(env);

        // construct the PastryNodeFactory, this is how we use rice.pastry.socket
        PastryNodeFactory factory = new SocketPastryNodeFactory(nidFactory, bindport, env);

        // This will return null if we there is no node at that location
        NodeHandle bootHandle = ((SocketPastryNodeFactory)factory).getNodeHandle(bootaddress);

        // construct a node, passing the null boothandle on the first loop will
        // cause the node to start its own ring
        PastryNode node = factory.newNode(bootHandle);

        // the node may require sending several messages to fully boot into the ring
        while(!node.isReady()) {
          // delay so we don't busy-wait
          Thread.sleep(100);
        }

        System.out.println("Finished creating new node "+node);
      }

### Let's examine each line:

1. The arguments that we start with are:
   1. `int bindport` — the local port to bind to.
   2. `InetSocketAddress bootaddress` — The address of our bootstrap node.
   3. `Environment env` — The environment. See lesson 0.b.

       public DistTutorial(int bindport, InetSocketAddress bootaddress, Environment env) throws Exception {

2. We begin by constructing our NodeIdFactory, which we are going to need to give to our PastryNodeFactory.

       NodeIdFactory nidFactory = new RandomNodeIdFactory(env);

3. Give the nidFactory to our SocketPastryNodeFactory. Additionally, we need to tell the SocketPastryNodeFactory what port to bind our pastry node to.

       PastryNodeFactory factory = new SocketPastryNodeFactory(nidFactory, bindport, env);

4. Turn the bootaddress into a NodeHandle. Note that this call blocks, and can take several seconds to complete. It is opening a socket to the specified address. If there is an error, or no PastryNode found at the bootaddress, then `getNodeHandle()` returns null.

       NodeHandle bootHandle = ((SocketPastryNodeFactory)factory).getNodeHandle(bootaddress);

5. Finally, create the PastryNode. If bootHandle is null then the factory will start a new ring.

       PastryNode node = factory.newNode(bootHandle);

6. Even though we don't have an application to run yet, it is important that you are aware of the call to `PastryNode.isReady()`. This method returns false until the node is fully booted into the ring (which entails establishing his neighbor set i.e. the routing table and the leafset). This simple loop is a typical way to wait for the node to fully boot into the ring.

       while(!node.isReady()) {
         Thread.sleep(100);
       }

Before we can run our program lets take a quick look at the `main()` method. We need to get 1) the local port to bind to, 2) the IP address of the bootstrap node, and 3) the port of the bootstrap node.

      /**
       * Usage:
       * java [-cp FreePastry-.jar] rice.tutorial.lesson1.DistTutorial localbindport bootIP bootPort
       * example java rice.tutorial.DistTutorial 9001 pokey.cs.almamater.edu 9001
       */
      public static void main(String[] args) throws Exception {
        // Loads pastry settings
        Environment env = new Environment();

        try {
          // the port to use locally
          int bindport = Integer.parseInt(args[0]);

          // build the bootaddress from the command line args
          InetAddress bootaddr = InetAddress.getByName(args[1]);
          int bootport = Integer.parseInt(args[2]);
          InetSocketAddress bootaddress = new InetSocketAddress(bootaddr,bootport);

          // launch our node!
          DistTutorial dt = new DistTutorial(bindport, bootaddress);
        } catch (Exception e) {
          // remind user how to use
          System.out.println("Usage:");
          System.out.println("java [-cp FreePastry-.jar] rice.tutorial.lesson1.DistTutorial localbindport bootIP bootPort");
          System.out.println("example java rice.tutorial.DistTutorial 9001 pokey.cs.almamater.edu 9001");
          throw e;
        }
      }

### Let's examine each line:

1. This line constructs the Environment. It has the side effect of starting a daemon thread.

           Environment env = new Environment();

2. This line parses the first command line argument into an int.

           int bindport = Integer.parseInt(args[0]);

3. These lines parse the IP and port and turn them into an InetSocketAddress.

           InetAddress bootaddr = InetAddress.getByName(args[1]);
           int bootport = Integer.parseInt(args[2]);
           InetSocketAddress bootaddress = new InetSocketAddress(bootaddr,bootport);

4. Finally we execute our constructor.

           DistTutorial dt = new DistTutorial(bindport, bootaddress);

5. We wrap the method with an indication of how to use the program in case the user inputs the wrong args.

           try {

             ...

           } catch (Exception e) {
             // remind user how to use
             System.out.println("Usage:");
             System.out.println("java [-cp FreePastry-.jar] rice.tutorial.lesson1.DistTutorial localbindport bootIP bootPort");
             System.out.println("example java rice.tutorial.DistTutorial 9001 pokey.cs.almamater.edu 9001");
             throw e;
           }

### Congratulations! You have built code to launch/join a FreePastry ring!
Lesson 2 will aid you in running your code.

# Lesson 2

## Execute the code to launch your new ring.

This is a short lesson that shows you how to run DistTutorial.java which you created in Lesson 1.

### Download the tutorial file DistTutorial.java.

After you compile the code, you can either run multiple nodes on 1 computer (but in separate processes) or if you have multiple computers, you can launch them on different machines as long as the computers can communicate with each other via IP. In other words, it won't work if the computers are behind different firewalls.

- Step 1: Compile
  1. Setup your directory as follows:

         FreePastry-1.4.2.jar
         rice/tutorial/lesson1/DistTutorial.java

  2. Compile the Java sources:

         javac -classpath FreePastry-1.4.2.jar rice/tutorial/lesson1/*.java

- Step 2: Launch the bootstrap node. Even though this is the first node, and therefore we know the bootstrap will not work, you need to place in a bogus ip/port. A smarter `main()` method in DistTutorial.java could fix this problem.

      java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson1.DistTutorial 9001 yourhost.domain 9001

  (In the above command, yourhost.domain can be the DNS name of your host or its IP address.) Your output will look something like this:

      java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson1.DistTutorial 9001 10.9.8.7 9001
      :1122932166578:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
      :1122932166578:No bootstrap node provided, starting a new ring...
      Finished creating new node SocketNodeHandle (<0xB7E151..>/FOO/10.9.8.7:9001 [-4233509936758121968])

  Note that the first 2 lines starting with the current system time are generated by the new logging system.
- Step 3: Launch another node.

  You can do this on another computer but make sure you fill in the name of the computer from step 2 in bootstraphost.domain.

      java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson1.DistTutorial 9002 bootstraphost.domain 9001

  Your output will look something like this:

      java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson1.DistTutorial 9002 10.9.8.7 9001
      Finished creating new node SocketNodeHandle (<0xE00352..>/FOO/10.9.8.7:9002 [4492232311666603357])

### Congratulations! You have just launched your first FreePastry ring!
Lesson 3 will show you how to send and receive messages by creating a commonAPI application.

# Lesson 3

## Write a simple application using the commonAPI.

### Download the tutorial files: DistTutorial.java (changed from Lesson 1!!!), MyApp.java, MyMsg.java into a directory called rice/tutorial/lesson3/.

This tutorial will show you how to create and run your first FreePastry application. You will be able to send/receive messages with this application.

## Terms:

- CommonAPI—Universal interface to structured overlays. The rice.p2p.commonapi package provides an interface similar to the one provided in 'Towards a Common API for Structured Peer-to-Peer Overlays', by F. Dabek, B. Zhao, P. Druschel, J. Kubiatowicz, and I. Stoica, published in the second International Workshop on Peer-to-Peer Systems, Berkeley, CA, February 2003. The API is designed to allow applications to be written in a protocol-independent manner, allowing the easy migration of applications from Pastry to Chord to CAN to Tapestry, etc.... Applications need only interact with the interfaces in the rice.p2p.commonapi package, and need not worry about anything in the rice.pastry packages.
- Application—A program that runs on a Node. This is an interface which all applications on top of a Node must export. This interface allows the underlying node to deliver message, inform the application of passing messages, and changes in the neighbor nodes.

  You can have multiple Applications on a single node, and you can even have multiple instances of an Application on the same node. Why would you want multiple instances on the same node? Many applications in FreePastry are intermediate applications that provide a service for higher level applications. It is convenient to have a separate instances of say... Scribe for each higher application that would like to multicast. This way you know that any messages that scribe delivers are in regards to your higher level traffic, and you don't need to distinguish that traffic between multiple high level applications.
- Endpoint.java—This interface represents an endpoint which applications can use to send messages from. An endpoint is obtained by the registerApplication() method in Node. The endpoint represents the applications' view of the world.
- Id.java—This interface is the abstraction of an Id in a structured overlay. The only assumption that is made is that the Id space is circular. An Id could represent a live node in the network, an object in the network, or simply a random Id. NodeId implements this interface.
- Message.java—This interface represents the abstraction of a message in the common API. Thus, messages sent to other nodes should extend or implement this class. FreePastry sends messages around the network by first converting them to a byte stream using java serialization.

### MyMsg

Let's start by taking a look at your message class. In FreePastry a Message is an Object that is Serializable. The transport layer serializes this object into bytes then sends it through the network. When the message is received, it is deserialized back into an Object and delivered to your application.

    public class MyMsg implements Message {

This class implements `rice.p2p.commonapi.Message`. Message extends Serializable and has a single method: `getPriority()`. Let's take a look at that method now.

For now always return Message.LOW_PRIORITY for your messages. It is important to not set application message priority too high, or you may interfere with Pastry's overlay maintenance traffic that keeps the ring functioning properly.

      public int getPriority() {
        return Message.LOW_PRIORITY;
      }

The "payload" of your message is created by making member variables in your message. Here is the payload for `MyMsg`. An Id is the commonAPI version of a NodeId, it is also used as the "key" when routing.

      /**
       * Where the Message came from.
       */
      Id from;
      /**
       * Where the Message is going.
       */
      Id to;

We will create a `toString()` so that we can print out the message.

      public String toString() {
        return "MyMsg from "+from+" to "+to;
      }

Finally we have the constructor that loads the payload.

      public MyMsg(Id from, Id to) {
        this.from = from;
        this.to = to;
      }

### MyApp

Now let's take a look at MyApp. MyApp is designed to log output whenever we send or receive a message.

The Endpoint is what we will call on to send messages.

      protected Endpoint endpoint;

The constructor generates an Endpoint from the node. The instance is designed to allow you to run the same app multiple times on the same node. The apps will not receive each other's messages. You will only be able to send messages to apps that generated endpoints with the same instance string. For most of your apps, you will only run one instance, so just make sure the instance is the same on all nodes.

      public MyApp(Node node) {
        // We are only going to use one instance of this application on each PastryNode
        this.endpoint = node.registerApplication(this, "myinstance");
      }

#### Sending a message:

In a Distributed Hash Table, or DHT, you typically want to route to the nearest node to the hash of an object that you are interested in. The commonAPI provides you with `Endpoint.route()` to accomplish this. This function sends a `MyMsg` to an id.

      /**
       * Called to route a message to the id
       */
      public void routeMyMsg(Id id) {
        System.out.println(this+" sending to "+id);
        Message msg = new MyMsg(endpoint.getId(), id);
        endpoint.route(id, msg, null);
      }

Note that `Endpoint.route()` takes 3 arguments. They are:

1. Id—the destination of the message (optional)
2. Message—the message to send
3. NodeHandle—a "hint": the node to route to first (optional)

Sometimes you need to send messages to a particular Node in the network, not just to the nearest key. For example, if you need to send messages that are large, they are going to take up a lot of bandwidth; it is more efficient to not route the messages through the overlay, but rather send directly to the node. You can use the `Endpoint.route()` method to accomplish this by passing a `null` argument as the Id and the target node's NodeHandle as the "hint" argument.

      /**
       * Called to directly send a message to the nh
       */
      public void routeMyMsgDirect(NodeHandle nh) {
        System.out.println(this+" sending direct to "+nh);
        Message msg = new MyMsg(endpoint.getId(), nh.getId());
        endpoint.route(null, msg, nh);
      }

#### Receiving a message:

Simply implement the `deliver()` method as is specified by the Application interface.

      /**
       * Called when we receive a message.
       */
      public void deliver(Id id, Message message) {
        System.out.println(this+" received "+message);
      }

For now you don't need to worry about the additional methods in the Application interface.

### Congratulations, you have an application. Let's integrate it into DistTutorial.java

Here is the new code we will add to the bottom of the DistTutorial constructor:

        // construct a new MyApp
        MyApp app = new MyApp(node);

        // wait 10 seconds
        Thread.sleep(10000);

        // as long as we're not the first node
        if (bootHandle != null) {

          // route 10 messages
          for (int i = 0; i < 10; i++) {
            // pick a key at random
            Id randId = nidFactory.generateNodeId();

            // send to that key
            app.routeMyMsg(randId);

            // wait a sec
            Thread.sleep(1000);
          }

          // wait 10 seconds
          Thread.sleep(10000);

          // send directly to my leafset
          LeafSet leafSet = node.getLeafSet();

          // this is a typical loop to cover your leafset.  Note that if the leafset
          // overlaps, then duplicate nodes will be sent to twice
          for (int i=-leafSet.ccwSize(); i<=leafSet.cwSize(); i++) {
            if (i != 0) { // don't send to self
              // select the item
              NodeHandle nh = leafSet.get(i);

              // send the message directly to the node
              app.routeMyMsgDirect(nh);

              // wait a sec
              Thread.sleep(1000);
            }
          }
        }

First we create the MyApp.

        // construct a new MyApp
        MyApp app = new MyApp(node);

After waiting 10 seconds, let's send some messages, but only if I am not the first node. After all, if I'm the only node in the ring, it is hardly interesting to send messages to myself.

        if (bootHandle != null) {

Loop 10 times.

          for (int i = 0; i < 10; i++) {

Reuse the RandomNodeIdFactory to generate random keys to route to.

            Id randId = nidFactory.generateNodeId();

Route.

            app.routeMyMsg(randId);

Wait a second and repeat.

            Thread.sleep(1000);
          }

After waiting another 10 seconds, let's send some messages directly to nodes. This section is also going to show you how to access the leafset from the PastryNode. Note that this is a FreePastry specific call, not a commonAPI call.

Get the leafset from the PastryNode:

          LeafSet leafSet = node.getLeafSet();

Iterate over all of the nodes in the leafset.

          for (int i=-leafSet.ccwSize(); i<=leafSet.cwSize(); i++) {

Don't send to myself. The local node is node zero in the leafset.

            if (i != 0) { // don't send to self

Extract the nodehandle at that index.

              NodeHandle nh = leafSet.get(i);

Wait a second and repeat.

            Thread.sleep(1000);
          }

Send the message.

              app.routeMyMsgDirect(nh);
          }

Now if you execute this code twice you should get something like:

(for Node1)

    java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson3.DistTutorial 9001 10.9.8.7 9001
    :1122933198281:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
    :1122933198296:No bootstrap node provided, starting a new ring...
    Finished creating new node SocketNodeHandle (<0xC20545..>/FOO/10.9.8.7:9001 [-4445364026872145996])
    MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xA67C20..>
    MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xBF799E..>
    MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xC4BEE7..>
    MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0x86ACA9..>
    MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0x9906E6..>
    MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0x8F5015..>
    MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xC20545..>
    MyApp <0xC20545..> received MyMsg from <0xDD90C6..> to <0xC20545..>

(for Node2)

    java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson3.DistTutorial 9002 10.9.8.7 9001
    Finished creating new node SocketNodeHandle (<0xDD90C6..>/FOO/10.9.8.7:9002 [5138450490561334965])
    MyApp <0xDD90C6..> sending to <0x2E5C63..>
    MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0x2E5C63..>
    MyApp <0xDD90C6..> sending to <0x03045C..>
    MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0x03045C..>
    MyApp <0xDD90C6..> sending to <0xA67C20..>
    MyApp <0xDD90C6..> sending to <0xF9C506..>
    MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0xF9C506..>
    MyApp <0xDD90C6..> sending to <0xBF799E..>
    MyApp <0xDD90C6..> sending to <0xC4BEE7..>
    MyApp <0xDD90C6..> sending to <0x86ACA9..>
    MyApp <0xDD90C6..> sending to <0x41F900..>
    MyApp <0xDD90C6..> received MyMsg from <0xDD90C6..> to <0x41F900..>
    MyApp <0xDD90C6..> sending to <0x9906E6..>
    MyApp <0xDD90C6..> sending to <0x8F5015..>
    MyApp <0xDD90C6..> sending direct to [SNH: <0xDD90C6..> -> <0xC20545..>/FOO/10.9.8.7:9001 [-4445364026872145996]]
    MyApp <0xDD90C6..> sending direct to [SNH: <0xDD90C6..> -> <0xC20545..>/FOO/10.9.8.7:9001 [-4445364026872145996]]

### Congratulations! You have just sent and received your first messages in FreePastry!

# Lesson 4

## Running multiple nodes in the same JVM.

### Download the tutorial files: DistTutorial.java (changed from Lesson 3!!!), MyApp.java, MyMsg.java into a directory called rice/tutorial/lesson4/.

This tutorial will show you how to create and run multiple FreePastry nodes within the same JVM. We will continue to use the Socket transport layer. This tutorial is very simple, all we will be doing is modifying lesson 3 to have multiple nodes.
Let's start with the changes to MyApp. We are going to make it keep a reference to the Node with a member variable named `node`. It will become apparent later in this tutorial.

      protected Node node;

In the constructor:

        this.node = node;

And here is the getter:

      public Node getNode() {
        return node;
      }

Now we will look at the changes to DistTutorial. We have a member variable `apps` that will keep track of the list of apps.

      Vector apps = new Vector();

We modify the constructor to take in how many nodes we wish to create:

      public DistTutorial(int bindport, InetSocketAddress bootaddress, int numNodes, Environment env) throws Exception {

After constructing the `PastryNodeFactory` we create a loop to construct the nodes, and add them to our `apps` Vector.

        for (int curNode = 0; curNode < numNodes; curNode++) {

    	    ...

    	    apps.add(app);
        }

When routing messages, we have created a subloop to do so for each app in the system:

          // for each app
          Iterator appIterator = apps.iterator();
          while(appIterator.hasNext()) {
            MyApp app = (MyApp)appIterator.next();

            ...

          }

Note that we eliminated this code, because it is no longer important to not route if we are the bootstrap.

        // as long as we're not the first node
        if (bootHandle != null) {

In the send direct loop we use our getter `MyApp.getNode()`. This prevents us from having to remember a parallel data structure for the Nodes.

        // for each app
        Iterator appIterator = apps.iterator();
        while(appIterator.hasNext()) {
          MyApp app = (MyApp)appIterator.next();
          PastryNode node = (PastryNode)app.getNode();

Other than that, we changed the delay in several locations so the program doesn't take as long to execute. Compile and run this program as before, but make sure to specify some the number of nodes to run as the 4th argument.

    java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson4.DistTutorial 9001 10.9.8.7 9001 10
    :1122933905343:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
    :1122933905359:No bootstrap node provided, starting a new ring...
    Finished creating new node SocketNodeHandle (<0x93426F..>/FOO/10.9.8.7:9001 [6666451457797044406])
    Finished creating new node SocketNodeHandle (<0xAE1B63..>/FOO/10.9.8.7:9002 [8904287830673843378])
    Finished creating new node SocketNodeHandle (<0x3E99E3..>/FOO/10.9.8.7:9003 [-1171813548315184530])
    Finished creating new node SocketNodeHandle (<0xC8715F..>/FOO/10.9.8.7:9004 [6496343980737954414])
    Finished creating new node SocketNodeHandle (<0x688826..>/FOO/10.9.8.7:9005 [-6779179910423314388])
    Finished creating new node SocketNodeHandle (<0x640179..>/FOO/10.9.8.7:9006 [3471226224877450778])
    Finished creating new node SocketNodeHandle (<0x7A6F99..>/FOO/10.9.8.7:9007 [7101267768505206817])
    Finished creating new node SocketNodeHandle (<0xD5DAAE..>/FOO/10.9.8.7:9008 [7899110252170216207])
    Finished creating new node SocketNodeHandle (<0x19C832..>/FOO/10.9.8.7:9009 [-4206970387543598705])
    Finished creating new node SocketNodeHandle (<0x490C38..>/FOO/10.9.8.7:9010 [-6742572645641651296])
    MyApp <0x93426F..> sending to <0x0E510A..>
    MyApp <0x19C832..> received MyMsg from <0x93426F..> to <0x0E510A..>
    MyApp <0xAE1B63..> sending to <0xE704A5..>
    MyApp <0xD5DAAE..> received MyMsg from <0xAE1B63..> to <0xE704A5..>
    MyApp <0x3E99E3..> sending to <0x533803..>
    MyApp <0x490C38..> received MyMsg from <0x3E99E3..> to <0x533803..>
    MyApp <0xC8715F..> sending to <0x779E75..>
    MyApp <0x7A6F99..> received MyMsg from <0xC8715F..> to <0x779E75..>
    MyApp <0x688826..> sending to <0x3FFFB9..>
    MyApp <0x3E99E3..> received MyMsg from <0x688826..> to <0x3FFFB9..>

    ...

    MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0xAE1B63..>/FOO/10.9.8.7:9002 [8904287830673843378]]
    MyApp <0xAE1B63..> received MyMsg from <0x93426F..> to <0xAE1B63..>
    MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0xC8715F..>/FOO/10.9.8.7:9004 [6496343980737954414]]
    MyApp <0xC8715F..> received MyMsg from <0x93426F..> to <0xC8715F..>
    MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0xD5DAAE..>/FOO/10.9.8.7:9008 [7899110252170216207]]
    MyApp <0xD5DAAE..> received MyMsg from <0x93426F..> to <0xD5DAAE..>
    MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0x19C832..>/FOO/10.9.8.7:9009 [-4206970387543598705]]
    MyApp <0x19C832..> received MyMsg from <0x93426F..> to <0x19C832..>
    MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0x3E99E3..>/FOO/10.9.8.7:9003 [-1171813548315184530]]
    MyApp <0x3E99E3..> received MyMsg from <0x93426F..> to <0x3E99E3..>
    MyApp <0x93426F..> sending direct to [SNH: <0x93426F..> -> <0x490C38..>/FOO/10.9.8.7:9010 [-6742572645641651296]]
    MyApp <0x490C38..> received MyMsg from <0x93426F..> to <0x490C38..>

    ...

### Congratulations! You have just created an entire ring inside a single JVM. This should make it much easier to test your application.

# Lesson 5

## Scheduling tasks on the FreePastry timer.

### Download the tutorial files: DistTutorial.java, MyApp.java into a directory called rice/tutorial/lesson5/.

This tutorial will show you how to schedule a task on the FreePastry thread. To be compatible with the commonAPI, the approach is to schedule a message to be sent to yourself at a regular interval.

Let's start with the changes to MyApp.

First a simple message.

      class MessageToSelf implements Message {
        public int getPriority() {
          return 0;
        }
      }

Note that due to laziness, I have constructed this Message as an inner class of MyApp. This is fine for what we are going to use it for, but beware that it you will get a NotSerializableException if you attempt to send this message over the wire. While the Message may be serializable it has an implicit reference to MyApp, the containing class. When the message is serialized, FreePastry will also attempt to serialize MyApp which will fail and throw the exception.

Here is the change to the constructor of MyApp, where we schedule the MessageToSelf. The first parameter is the `Message` that will be sent to MyApp. The second is the delay (in millis) until it is first sent. The third parameter is the period at which the message should be repeated.

        // Send MessageToSelf every 5 seconds, starting in 3 seconds
        messageToSelfTask = endpoint.scheduleMessage(new MessageToSelf(), 3000, 5000);

As you can see, we have asked FreePastry to deliver the MessageToSelf in 3 seconds, and then continue to deliver it again every 5 seconds. Note that there is also a version of scheduleMessage() that doesn't take the 3rd parameter, and therefore doesn't repeat.

We stored the task as a member variable so that we can cancel it later.

      CancellableTask messageToSelfTask;

Whenever we receive the MessageToSelf we are just going to print out the current time. However, in your application you could call a method to do routine maintenance, send messages to other nodes etc.

      public void deliver(Id id, Message message) {
        System.out.println(this+" received "+message);
        if (message instanceof MessageToSelf) {
          // This will get called every 5 seconds, on Pastry's thread.
          // Thus now we can assume we are on Pastry's thread.
          // TODO: whatever... send messages to other nodes? print out status?
          System.out.println("I got the MessageToSelf at time:"+System.currentTimeMillis());
        }
      }

Finally, we make a public method to cancel the task.

      public void cancelTask() {
        messageToSelfTask.cancel();
      }

The DistTutorial.java is based on lesson3, but we removed the code to print the leafset. For this example it is unnecessary to launch multiple pastry nodes, so we will only launch one. We added a bit of code after constructing the MyApp to cancel the task after 15 seconds.

        // construct a new MyApp
        MyApp app = new MyApp(node);

        // wait 15 seconds
        Thread.sleep(15000);

        // cancel the task
        app.cancelTask();

Ok, time to run it! Your output should resemble the following:

    java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson5.DistTutorial 9001 localhost 9001
    :1122932166578:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
    :1122932166578:No bootstrap node provided, starting a new ring...
    Finished creating new node SocketNodeHandle (<0x3C1EEA..>/FOO/10.9.8.7:9001 [9038308767327354496])
    MyApp <0x3C1EEA..> received rice.tutorial.lesson5.MyApp$MessageToSelf@d70d7a
    I got the MessageToSelf at time:1115228553359
    MyApp <0x3C1EEA..> received rice.tutorial.lesson5.MyApp$MessageToSelf@d70d7a
    I got the MessageToSelf at time:1115228558359
    MyApp <0x3C1EEA..> received rice.tutorial.lesson5.MyApp$MessageToSelf@d70d7a
    I got the MessageToSelf at time:1115228563359

### Congratulations! Now you can schedule and cancel tasks without launching your own thread.

# Lesson 6

## Introducing Scribe.

### Download the tutorial files: MyScribeClient.java, MyScribeContent.java, ScribeTutorial.java into a directory called rice/tutorial/lesson6/.

Scribe is an application that allows you to subscribe to groups and publish messages to that group. This tutorial will show you how to get scribe up and running. You will learn how to do the following:

- Create a topic.
- Create a ScribeClient.
- Subscribe to a topic.
- Publish (Multicast) content.
- Receive content.
- Anycast content.
- Introspect into the tree.

Lesson 9: Advanced Scribe: Policies, will show you how to control tree formation and anycast selection.

## Terms:

- Scribe—A scalable group communication system for topic-based publish-subscribe applications.Scribe builds an efficient multicast tree for dissemination of events to a topic.
- Topic—Group. A topic builds a hash of the group name which is used as a unique identifier for the topic, as well used as a rendezvous point in Pastry.
- IdFactory—A hash function. An IdFactory implements a hash function to build Ids that are compatable with pastry.
- PastryIdFactory—A commonly used IdFactory in Pastry. The PastryIdFactory uses SHA1 as it's underlieing hash function.
- ScribeContent—A scribe message.
- ScribeClient—An application that receives ScribeContent. The client can subscribe to one or more topics.
- Multicast—A broadcast received by everyone subscribed to the corresponding topic.
- Anycast—A message that is received by a single node in a group. Anycast is most commonly used to find a single available service provider. The anycast message will be rejected until it finds a node willing to supply the service that is being requested.
- ScribePolicy—A policy to determine application specific details of Scribe. This includes formation and anycast selection.

### Creating a topic.

This is fairly straightforward, but each node will have to do this for each topic of interest.

      Topic myTopic = new Topic(new PastryIdFactory(), "example topic");

This constructs a topic with the "common name" of "example topic". It uses the PastryIdFactory to generate an appropriate Id for this topic.

### Creating a ScribeClient.

Let's take a look at MyScribeClient. We are only going to subscribe to a single topic. We are going to publish content every 5 seconds using the FreePastry timer. See Lesson 5 for more details on the timer. The client is also going to implement rice.p2p.commonapi.Application in addition to rice.p2p.scribe.ScribeClient. This will allow us to send and receive non-scribe messages should this be important. Specifically it will allow us to receive messages queued on the timer.
Here's the constructor and some member variables:

      Scribe myScribe;
      Topic myTopic;
      protected Endpoint endpoint;
      public MyScribeClient(PastryNode node) {
        // you should recognize this from lesson 3
        this.endpoint = node.registerApplication(this, "myinstance");
        // construct Scribe
        myScribe = new ScribeImpl(node,"lesson6instance");
        // construct the topic
        myTopic = new Topic(new PastryIdFactory(), "example topic");
        System.out.println("myTopic = "+myTopic);
      }

The only thing that should be new here is the construction of the ScribeImpl. The instance name "lesson6instance" allows you to remain independent of other applications running on the same ring who also use scribe. They will have their own instance of Scribe that won't be confused with your instance.
In this example, one of the nodes is going to publish content every 5 seconds. We use the timer pattern described in Lesson 5. Each time we are going to send 1 multicast, and 1 anycast.

      class PublishContent implements Message {
        public int getPriority() {
          return 0;
        }
      }
      public void startPublishTask() {
        publishTask = endpoint.scheduleMessage(new PublishContent(), 5000, 5000);
      }
      public void deliver(Id id, Message message) {
        if (message instanceof PublishContent) {
          sendMulticast();
          sendAnycast();
        }
      }

In case you forgot, the `PublishContent` is similar to Lesson 5's `MessageToSelf`. The `startPublishTask()` method schedules this to be delivered locally every 5 seconds. The `deliver(id,message)` method calls `sendMulticast()` when the `PublishMethod` is received. In your application you will likely have some other event that causes content to be published. We will look at `sendMulticast()` and `sendAnycast()` shortly.

### Subscribing to a group.

Subscribing is very easy. Just call `Scribe.subscribe()` and provide the topic, and your client.

      public void subscribe() {
        myScribe.subscribe(myTopic, this);
      }

### Multicasting content.

First, we need some content to send. MyScribeContent implements ScribeContent and takes a NodeHandle sender, and an int sequence number. These are just so the output of the program is more interesting.

    public class MyScribeContent implements ScribeContent {
      NodeHandle from;
      int seq;

      public MyScribeContent(NodeHandle from, int seq) {
        this.from = from;
        this.seq = seq;
      }

      public String toString() {
        return "MyScribeContent #"+seq+" from "+from;
      }
    }

To send the content, simply construct the message, then call Scribe.publish(). You give it the topic and the message. The rest of this function is just to print output and update the sequence number.

      public void sendMulticast() {
        System.out.println("Node "+endpoint.getLocalNodeHandle()+" broadcasting "+seqNum);
        MyScribeContent myMessage = new MyScribeContent(endpoint.getLocalNodeHandle(), seqNum);
        myScribe.publish(myTopic, myMessage);
        seqNum++;
      }

### Receiving content.

Receiving content is as easy as any other p2p application. The method signature is only slightly different:

      public void deliver(Topic topic, ScribeContent content) {
        System.out.println("MyScribeClient.deliver("+topic+","+content+")");
      }

All we are doing here is printing output to stdout.

### Anycasting.

Anycast will get called on your clients until one returns true. This occurs on a call to `ScribeClient.anycast()`. To make this interesting, we're going to only accept the message 1/3 of the time, randomly. Your application will ususally want to do something more interesting, such as see if a requested resource is available.
Here is `sendAnycast()` which is nearly identical to `sendMulticast()` except for the call to `Scribe.anycast()` instead of `Scribe.publish()`.

      public void sendAnycast() {
        System.out.println("Node "+endpoint.getLocalNodeHandle()+" anycasting "+seqNum);
        MyScribeContent myMessage = new MyScribeContent(endpoint.getLocalNodeHandle(), seqNum);
        myScribe.anycast(myTopic, myMessage);
        seqNum++;
      }

Here is the code that only accepts the anycast 1/3 of the time. This will allow us to see that the anycast message can be rejected and sent elsewhere.

      public boolean anycast(Topic topic, ScribeContent content) {
        boolean returnValue = rng.nextInt(3) == 0;
        System.out.println("MyScribeClient.anycast("+topic+","+content+"):"+returnValue);
        return returnValue;
      }

### Examining the tree.

Lastly we have code to introspect the tree. Note that it is only easy to print out the entire tree because we are running all of the nodes in the same VM and have global information. It is significantly more difficult to print the scribe tree in an actual distributed environment, as scribe does not provide this information automatically. Furthermore this information could change rapidly as nodes join and leave.
Because we are only going to run the application with a small number of nodes, the tree will most likely be only 1 level deep. (The root being level 0). However, this will show you how to print out your parent and children.
The first thing to note is that we have 3 accessor methods at the bottom of `MyScribeContent`:

      public boolean isRoot() {
        return myScribe.isRoot(myTopic);
      }

      public NodeHandle getParent() {
        return myScribe.getParent(myTopic);
      }

      public NodeHandle[] getChildren() {
        return myScribe.getChildren(myTopic);
      }

Note that these simply call through to the same method on myScribe, with the correct topic.
This code can be found in `ScribeTutorial.java`
`printTree()` does the following:

1. Create a table mapping `NodeHandle` to `MyScribeClient`.
2. Recursively traverse the tree to the root, using the helper: `getRoot()`.
3. Recursively traverse the tree down from the root, depth first and print the nodes using the helper: `recursivelyPrintChildren()`.

      public static void printTree(Vector apps) {
        // build a hashtable of the apps, keyed by nodehandle
        Hashtable appTable = new Hashtable();
        Iterator i = apps.iterator();
        while (i.hasNext()) {
          MyScribeClient app = (MyScribeClient)i.next();
          appTable.put(app.endpoint.getLocalNodeHandle(), app);
        }
        NodeHandle seed = ((MyScribeClient)apps.get(0)).endpoint.getLocalNodeHandle();

        // get the root
        NodeHandle root = getRoot(seed, appTable);

        // print the tree from the root down
        recursivelyPrintChildren(root, 0, appTable);
      }

1. `getRoot()` looks up the client for the seed handle from the appTable.
2. If the seed is the root, it is returned.
3. Otherwise, it calls `getRoot()` on the parent.

      public static NodeHandle getRoot(NodeHandle seed, Hashtable appTable) {
        MyScribeClient app = (MyScribeClient)appTable.get(seed);
        if (app.isRoot()) return seed;
        NodeHandle nextSeed = app.getParent();
        return getRoot(nextSeed, appTable);
      }

1. `recursivelyPrintChildren()` prints the curNode with appropriate whitespace based on the depth in the tree.
2. Then calls recursivelyPrintChildren() on all children (if it has any)

      public static void recursivelyPrintChildren(NodeHandle curNode, int recursionDepth, Hashtable appTable) {
        // print self at appropriate tab level
        String s = "";
        for (int numTabs = 0; numTabs < recursionDepth; numTabs++) {
          s+="  ";
        }
        s+=curNode.getId().toString();
        System.out.println(s);

        // recursively print all children
        MyScribeClient app = (MyScribeClient)appTable.get(curNode);
        NodeHandle[] children = app.getChildren();
        for (int curChild = 0; curChild < children.length; curChild++) {
          recursivelyPrintChildren(children[curChild], recursionDepth+1, appTable);
        }
      }

### Initializing the apps.

The majority of the ScribeTutorial is identical to Lesson 4 where we ran multiple nodes within the same JVM. The last part is listed below. On each app, we call `subscribe()`, and on the first one we call `startPublishTask()`. After that we wait a few seconds then print the tree.

        Iterator i = apps.iterator();
        MyScribeClient app = (MyScribeClient)i.next();
        app.subscribe();
        app.startPublishTask();
        while(i.hasNext()) {
          app = (MyScribeClient)i.next();
          app.subscribe();
        }

        Thread.sleep(3000);
        printTree(apps);

### Execution.

The parameters are identical to those in Lesson 4:

1. Local bind port.
2. Bootstrap host. (the local host address)
3. Bootstrap port. (usually whatever you passed in the first arg)
4. The number of nodes to launch.

Your output will resemble:

    java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson6.ScribeTutorial 9001 10.9.8.7 9001 10
    :1122932166578:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
    :1122932166578:No bootstrap node provided, starting a new ring...
    Finished creating new node SocketNodeHandle (<0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189])
    myTopic = [TOPIC <0x19A8F5..>]
    Finished creating new node SocketNodeHandle (<0x8904D9..>/FOO/10.9.8.7:9002 [4410881318286179245])
    myTopic = [TOPIC <0x19A8F5..>]
    Finished creating new node SocketNodeHandle (<0x281817..>/FOO/10.9.8.7:9003 [1144941711194723161])

    ...

    <0x281817..>
      <0x8904D9..>
      <0x061BB8..>
      <0x85DA64..>
      <0x489BCB..>
      <0x0A9FCC..>
      <0x39CE29..>
      <0xA20DF8..>
      <0x7D350E..>
      <0xCF76F1..>
    Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] broadcasting 0
    Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] anycasting 1
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #1 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #1 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):true
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x8904D9..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0xA20DF8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0xCF76F1..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x0A9FCC..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x7D350E..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x85DA64..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #0 from [SNH: <0x061BB8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] broadcasting 2
    Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] anycasting 3
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #3 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #3 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):true
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x8904D9..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0xA20DF8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0xCF76F1..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x0A9FCC..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x7D350E..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x85DA64..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #2 from [SNH: <0x061BB8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] broadcasting 4
    Node [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]] anycasting 5
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x281817..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x8904D9..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0xA20DF8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0xCF76F1..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x0A9FCC..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x7D350E..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x85DA64..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x489BCB..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.deliver([TOPIC <0x19A8F5..>],MyScribeContent #4 from [SNH: <0x061BB8..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]])
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x39CE29..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):false
    MyScribeClient.anycast([TOPIC <0x19A8F5..>],MyScribeContent #5 from [SNH: <0x8904D9..> -> <0x489BCB..>/FOO/10.9.8.7:9001 [3776408266594462189]]):true

Note that the tree is only 1 level deep.
Note how each publish message is delivered to each node in the group.
Note that anycast is called on different nodes until a node returns true.

### Congratulations! You have built and run your first scribe application!

# Lesson 7

## Past &mdash FreePastry's DHT.

### Download the tutorial files: MyPastContent.java, PastTutorial.java into a directory called rice/tutorial/lesson7/.

Past is FreePastry's Distributed Hash Table (DHT). This tutorial will show you how to get scribe up and running. You will learn how to do the following:

- Create a PastContent.
- Create a Past instance.
- Put an object in Past.
- Get an object from Past.
- A failed Get.

## Terms:

- Past—A large-scale, peer-to-peer archival storage facility. Past is FreePastry's DHT. It has insert() and lookup() operations.
- Content—A possible value of the DHT.
- IdFactory—A hash function. An IdFactory implements a hash function to build Ids that are compatable with pastry.
- PastryIdFactory—A commonly used IdFactory in Pastry. The PastryIdFactory uses SHA1 as it's underlying hash function.
- Storage—provides a local storage service for example, a persistence storage service or an in-memory storage service
- MemoryStorage—an implementation of Storage which provides in-memory storage. This class is specifically not designed to provide persistent storage.
- PersistentStorage—an implementation of Storage which provides persistent storage to disk.
- Cache—finite temporary storage for rapid access.
- LRUCache—implemetation of a least-recently-used (LRU) cache.
- StorageManager—a storage attached to a cache. Objects inserted and retrieved from the storage are auto-matically cached, to speed up future accesses.

### Create a PastContent.

Unlike Scribe, you do not need to write a client to utilize Past. However, you do need to create a content object to put/get. This is called a PastContent. The easiest way to create a PastContent is to extend rice.p2p.past.ContentHashPastContent. Here is the code for MyPastContent.

    public class MyPastContent extends ContentHashPastContent {
      /**
       * Store the content.
       *
       * Note that this class is Serializable, so any non-transient field will
       * automatically be stored to to disk.
       */
      String content;

      /**
       * Takes an environment for the timestamp
       * An IdFactory to generate the hash
       * The content to be stored.
       *
       * @param idf to generate a hash of the content
       * @param content to be stored
       */
      public MyPastContent(Id id, String content) {
        super(id);
        this.content = content;
      }

      /**
       * A descriptive toString()
       */
      public String toString() {
        return "MyPastContent ["+content+"]";
      }
    }

Note that the super constructor needs an Id. Other than that the class must be serializable.

### Create a Past instance.

Now on to the PastTutorial. The first part of PastTutorial looks identical to ScribeTutorial. The interesting part is creating the Past application.

          // used for generating PastContent object Ids.
          // this implements the "hash function" for our DHT
          PastryIdFactory idf = new rice.pastry.commonapi.PastryIdFactory(env);

          // create a different storage root for each node
          String storageDirectory = "./storage"+node.getId().hashCode();

          // create the persistent part
          Storage stor = new PersistentStorage(idf, storageDirectory, 4 * 1024 * 1024, node
              .getEnvironment());
          Past app = new PastImpl(node, new StorageManagerImpl(idf, stor, new LRUCache(
              new MemoryStorage(idf), 512 * 1024, node.getEnvironment())), 3, "");

Recall the PastryIdFactory is basically a hash funciton that produces Ids. The rest of the code sets up a typical PastImpl. It uses PersistentStorage for the primary store, and an LRUCache backed by a MemoryStorage as the cache.
Here is an overview of what is going on.
PastImpl needs the following:

- The Node
- StorageManager
- The number of Replicas to maintain — this is a parameter that you can tune to adjust overhead vs. chance of data-loss
- An instance name

StorageManagerImpl (the StorageManager) needs the following:

- An IdFactory
- A storage — we'll use the PersistentStorage
- A cache — we'll use the LRUCache

The PersistentStorage needs the following:

- An IdFactory
- The directory to put the storage root. — Note that we need a different storage root for each node, otherwise they will use the same one. This is only necessary because we are running multiple nodes in the same JVM.
- The max size (in bytes) — 1 Meg
- The environment

The LRUCache needs the following:

- A storage — we'll use a MemoryStorage
- The max size (in bytes) — 0.5 Meg
- The environment

### Put an object in Past.

Now lets store some data. The loop around this code is so we can later look up the data that was stored.

          // these variables are final so that the continuation can access them
          final String s = "test" + env.getRandomSource().nextInt();

          // build the past content
          final PastContent myContent = new MyPastContent(localFactory.buildId(s), s);

          storedKey[ctr] = myContent.getId();

          // pick a random past appl on a random node
          Past p = (Past)apps.get(env.getRandomSource().nextInt(numNodes));
          System.out.println("Inserting " + myContent + " at node "+p.getLocalNodeHandle());

          // insert the data
          p.insert(myContent, new Continuation() {
            // the result is an Array of Booleans for each insert
            public void receiveResult(Object result) {
              Boolean[] results = ((Boolean[]) result);
              int numSuccessfulStores = 0;
              for (int ctr = 0; ctr < results.length; ctr++) {
                if (results[ctr].booleanValue())
                  numSuccessfulStores++;
              }
              System.out.println(myContent + " successfully stored at " +
                  numSuccessfulStores + " locations.");
            }

            public void receiveException(Exception result) {
              System.out.println("Error storing "+myContent);
              result.printStackTrace();
            }
          });

Note that we call `Past.insert()` with our newly created MyPastContent object, and a Continuation. `Past.insert()` calls `receiveResult()` on the continuation with a Boolean[] that represents the success/failure of the individual stores for this object in the ring. Our continuation prints how many times this node was stored. Because our replication factor is 3, it should be stored 4 times (primary+3 replicas). If an error occurs it will call `receiveException()`.

### Get an object from Past.

We choose a random application and call Past.lookup()

          final Id lookupKey = storedKey[ctr];

          // pick a random past appl on a random node
          Past p = (Past)apps.get(env.getRandomSource().nextInt(numNodes));

          System.out.println("Looking up " + lookupKey + " at node "+p.getLocalNodeHandle());
          p.lookup(lookupKey, new Continuation() {
            public void receiveResult(Object result) {
              System.out.println("Successfully looked up " + result + " for key "+lookupKey+".");
            }

            public void receiveException(Exception result) {
              System.out.println("Error looking up "+lookupKey);
              result.printStackTrace();
            }
          });

### A failed Get.

Now let's see what happen if the Past doesn't have the requested content. Here's a hint, it calls `receiveResult()` with a null result.

        final Id bogusKey = localFactory.buildId("bogus");

    ...

        System.out.println("Looking up bogus key " + bogusKey + " at node "+p.getLocalNodeHandle());
        p.lookup(bogusKey, new Continuation() {
          public void receiveResult(Object result) {
            System.out.println("Successfully looked up " + result + " for key "+bogusKey+".  Notice that the result is null.");
          }

          public void receiveException(Exception result) {
            System.out.println("Error looking up "+bogusKey);
            result.printStackTrace();
          }
        });

When you run the code, your output will resemble:

    java -cp .:FreePastry-1.4.2.jar rice.tutorial.lesson7.PastTutorial 9001 10.9.8.7 9001 5
    :1122487192406:Error connecting to address /10.9.8.7:9001: java.net.ConnectException: Connection refused: no further information
    :1122487192421:No bootstrap node provided, starting a new ring...
    Finished creating new node SocketNodeHandle (<0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116])
    Finished creating new node SocketNodeHandle (<0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790])
    Finished creating new node SocketNodeHandle (<0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501])
    Finished creating new node SocketNodeHandle (<0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085])
    Finished creating new node SocketNodeHandle (<0x4AE14A..>/FOO/10.9.8.7:9005 [-7392240638718374041])
    Storing 5 keys
    Inserting MyPastContent [test1382830848] at node [SNH: <0x22D2E1..> -> <0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116]]
    Inserting MyPastContent [test-618483902] at node [SNH: <0xED50E1..> -> <0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085]]
    Inserting MyPastContent [test482928483] at node [SNH: <0x22D2E1..> -> <0x22D2E1..>/FOO/10.9.8.7:9001 [6702844014892124116]]
    Inserting MyPastContent [test-919775810] at node [SNH: <0x4AE14A..> -> <0x4AE14A..>/FOO/10.9.8.7:9005 [-7392240638718374041]]
    Inserting MyPastContent [test1099145466] at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
    MyPastContent [test1382830848] successfully stored at 4 locations.
    MyPastContent [test-618483902] successfully stored at 4 locations.
    MyPastContent [test482928483] successfully stored at 4 locations.
    MyPastContent [test-919775810] successfully stored at 4 locations.
    MyPastContent [test1099145466] successfully stored at 4 locations.
    Looking up the 5 keys
    Looking up <0x9515FC..> at node [SNH: <0x16C2C6..> -> <0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501]]
    Looking up <0xB987B7..> at node [SNH: <0x16C2C6..> -> <0x16C2C6..>/FOO/10.9.8.7:9003 [419583566816935501]]
    Looking up <0x999A32..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
    Looking up <0xC9B06E..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
    Looking up <0xC2EF27..> at node [SNH: <0x3EDF4C..> -> <0x3EDF4C..>/FOO/10.9.8.7:9002 [-4218714042288854790]]
    Successfully looked up MyPastContent [test1099145466] for key <0xC2EF27..>.
    Successfully looked up MyPastContent [test-618483902] for key <0xB987B7..>.
    Successfully looked up MyPastContent [test482928483] for key <0x999A32..>.
    Successfully looked up MyPastContent [test-919775810] for key <0xC9B06E..>.
    Successfully looked up MyPastContent [test1382830848] for key <0x9515FC..>.
    Looking up a bogus key
    Looking up bogus key <0x216ACE..> at node [SNH: <0xED50E1..> -> <0xED50E1..>/FOO/10.9.8.7:9004 [-7110190430495110085]]
    Successfully looked up null for key <0x216ACE..>.  Notice that the result is null.

Also 5 directories were created, each containing a FreePastry-Storage-Root subfolder:

    ls storage*
    storage-1404778543:
    FreePastry-Storage-Root

    storage-1501429183:
    FreePastry-Storage-Root

    storage-1862004272:
    FreePastry-Storage-Root

    storage2042841361:
    FreePastry-Storage-Root

    storage848913672:
    FreePastry-Storage-Root