The serv-comp sub-package

This sub-package has the bulk of the cluster-manager functionality. It has the following files:

Perl scripts

Service composition software

Configuration files

These are the different configuration files:

Running the software

In general, all programs give the usage, when run without any arguments. Here are the important ones:

Interface specifications

This is probably the most important part if you are trying to add a service, or a new client to the software package. The interface with the client, as well as the service instances are implemented on top of the UDP library in the udp-lib sub-package.

The interface between the cluster-manager and a client

A path is identified by a tuple: "pathID:path_dest_scid". The path_dest_scid is the SCID of the destination overlay node of the service-level path. The client makes its path construction request to its nearest overlay node's cluster-manager. And this overlay node forms the path's destination.

Ideally, the pathID should be chosen by the path_dest_scid cluster-manager. But, in implementation, it was just easier to have the client choose it -- it was easier to detect duplicates (see the "at-least once" semantics in the udp-lib sub-package documentation).

A negative consequence of this is that the clients have to choose unique pathID's during the lifetime of a run of the software. This can be difficult if there are multiple clients. But since this software was written for a design study and evaluation, and not for production operation, I considered this kludge to be manageable.

The client specifies the following in its request packet to the cluster-manager:

In the reply from the cluster-manager to the client, the field that is relevant is the "success" field of the ServCompIFPkt structure. This indicates success with a value of 1, and failure with a value of 0.

The interface between the cluster-manager and a service instance

During path construction, the cluster-manager communicates with service instances in its cluster -- to "setup" the path for the client session. The cluster-manager sends a request over the at-least once UDP layer of the udp-lib sub-package, and the service instance responds to it. Note that the port where the service instance is listening for the request, is given in the "cfgs/*services*.cfg" configuration files -- whatever is given as argument to the cluster-managers when they were started up.

The cluster-manager specifies the following information in the request packet:

In its reply, the service instance specifies the following to the cluster-manager:

This is so far as the path creation interface goes. The interface also includes ways to redirect the service instance to send data to an alternate downstream location, and to kill the service instance.

The APP_CHANGE_SERV_INST ServCompIFPkt type is used by the cluster-manager to ask the service instance to switch its downstream node. The cluster-manager specifies a new downstream service instance to direct data to. This is actually used during local recovery, and not during end-to-end recovery.

The APP_KILL_SERV_INST ServCompIFPkt type is used to terminate the particular client session in the service instance. This is not actually used as far as I know. The client session is terminated using an application level signaling. Also, theer is an application-level soft-state refresh, which if times out, can trigger the service instance to kill the client session.

The path construction process

This is best explained with an example -- we choose the text-to-speech composed service. The following picture shows the signaling that happens for path creation.



At this point, application-level communication begins. For the text-to-speech service, this consists of several data exchanges, shown the picture below. Communication starts with the upstream node sending information about itself to the next downstream node. This is part of the generic_client_info of the app sub-package. The no-op nodes participate in this. All other communication is not noticed by the no-op services -- they simply forward the data either way.



The path recovery process

Each path is identified by a version number -- this starts from 100, and is incremented by 100 in each step. We have implemented end-to-end recovery. Here, when a failure happens on an overlay link, the failure information is sent downstream to the destination overlay node of the path. Then, the destination overlay node sets up an alternate service-level path, with an alternate choice of service instances (or at least a different path in the overlay graph). This new path is given a version number of 200. If this fails as well, the next version is 300, and so on.

The path recovery process is the same as path setup, except for the version number.


Bhaskaran Raman, bhaskar@cs.berkeley.edu
Last modified: Tue Jan 22 15:33:43 PST 2002