This page is about the FAST system, which I developped during my PhD. It allows the scheduler to get informations about the current state of the system as well as the needs of the routine to schedule.

I modified NetSolve, a RPC based client/agent/server system developed at the University of Teneesee to make use of FAST. The result of this integration is described in this article. I also integrated FAST into DIET, a grid enabled Problem Solving Environment project in which my research team is involved trough the ARC INRIA called OURAGAN.

This project is now discontinued.

About FAST

FAST (Fast Agent System Timer) is a tool for dynamic performance forecasting in a Grid environment. FAST is composed of several layers and relies on low level software. First, it uses a network and CPU monitoring software to handle dynamically changing resources, like workload or bandwidth. FAST uses the Network Weather Service (NWS), a distributed system that periodically monitors and dynamically forecasts the performance of various network and computational resources. The dynamic data acquisition module of FAST uses and enhances NWS. FAST also includes routines to model the time and space needs for each triplet (problem ; machine ; parameters set). They are based on benchmarking at installation time on each machine for a representative set of parameters and polynomial data fitting. To store these static data, FAST uses a LDAP tree.

This is Free Software (GPL). For more information, you can check the my publications, or this presentation (in french).

Download: You can download all dependencies in one big tarball. It contains a small makefile, extracting all software in ~/DIET (change the ROOT variable put it somewhere else), configure, and install them. It should work to on linux/{i386,alpha,sparc} and solaris/sparc.

Related software

FAST may leverage several software when found, but none is absolutely mandatory. If compiled without any of those friends, FAST will still be able to give the current load of the machine it runs on. It is really limited, and you may want to install some of the following software to increase FAST's abilities.

What is really needed?
I would say that NWS is almost mandatory, that LDAP and GSL are welcome, and that BDB is useless for normal human beings. But of course, it depends on your personal use of the beast.

 <b>If you don't install this</b>: you won't be able to use FAST to
 compute the resources needed by a given function (ie, to benchmark). 
 So, this is useless if you already have a ldif file describing your
 operation on your machine (or don't care), and mandatory if you don't
 have this file and want to forecast the time/memory an operation will
 take.
 </td>

  <b>If you don't install this</b>: You won't lose any FAST
   functionality. Simply, during the benchmarkings, the intermediate
   results won't get buffered. If the benchmark works well, you don't
   care. If it breaks in the middle, you'll have to re-run it from the
   beginning. Not a big deal usually.</td> 

<b>If you don't install this</b>: The only resource availability
 measurement you'll be able to do is the current load of the current
 host.  Forget about remote host measurement, about future trend
 forecasting and about network measurements. I would say that even DIET
 users in a hurry need this package.</td>

 <b>If you don't install this</b>: You won't be able to use at run time
  the results of the benchmarking.<br>

 <b>Note</b>: We may want to provide another storage solution in the future
  since most FAST users (ie, the DIET ones ;) don't really care about
  the ability to retrive the benchmark results from a remove host. If
  so, LDAP is useless and a file storage should be enough. If you get
  bored and want to implement this yourself, you're welcome. Just
  contact me and I'll help you to help me.</td>      

Installing Fast

FAST depends on several other programs and library, and the resulting system is highly distributed. This makes the installation process a bit tedious. This page tries to explain how to do that.

As stated earlier, the system is composed of two subparts. First the bencher, which should be run only once on every machines which will host computational servers, and an interrogation library, which have to be linked in the program wanting to usee FAST forecasts.

This page is organized as follow: we will first install the benching part and its requirements, then install the runtime library, then we will show how to do some benchmarks, and finally, how to install and use a program using FAST.

Installing the bencher and its requirements

The bencher's work is to do some benchmarks of operations on each computational server, and then get a polynomial function modeling these results. For that, it depends on two libraries. Since the benchmarking process can be pretty long, every result is cached in a database using the Berkley Database library. Then, once FAST have all the needed results, the GNU Scientific Library is used to do the polynomial regression.

Once BDB is installed, you can compile FAST to have only the bencher without the interpolations, but that's not that useful since you won't be able to use the result. So I won't explain how to compile FAST in such conditions. Let's install GSL to be able to do the polynomial regression.

Once GSL is installed, the bencher is fully functional. Let's see how to use it.

Doing some benchs

(see the documentation within the tarball)

Installing the runtime library and its requirements

The runtime library is composed of two main parts.

• The first one is able to get the requirements of a computation on a given machine. This part needs the result of the bencher, which are stored in a LDAP database.
• The second part of the runtime library knows about the actual availability of the system, which are given by the Network Weather Service (NWS).

To install NWS, simply do make, make install. Configuring NWS is a bit more tricky, but you should succeed anyway.

To install LDAP, make, make install in enough. Trick: add

schemacheck off

Then, you wan to put the result of benchs in LDAP. When running the benchs, you get some ldif files you have to add in the LDAP tree so that FAST can use them. First of all, make sure your server slapd is running before trying to add anything to the tree

Adding the root of the tree
The first file to add is base.ldif. This file adds the entries constituting the root of the LDAP tree. To add this file to the tree, type a command like the following:

ldapadd -H ldap://veloce:9009 -D
      cn=admin,ou=People,dc=ens-lyon,dc=fr -w toto -x -c -f
      base.ldif<br><br>

where:

• -H ldap://veloce:9009 specifies which LDAP server should be contacted. That's a regular URL, so keep the ldap://, which says the protocol used, change "veloce" to the actual name of your server, and change "9009" to the actual port number on which your server ears.
• -D cn=admin,ou=People,dc=ens-lyon,dc=fr specifies who is trying to do the modification (that's the administrator of the LDAP tree, and should be the same as specified in your slapd.conf
• -w toto is the password of the user trying to modify the tree. Should be the same than specified in slapd.conf.
• -x specifies that we want to use the simpliest way of authentication.
• -c ask to not check the syntax of the ldif file too hardly.
• -f base.ldif gives the name of the file to add.

Adding the result of the benchs
Once the root of the tree is added, you can add the files resulting of the benchmarking on the servers. The command is the same as above for each one.