Four programs located in your installation bin subdirectory start NWS daemon processes, called "hosts", that provide directory services, persistent storage, resource monitoring, and forecasting. Each of these hosts listens for service requests on a particular port. You specify an NWS host by giving the machine it's running on and the port it's listening to in the form machine[:port] (the DNS name or IP address followed optionally by a colon and port number). For example, the NWS host specified as vermillion.ufo.edu:8090 is a process located on the machine vermillion.ufo.edu that listens on port 8090.
The other programs located in the bin subdirectory are utility programs designed to
Most of the NWS programs require you to enter host names, port numbers, and other initialization information. You can specify default values for much of this information, either by setting environment variables before running the programs or by storing the defaults in the nws initialization file ~/.nwsrc. The NWS programs recognize each of the names listed below. During execution the programs look first for an environment variable with this name; if none has been set, they then search the initialization file for a line consisting of that name followed by an equals sign and a value. Lines in the initialization file that begin with "#" are ignored, so you can include comments in your file. Any values you specify via the command line override the environment values.
| EXTRACT_FIELDS | The fields printed by nws_extract. If this variable is not set, the program prints, in order, time, measurement, MAE forecast, MAE error, MSE forecast, MSE error, source, destination, and resource for each forecast. |
| FORECASTER_PORT | The port forecasters should listen to for incoming messages. Forecasters use port 8070 if this variable is not set. |
| JOURNAL_SIZE | Then number of records memories store in their journals, which record all measurements stored by memories. Memories store 2000 records if this variable is not set. Memory journals are intended for future NWS development, so you should have no need to set this variable. |
| MEMORY_DIR | The directory where memories should store measurements. Memories use the current working directory if this variable is not set. |
| MEMORY_PORT | The port memories should listen to for incoming messages. Memories use port 8050 if this variable is not set. |
| MEMORY_SIZE | The number of measurements memories should retain for each resource. Memories store 2000 measurements if this variable is not set. |
| NAME_SERVER | The host specification of the name server other hosts should contact for directory information. The default name server is "noname", so you must always specify a name server through environment variables, the initialization file, or the command line. |
| NAME_SERVER_PORT | The port name servers should listen to for incoming messages. Name servers use port 8090 if this variable is not set. |
| NAME_SERVER_LDAP | If this variable exists, regardless of its value, then NWS processes will use LDAP instead of the NWS wire protocol to communicate with name servers. |
| SENSOR_MEMORY | The host specification of the memory that sensors should contact in order to store their resource measurements. Sensors contact the memory running on the same machine, listening to the default memory port, if this variable is not set. |
| SENSOR_PORT | The port sensors should listen to for incoming messages. Sensors use port 8060 if this variable is not set. |
A typical ~/.nwsrc file might look like this:
NAME_SERVER = vermillion.ufo.edu NAME_SERVER_PORT = 8888 MEMORY_DIR = /tmp/nwsinfo
This would indicate that hosts should contact vermillion.ufo.edu at port 8888 for registration information. The third line indicates that memories should place measurements into files in the directory /tmp/nwsinfo.
You'll probably want to start by setting up an NWS name server. This is an NWS host that provides all other hosts with information about the location and abilities of other NWS hosts. You can use the nws_search program to view registrations stored with a name server. Run the program as a background process.
nws_nameserver [-c seconds] [-e file] [-f file] [-l file] [-p port] &
| -c | Specifies how frequently the name server should delete expired registrations from the registration file. The default is 3600 (once an hour). |
| -e | Specifies a file where the name server should record error messages. By default, error messages are sent to standard error. |
| -f | Specifies a file where the name server should store registrations. The default is the file registrations in the current working directory. |
| -l | Specifies a file where the name server should record log messages. By default, log messages are sent to standard output. |
| -p | Specifies the port that the name server should listen to for messages. This overrides any default value from the environment or ~/.nwsrc. |
This command, executed on orange.ufo.edu, starts an NWS name server that uses the default port and stores registrations and message output in the directory $HOME/NWSdir:
nws_nameserver -e $HOME/NWSdir/nameserver.err \
-f $HOME/NWSdir/registrations \
-l $HOME/NWSdir/nameserver.log &
NWS registrations follow the LDAP model. Each registered object consists of a set of attributes, each of which has a name and a value. Every object has a name attribute, which (usually uniquely) identifies the object, and an objectclass attribute, which specifies what sort of item the object represents. Other attributes are specific to particular objectclass values.
NWS host registrations have an objectclass value of nwsHost. The other attributes associated with NWS host registrations are:
| hostType | One of forecaster, memory, or sensor. No registration record is presently entered for the name server itself. |
| ipAddress | The IP address used to contact the host. This attribute may have multiple values if the host can be contacted through multiple addresses. |
| owner | The login of the person who started the host. |
| port | The port on which the host receives messages. |
| started | The date and time when the host was started. |
| version | The version of the NWS sources used to create the host. |
NWS sensors register objects with an objectclass value of nwsSkill. These objects represent services that the sensor can perform, and different sensors may register different skills. These skills are used with the nws_start_activity program to configure NWS resource monitoring. In addition to name and objectclass attributes, each registered skill has a host attribute, which is the registration name of the sensor that registered the skill, and a skillName attribute that identifies the type of service that the skill represents. In addition, each skill has a set of option attributes which indicate ways in which the skill can be configured. The values of these configuration options indicate the number and type of configuration values that you can specify for these options when requesting the service. For example, a value of 0_to_10_int for the cpuMonitor skill's nice option indicates that you can specify up to 10 integer values for the nice option when requesting that an NWS sensor begin monitoring CPU availability. The currently supported skills and their associated options are:
| cpuMonitor | This skill monitors the fraction of CPU available to both newly-started and existing processes. It has the option attribute nice:0_to_10_int, which indicates that you can monitor the CPU available for processes running at up to 10 different nice values. If you do not specify any nice values, the NWS monitors CPU availability only for nice 0 processes. |
| diskMonitor | This skill monitors the amount of space available on a disk. It has the option attribute path:1_to_10_string, which indicates that you must specify between 1 and 10 file paths. The sensor monitors the amount of space available on the disks that hold the directories or files specified by these paths. |
| memoryMonitor | This skill monitors the amount of free memory available on the machine. It has no option attributes and is available only on Linux systems. |
| tcpConnectMonitor | This skill monitors the time required to establish a TCP connection between each pair of a set of machines. It has the option attribute target:1_to_100_sensor, which indicates that you must specify between 1 and 100 NWS sensors to contact. (This only applies when you use this skill under the periodic control (see below). When run under the clique control, the tcpConnectMonitor target sensors are taken from the clique membership.) |
| tcpMessageMonitor | This skill monitors the TCP bandwidth and latency between each pair of a set of machines. It has the attributes size:0_to_1_int, message:0_to_1_int, and buffer:0_to_1_int. These allow you to specify, in kilobytes, the total data size, individual send() message size, and socket buffer size used for bandwidth tests. If you do not specify these values, the NWS sends 64kB of data in four 16kB messages, using a socket buffer size of 32kB. Like the tcpConnectMonitor skill, tcpMessageMonitor also has the option attribute target:1_to_100_sensor which must be specified when using the skill under the periodic control. |
Skills operate under the control of sensor control objects, which determine the frequency and timing of experiments. These objects have an objectclass value of nwsControl. In addition to name and objectclass attributes, each registered control has a host attribute, which is the registration name of the sensor that registered the control, and a controlName attribute that identifies the type of control. Different controls support different skills, and each control has a set of configuration options that you can use to guide its operation. The currently supported controls and their associated skills and options are:
| clique | This control coordinates experiments between a set of NWS sensors, called the members of the clique. These members take turns conducting experiments, and only one member will conduct experiments at any time. This is useful for avoiding contention when conducting network experiments between machines. The clique control can be used with either the tcpConnectMonitor or tcpMessageMonitor skill. It has the option attributes member:2_to_100_sensor, indicating that you must specify between 2 and 100 NWS sensors as participating members of the clique, and period:0_to_1_int, which indicates that you may specify the number of seconds between connection attempts between any pair of hosts. If you do not specify a period, the NWS will compute one. |
| periodic | This control can be used with any of the skills listed above to conduct experiments at fixed intervals. It has the option attribute period:0_to_1_int that allows you to specify how often experiments should be conducted. |
When an NWS sensor receives a service request, it registers an nwsActivity object. The attributes of an activity object are the union of those of the control and skill objects used to start the activity, but the option attributes in the activity object indicate the values used for configuration. For example, a cpuMonitor activity might have a nice:0,19 attribute, indicating that the sensor is monitoring CPU availability for both nice 0 and nice 19 processes. Each activity also has a resource attribute that lists the resources it is measuring. The resources that the NWS presently measures are:
| availableCpu | The fraction of CPU available to a newly-started process. This resource is measured by the cpuMonitor skill. |
| bandwidthTcp | The speed with which data can be sent to a target sensor, in megabits per second. This resource is measured by the tcpMessageMonitor skill. |
| connectTimeTcp | The amount of time, in milliseconds, required to establish a TCP connection to a target sensor. This resource is measured by the tcpConnectMonitor skill. |
| currentCpu | The fraction of CPU available to a process that is already running. This resource is measured by the cpuMonitor skill. |
| freeDisk | The amount of space, in megabytes, unused on a disk. This resource is measured by the diskMonitor skill. |
| freeMemory | The amount of space, in Megabytes, unused in memory. This resource is measured by the memoryMonitor skill. |
| latencyTcp | The amount of time, in milliseconds, required to transmit a TCP message to a target sensor. This resource is measured by the tcpMessageMonitor skill. |
An activity generates one or more series of measurements, each of which is registered as an nwsSeries object. The nws_extract program uses these registrations to retrieve measurements for display and forecasting. Along with name and objectclass attributes, nws_series registrations include the following attributes:
| host | This is the registration name of the NWS sensor that is generating the series measurements. |
| memory | This is the registration name of the NWS memory that is storing the series measurements. |
| resource | This indicates what resource the series measures. |
| target | For series that measure resource availability of an inter-machine resource (e.g. bandwidth), this attribute is the specification of the NWS sensor that is the target of the measurements. Single-machine series (e.g. CPU availability) do not have this attribute. |
| unitLabel | This is a string that indicates what units the measurement values in the series represent. For example, the unitLabel attribute value for bandwidth series is Megabits/second, while the value for CPU series is CPU Fraction. |
After setting up your name server, you'll need to start an NWS memory. This is an NWS host that stores and retrieves measurements for other hosts. Run the program as a background process.
nws_memory [-C cache entries] [-d directory] [-e file] [-l file] [-N host] [-L] [-p port] [-s size] &
| -C | Specifies the number of cache entries to use internally. nws_memory maintains an internal write-thru LRU cache of time-series data to speed fetch times. Each entry is fileSize * 50 bytes in size, where fileSize is the size set by the -s described below. The default is 256 entries and a fileSize of 2000 records implying a default memory footprint of approximately 25 megabytes. Increasing this value will aid fetch performance at the expense of in-core memory footprint. |
| -d | Specifies the directory where the memory should store its measurements. This overrides any default value from the environment or ~/.nwsrc. The amount of required storage space depends on how many resources you monitor, but 10-15 MB is typical. |
| -e | Specifies a file where the memory should record error messages. By default, error messages are sent to standard error. |
| -l | Specifies a file where the memory should record log messages. By default, log messages are sent to standard output. |
| -N | Specifies the NWS name server that this memory should contact. This overrides any default value from the environment or ~/.nwsrc. |
| -L | Specifies that this memory should communicate with its name server using the LDAP protocol instead of the NWS wire protocol. |
| -p | Specifies the port that the memory should listen to for messages. This overrides any default value from the environment or ~/.nwsrc. |
| -s | Specifies how many measurements should be retained for each resource. This overrides any default value from the environment or ~/.nwsrc and has a default value of 2000 records. |
This command, executed on orange.ufo.edu, starts an NWS memory that uses port 8055, registers itself with the name server previously started on orange, and stores measurements and message output in the directory $HOME/NWSdir:
nws_memory -d $HOME/NWSdir \
-e $HOME/NWSdir/memory.err \
-l $HOME/NWSdir/memory.log \
-N orange.ufo.edu \
-p 8055 &
This command, executed on red.ufo.edu, starts an NWS memory that uses the default port, stores measurements in the subdirectory RedNWS, and registers itself with the name server previously started on orange:
nws_memory -d RedNWS/ \
-N orange.ufo.edu &
After starting a memory host, you can set up an NWS sensor host on one or more machines to monitor resource availability. After starting a sensor host, you can use the nws_start_activity program to specify which resources it should monitor. Run the program in the background.
nws_sensor [-c yes/no] [-a name] [-e file] [-l file] [-M host] [-N host] [-L] [-p port] &
| -a | Specifies a name for the host other than the default. This is useful for hosts that are multi-homed. |
| -c | Specifies whether or not the sensor should immediately begin monitoring CPU availability for nice 0 processes (the cpuMonitor skill). The default is yes. |
| -e | Specifies a file where the sensor should record error messages. By default, error messages are sent to standard error. |
| -l | Specifies a file where the sensor should record log messages. By default, log messages are sent to standard output. |
| -M | Specifies the NWS memory that the sensor should use to store its measurements. This overrides any default value from the environment or ~/.nwsrc. |
| -N | Specifies the NWS name server that this sensor should contact. This overrides any default value from the environment or ~/.nwsrc. |
| -L | Specifies that this sensor should communicate with its name server using the LDAP protocol instead of the NWS wire protocol. |
| -p | Specifies the port that the sensor should listen to for messages. This overrides any default value from the environment or ~/.nwsrc. |
This command, executed on orange.ufo.edu, starts an NWS sensor that uses the memory started above on red.ufo.edu to store its measurements:
nws_sensor -M red.ufo.edu -N orange.ufo.edu &
This command, executed on red.ufo.edu, also uses the memory running on red:
nws_sensor -N orange.ufo.edu &
NWS forecaster hosts analyze resource availability measurements collected by NWS sensors to forecast how much of the resources will be available in the near future. Since the nws_extract program computes its forecasts directly, you only need to start a forecaster process if you want to write programs that use remote forecasting services. Run the forecaster program in the background.
nws_forecast [-D] [-e file] [-l file] [-N host] [-L] [-p port] &
| -D | Enables debugging output. |
| -e | Specifies a file where the forecaster should record error messages. By default, error messages are sent to standard error. |
| -l | Specifies a file where the forecaster should record log messages. By default, log messages are sent to standard output. |
| -N | Specifies the NWS name server that this forecaster should contact. This overrides any default value from the environment or ~/.nwsrc. |
| -L | Specifies that this forecaster should communicate with its name server using the LDAP protocol instead of the NWS wire protocol. |
| -p | Specifies the port that the forecaster should listen to for messages. This overrides any default value from the environment or ~/.nwsrc. |
This command, executed on green.ufo.edu, starts an NWS forecaster:
nws_forecast -e GreenNWS/fore.err \
-l GreenNWS/fore.log \
-N orange.ufo.edu &
The nws_add_forecasts program is a filter that derives forecasts from a series of measurements. It takes from standard input a series of lines that contain time-stamp/measurement pairs and prints these lines to standard output along with forecast and forecast error values. Any text that follows the time-stamp and measurement is taken to be a series name, and forecasts can be added to multiple series.
For example, given the input
953834769 1.007150 blue.ufo.edu numbers 953834779 1.012400 953834789 1.020350 953834799 1.025690 953834809 1.031060 953834819 1.033760 953834829 0.996740 953834839 0.966310 953834849 0.941820 953834859 0.929860 953834857 0.689050 green.ufo.edu numbers 953834867 0.734150 953834877 0.797280 953834888 0.793330 953834898 0.812850 953834908 0.829850 953834918 0.851220 953834928 0.889380 953834938 0.909760 953834948 0.771710
nws_add_forecast produces the output
953834769 1.007150 1.007150 0.000000 blue.ufo.edu numbers 953834779 1.012400 1.012400 0.005250 953834789 1.020350 1.020350 0.006600 953834799 1.025690 1.025690 0.006180 953834809 1.031060 1.031060 0.005978 953834819 1.033760 1.033760 0.005322 953834829 0.996740 1.010508 0.006850 953834839 0.966310 0.966310 0.010219 953834849 0.941820 0.941820 0.012003 953834859 0.929860 0.929860 0.011998 953834857 0.689050 0.689050 0.000000 green.ufo.edu numbers 953834867 0.734150 0.734150 0.045100 953834877 0.797280 0.797280 0.054115 953834888 0.793330 0.793330 0.037393 953834898 0.812850 0.812850 0.032925 953834908 0.829850 0.829850 0.029740 953834918 0.851220 0.851220 0.028345 953834928 0.889380 0.889380 0.029747 953834938 0.909760 0.909760 0.028576 953834948 0.771710 0.774437 0.025738
The third column of output lists the forecast derived from previous measurements for the series; the fourth column the observed error in the forecasts to that point.
You can use the nws_ctrl_host program to send one of a selection of commands to a running NWS host. These commands control and test the operation of the target hosts.
nws_ctrl_host [-t seconds] [-z] command host [host ...]
| -t | Specifies a time out value for attempts to contact the target hosts. The default is 10 seconds, which may be insufficient for contacting hosts over a very slow network. |
| -z | Suppresses the production of error messages. This can be useful for scripts that automate host testing. |
command must be one of the following. Abbreviations are allowed.
| error | Directs the host to suppress or resume error message output. |
| halt | Directs the host to exit. |
| log | Directs the host to suppress or resume log message output. |
| register | Directs the host to register with a new name server using the NWS wire protocol and start using that in place of its old name server. The first host given on the command line is used as the name server with which the rest of the hosts listed should register. |
| ldap_register | Directs the host to register with a new name server using the LDAP protocol and start using that in place of its old name server. The first host given on the command line is used as the name server with which the rest of the hosts listed should register. |
| test | Directs the host to check its registration with the NWS name server and report the results. |
The nws_halt_activity program allows you to tell the NWS to stop monitoring a resource or set of resources.
nws_halt_activity [-N host] [-L] [-S host] name
| -N | Specifies the NWS name server that contains directory information for the activity. This overrides any default value from the environment or ~/.nwsrc. |
| -L | Specifies that communication with the name server should be done using the LDAP protocol instead of the NWS wire protocol. |
| -S | Specifies a sensor to contact to stop the activity. The program normally determines this information by contacting the name server. Specifying the sensor yourself can sometimes be useful if your connection to the name server is slow, or if you want to use a specific clique member to halt a clique. |
The nws_host program prints IP addresses and DNS names for each address or name you give on the command line. This is similar to the standard tool of the same name that is available on many Unix systems.
nws_host name [...]
A pair of examples:
nws_host www.whitehouse.gov yahoo.com www.whitehouse.gov 198.137.240.92 198.137.240.91 yahoo.com 204.71.200.243 204.71.200.245 nws_host 198.137.240.91 www1.whitehouse.gov 198.137.240.91
The html-hosts script uses the nws_extract program to produce an HTML table of inter-machine bandwidth and latency measurements and forecasts.
html-hosts host host [...]
For example, assuming that the NWS activities were conducting tcpMessageMonitor activities between the machines blue.ufo.edu, green.ufo.edu, orange.ufo.edu and red.ufo.edu, the following command would produce a 4-by-4 table that could be incorporated into a web page:
html-hosts blue.ufo.edu green.ufo.edu orange.ufo.edu red.ufo.edu
The nws-hostadmin script can assist you in the task of tracking NWS hosts. It is especially useful for keeping track of NWS hosts running on remote machines.
nws-hostadmin command [ [-b path] [-B path] [-f path] [-m path] [-M path] [-l login] [-L login] [-s switches] [-S switches] hosts ... ]
| -b -B | Use the specified path to find the NWS binaries for the next host set (-b) or for all subsequent host sets (-B). By default, the script expects to find the NWS binaries in the directory $HOME/nws/bin. |
| -f | Use the specified path to find the NWS hosts data base (see below). By default, the script uses $HOME/.nwshosts. |
| -l -L | Use the specified login as the remote user id for the next host set (-l) or for all subsequent host sets (-L). By default, the script uses $USER. |
| -m -M | Use the specified path as the remote shell program for the next host set (-m) or for all subsequent host sets (-M). By default, the script uses ssh. |
| -s -S | Pass the specified switches when starting the next host set (-s) or when starting all remaining host sets (-S). By default, the script passes -e and -l switches that direct the host to place its error and log output into the directory $HOME/nws/machine. Memories are also passed a -d switch that directs them to store data files in the same directory. |
command must be one of the following. Abbreviations are allowed.
| add | Adds the host to the NWS hosts data base (see below), without affecting the host itself. |
| cycle | Equivalent to halt followed by start. |
| forget | Removes the host from the NWS hosts data base (see below), without affecting the host itself. |
| halt | Sends a message indicating that the host should exit. |
| kill | A stronger form of halt. It sends a SIGKILL signal to the host process(es). |
| purge | Equivalent to halt followed by forget. |
| revive | Equivalent to test, followed by start if the host is dead. |
| start | Begins execution of the host. |
| test | Displays the status of the host. |
hosts has the format machine[:type[:port][,type[:port]...]]. The type is one of forecaster, memory, nameserver, or sensor, indicating the NWS host of interest. Abbreviations can be used. You only need to specify a port if more than one host of a particular type is running on the same machine, or if you're starting a host and wish to use a port other than the default. If only the machine is specified, the script applies the command to all hosts running on the machine.
If hosts is omitted altogether, the script applies the command to all hosts listed in the NWS hosts data base. The script stores in this data base a list of NWS hosts that have been specified as part of either an add or a start command. Each record lists the machine name, host type, port, NWS binary path, and switches for a particular NWS host. When the script is asked to manage NWS hosts, it queries this data base to determine what hosts are running and how they can be contacted. The script uses the nws_ctrl_host program to contact NWS hosts, so this program must be available in some directory listed in your PATH environment variable.
As one example, this command would use rsh to start a memory, sensor, and forecaster on orange.ufo.edu and ssh (the default) to start the same three hosts on yellow.ufo.edu:
nws-hostadmin start -m rsh orange.ufo.edu -b /usr/local/nws yellow.ufo.edu
("-M rsh" would indicate that rsh should be used for both orange and yellow.) The nws binaries on orange.ufo.edu are located in $HOME/nws/bin, while those on yellow.ufo.edu are found in /usr/local/nws. Assuming that the NWS hosts data base did not previously exist, it would now contain six records, one for each started host:
orange.ufo.edu memory:8050 nws/bin orange.ufo.edu sensor:8060 nws/bin orange.ufo.edu forecaster:8070 nws/bin yellow.ufo.edu memory:8050 /usr/local/nws yellow.ufo.edu sensor:8060 /usr/local/nws yellow.ufo.edu forecaster:8070 /usr/local/nws
The subsequent command
nws-hostadmin test
would display the current status of each of these six hosts. If only the sensor and memory running on orange were of interest, then the command
nws-hostadmin test orange.ufo.edu:sen,mem
would display the desired information. The sequence of commands
nws-hostadmin halt yellow.ufo.edu:forecaster nws-hostadmin revive yellow.ufo.edu
would have the effect of restarting the yellow forecaster. The first command terminates the running forecaster, while the second tests the status of all three yellow hosts, restarting the forecaster when it determines that the host is no longer running. A simpler way to do this would be the single command
nws-hostadmin cycle yellow.ufo.edu:forecaster
As it operates, the script displays a status for each NWS host of interest:
| confused | The host is running but responds strangely to test messages. |
| cycled | The host has been halted and started again. |
| dead | The host is not running. |
| forgotten | The host has been removed from the NWS hosts data base. |
| halted | The host has been halted. |
| healthy | The host is responding properly to test messages. |
| killed | The host has been killed. |
| purged | The host has been halted and forgotten. |
| revived | The host was dead and has been started again. |
| sick | The host is running, but it may have lost contact with the name server. |
| started | The host has been started. |
| uncycled | An attempt to restart a host after halting it has failed. |
| unhaltable | An attempt to halt the host has failed. |
| unkillable | An attempt to send a kill signal to the host has failed. |
| unresponsive | The host is running, but it is not responding to test messages. A slow network connection may be the only problem. |
| unrevivable | An attempt to restart a dead host has failed. |
| unstarted | An attempt to start the host has failed. |
The nws_extract program provides easy access to NWS measurements and forecasts.
nws_extract [-a] [-f fieldList] [-h n] [-M host] [-n n] [-N host] [-L] [-t n] [-w] resource [filter] host [host ...]
| -a | Treat all listed machines as experiment sources. By default, the program treats the first machine as the experiment source and the others as experiment destinations. Use this switch to get information about all pairs of machines. |
| -f | Specifies which information you would like displayed. fieldList is a comma-delimited list of forecast field names that you would like to see. The recognized names are destination, the destination host specification, mae_error, the computed error in the Mean Absolute Error forecast, mae_forecast, the Mean Absolute Error forecast itself, mae_method, the forecasting method used to compute the Mean Absolute Error forecast, measurement, the measured resource availability, mse_error, the computed error in the Mean Square Error forecast, mse_forecast, the Mean Square Error forecast itself, mse_method, the forecasting method used to compute the Mean Square Error forecast, resource, the name of the resource measured, source, the source host specification, and time, the time the measurement was taken. These field names may all be abbreviated, so you can, for example, use "me" instead of "measurement" and "t" instead of "time." This overrides any default value from the environment or ~/.nwsrc. |
| -h | Include a column header in the display after every n measurements. A value of 0 suppresses the column header. The default value is 20. |
| -M | Specifies the memory to contact to retrieve measurements. nws_extract normally determines this information by contacting the name server. Specifying this switch may improve response if all the resource measurements you are retrieving are stored by a single memory. |
| -n | Specifies the number of forecasts or measurements to display initially for each series. The program prints 20 by default. |
| -N | Specifies the NWS name server that contains directory information for the NWS hosts of interest. This overrides any default value from the environment or ~/.nwsrc. You only need to specify a name server if the memories that are storing the resource measurements of interest are registered with a name server other than the default and you have not specified a -M switch. |
| -L | Specifies that communication with the name server should be done using the LDAP protocol instead of the NWS wire protocol. |
| -t | Specifies how far in the past to display measurements. By default, the program displays an hour's worth of information. |
| -w | Specifies that the program, after displaying the initial measurements and forecasts, should continue to display additional information as new measurements are taken. The program will continue to update its display until you abort it with ^C. |
resource specifies the resource of interest. See the discussion of registrations above for the list of resources measured by the NWS. You can use abbreviations of any of these standard resource names. Alternately, you can specify a resource name that specifies measurements stored in the NWS using the nws_insert program.
filter has the same format as the filter used with the nws_search program. It can be used to differentiate between multiple measurement series being collected for the same resource. If you do not specify a filter, nws_extract displays all measurement series being collected for the resources you list.
Using the colors clique example described above, you could get a running report of red -> orange bandwidth measurements and forecasts by the command:
nws_extract -w band red.ufo.edu orange.ufo.edu
Dropping the "-w" from this command would give you a single list of measurements and forecasts for the past hour instead of a running report, while adding "-a" would give you both red -> orange and orange -> red values. Similarly,
nws_extract -f time,measurement -t 900 avail purple.ufo.edu
would give you a snapshot report of available CPU fraction over the past 15 minutes.
Supposing that the NWS was collecting green.ufo.edu available cpu measurements for both nice 0 and nice 19 processes, then the commands
nws_extract avail -f time,measurement "(nice=0)" green.ufo.edu nws_extract avail -f time,measurement "(nice=19)" green.ufo.edu nws_extract avail -f time,measurement green.ufo.eduwould display, respectively, only the nice 0 measurements, only the nice 19 measurements, and both sets of measurements.
The nws_search displays objects registered with the NWS name server.
nws_search [-a] [-f yes/no] [-N host] [-L] [-v] filter [attribute ...]
| -a | Indicates that only attribute names should be displayed, not values. |
| -f | Indicates whether to format the program output, listing one attribute per line and padding the names. The default is yes. |
| -N | Specifies the NWS name server of interest. This overrides any default value from the environment or ~/.nwsrc. |
| -L | Specifies that communication with the name server should be done using the LDAP protocol instead of the NWS wire protocol. |
| -v | Indicates that only values should be displayed, not attribute names. |
The search filter has a format patterned after LDAP filters and determines which objects should be retrieved. Each term in a search filter specifies, in parentheses, an attribute name and a value or value range which the retrieved objects must have. See the discussion of NWS registrations above for the set of attributes supported by the NWS. nws_search supports the relational operators =, <=, and >=, and wild cards (*) may be included in the value for the = operator. (nws_search recognizes the LDAP "sounds like" operator, ~=, but treats it identically to =.) Logical and, or, and not operations (&, |, !) may be used as prefix operators to combine filter terms into more complex filters. In addition, the shorthand filters activities, cliques, forecasters, hosts, memories, sensors, series and skills (or an abbreviation of these) can be used instead of the longer objectclass specifications. Some examples:
Retrieve all activity registrations: nws_search "(objectclass=nwsActivity)" or nws_search activities Retrieve all clique registrations: nws_search "(&(objectclass=nwsActivity)(control=clique))" or nws_search cliques Retrieve all forecaster host, memory host, or sensor host registrations: nws_search "(hostType=forecaster)" or nws_search forecasters nws_search "(hostType=memory)" or nws_search memories nws_search "(hostType=sensor)" or nws_search sensors Retrieve all control registrations: nws_search "(objectclass=nwsControl)" or nws_search controls Retrieve all host registrations: nws_search "(hostType=*)" nws_search hosts Retrieve all series registrations: nws_search "(objectclass=nwsSeries)" or nws_search series Retrieve all skill registrations: nws_search "(objectclass=nwsSkill)" or nws_search skills Retrieve all hosts running on 111.222.333.444: nws_search "(ip=111.222.333.444)" Retrieve all cliques of which green.ufo.edu is not a member: nws_search "(&(objectclass=nwsActivity)(control=clique)(!(member=*green.ufo.edu:*)))"
The nws_insert program provides a way to store measurements from other programs in an NWS memory. After storing measurements in the NWS, you can use the nws_extract program to generate forecasts based on these measurements.
nws_insert [-f file] [-M host] resource machine [machine]
| -f | Specifies a file name from which the measurements should be read. The program reads from standard input by default. |
| -M | Specifies a memory in which to store the measurements. By default, the program attempts to select the best available memory. |
nws_insert recognizes the standard NWS resource names (see the discussion of NWS registrations above), so you can, for example, store your own bandwidth measurements. However, a more likely use of this program is to store measurements for a resource that the NWS does not currently track, such as disk access time. Any resource name of up to 30 characters can be used for this purpose. The NWS remembers the name you use, and you can later use the same name with nws_extract in order to retrieve measurements or generate forecasts. The two machines can be given as either DNS names or IP addresses. The first specifies the measurement source; the second specifies the measurement destination for inter-machine resources.
Each line of input to nws_insert consists of a pair of numbers. The first indicates the time that the measurement was taken and is represented by a number of seconds since midnight, 1/1/1970 GMT (the value returned by the Unix time() system call). The second is a floating point number that represents the value of the measurement. Blank lines and lines beginning with '#' are ignored; these can be used to format and comment your data files. For example, an input file named "speedo.dat" for nws_insert might contain these lines:
# These are latency measurements taken periodically by the speedo utility. # They measure latency between orange.ufo.edu and green.ufo.edu. 911826158 1.045000 911828415 0.445000 911829922 1.030000 911831030 1.182000 911831947 1.042000 911832926 1.003000 911834240 1.040000 911835179 0.999000 911836339 1.047000 911837431 1.098000 911838593 1.057000 911839682 1.056000 911841052 1.043000 911842529 1.022000 911844053 1.026000 911844935 1.057000 911846314 0.917000 911847821 1.045000 911849317 1.023000 911850663 1.031000
You could store these measurements in the NWS using the command
nws_insert -f speedo.dat speedo_latency orange green
Afterward, you could generate a forecast for these measurements:
nws_extract -h 0 speedo_latency orange green 911826158 1.045000 1.045000 0.225317 Median orange green speedo_latency 911828415 0.445000 1.045000 0.225854 Median 911829922 1.030000 1.045000 0.224959 Median 911831030 1.182000 1.045000 0.224144 Median 911831947 1.042000 1.042000 0.223261 Median 911832926 1.003000 1.042000 0.222392 Median 911834240 1.040000 1.042000 0.221523 Median 911835179 0.999000 1.042000 0.220668 Median 911836339 1.047000 1.045000 0.219813 Median 911837431 1.098000 1.050353 0.218975 Trimmed_Median 911838593 1.057000 1.059176 0.218133 Trimmed_Median 911839682 1.056000 1.057771 0.217297 Adjusted_Mean 911841052 1.043000 1.041021 0.216469 Adjusted_Mean 911842529 1.022000 1.040234 0.215647 Adjusted_Mean 911844053 1.026000 1.039937 0.214831 Adjusted_Mean 911844935 1.057000 1.043271 0.214021 Adjusted_Mean 911846314 0.917000 1.039064 0.213277 Adjusted_Mean 911847821 1.045000 1.042239 0.212478 Adjusted_Mean 911849317 1.023000 1.041830 0.211687 Adjusted_Mean 911850663 1.031000 1.039059 0.210900 Trimmed_Median
The nws_ping program contacts NWS sensors on remote machines and reports the observed latency and bandwidth.
nws_ping [-repeat seconds] [-size experiment,buffer,message] [-timeout seconds] host [ ... ]
| -repeat | Indicates the program should continuously re-establish connections and conduct experiments with the sensors every seconds seconds. If this switch is not present, the program exits after one test. |
| -size | Allows configuration of the data sizes used in communicating to the sensors. The experiment value specifies the total number of kilobytes to transmit, the message value the number of kilobytes per message, and the buffer value the TCP buffer size. If this switch is not present, the program sends 64 kilobytes in four 16-kilobyte messages, using a buffer size of 32 kilobytes. |
| -timeout | Indicates the number of seconds that should be allowed to lapse before a connection is considered to have failed. The default is 10. |
This command measures the bandwidth and latency to blue.ufo.edu and green.ufo.edu:
nws_ping blue.ufo.edu green.ufo.edu (64k,32k,16k) to blue.ufo.edu:8060: bandwidthTcp: 0.963341 Megabits/second latencyTcp: 98.685000 Milliseconds (64k,32k,16k) to green.ufo.edu:8060: bandwidthTcp: 0.387839 Megabits/second latencyTcp: 156.394000 Milliseconds
You can use the start_activity program to control which resources NWS sensors monitor.
start_activity [-F] [-a] [-f file] host [optionAttribute ...]
| -F | Force the sensor to start this activity even a record of it still exists in the nws_nameserver. | -a | Restart all activities of this host (in this case, other arguments are not read). |
| -f | Specifies a file from which option attribute values should be read. Blank lines in this file are ignored, as is any text following a "#". Additional option attribute values may be listed on the command line. |
host specifies the sensor to contact. Each optionAttribute is a name:value pair that specifies one configuration parameter. The name, controlName, and skillName option attributes are valid for all activities. name specifies the name with which the activity is registered with the NWS name server. If you do not specify a name attribute, the program will generate one for you. skillName and controlName determine which resources the activity monitors and how frequently measurements are made. Which other option attribute names are recognized depends on the skill and control. If you do not specify a controlName or a skillName attribute, nws_start_activity will try to determine the values from the other options you list. See the discussion on NWS registrations above for a list of supported skills and controls and their associated option attributes.
For example, here are the contents of a options attributes file that configures a tcpMessageMonitor activity between four machines:
name:colors controlName:clique skillName:tcpMessageMonitor member:orange.ufo.edu member:yellow.ufo.edu member:red.ufo.edu member:green.ufo.edu size:32 period:120
The ordering of the lines and the number of option attributes specified on each line are unimportant. A file with these contents would specify the same option attributes as the one above:
member:yellow.ufo.edu size:32 skillName:tcpMessageMonitor period:120 member:orange.ufo.edu name:colors member:red.ufo.edu controlName:clique
Assuming that this file was named "myclique", a command to start this activity running would be:
nws_start_activity -f myclique red.ufo.edu
Specifying any of the other participating sensors -- orange, yellow, or green -- instead of red as the host to contact would work equally well. If you later wanted to replace green with purple, you would edit the option attributes file to read:
name:colors controlName:clique skillName:tcpMessageMonitor member: orange.ufo.edu member: yellow.ufo.edu member: red.ufo.edu # member: green.ufo.edu member: purple.ufo.edu period: 120
(you could delete the line containing green rather than commenting it), use the nws_halt_activity program to stop colors, then invoke nws_start_activity again:
nws_start_activity -f myclique yellow.ufo.edu
This command would direct the sensor on orange.ufo.edu to begin measuring CPU availability for both nice 0 and nice 19 processes:
nws_start_activity start orange.ufo.edu nice:0 nice:19
NWS measurements are stored using a time-stamp, expressed as a number of seconds since 1/1/1970 GMT (the value returned by the Unix time() system call). The nws_whattime program takes a list of these numbers and displays them in readable form.
nws_whattime time-stamp [...]
For example:
nws_whattime 953834769 911837431 953834769 == Thu Mar 23 10:06:09 2000 911837431 == Mon Nov 23 08:10:31 1998