The Mstone multi-protocol benchmarking system
Mstone is a stress-testing tool that system administrators can use to determine server performance under load. It was originally developed to support e-mail based systems (which use SMTP, POP, IMAP, or web based protocols), but recently it has been extended to support the subversion revision control system. Mstone should work on all modern Unix platforms1.
Mstone simulates tasks that users might perform. It lets you simulate worst-case loads—for example, Monday at 9 a.m., when most users are checking their mail and sending and receiving messages—so you can determine what maximum load your messaging server can take.
Mstone can be used to test the load-carrying capabilities of a system configuration before it is put into use. Based on how well a mail server handles the loads generated by Mstone, you can determine whether your current deployment configuration is adequate, or whether you need additional or more powerful server machines.
Mstone measures both throughput and response time of the mail server being tested. Mstone reports the number and size of messages sent, the rate at which messages (and bytes) pass through the system, and the average time taken to both transmit a message to the server and receive a message from the server. Mstone can present its results in both text and graphical format.
Mstone is run from a client machine (referred to as testmaster in these instructions) and controls other client machines to generate the requested load. The client machines can be any mix of supported operating systems.
The Mstone testbed consists of the testmaster machine, any testclient machines, the Messaging Server to be tested, plus the network configuration. There are often supporting machines such as a Directory Server that provide the user account names and passwords. (See Mstone Testbed ).
Mstone Testbed
When Mstone is run on the testmaster machine, the following happens:
Command line arguments are combined with configuration files to define the test.
The test processes are executed across the testbed using ssh.
Results are collected and displayed on the HTML results page.
Each test is defined by one or more workload configuration files. If you can't stand editing text files, then you won't like mstone (sorry, we've just never had time to develop a full GUI). The good news is that it is easy to see exactly how the test was run and reproduce it.
If you need to build from source, see Building.txt in the source distribution.
To transfer, extract, and install the all-platforms version of the Mstone archive files follow these steps:
Locate the Mstone archive file for your operating system platform. For example: mstone-4.2-linux-i586.tar.gz
Choose the testmaster machine on which you will run Mstone and copy the archive file to a temporary installation directory, such as /tmp, on that machine. The testmaster machine can be the same machine as the server being tested, or a different machine.
Go to the directory on the testmaster immediately above where you want the Mstone software to be installed.
For
example, if you go to /usr/netscape Mstone will be installed in
/usr/netscape/mstone.
If you already have mstone installed, save your configuration file before extracting the new mstone distribution.
cp -rp mstone/conf /tmp
Extract the Mstone files (See Important Mstone Files for detailed descriptions).
tar xzf /tmp/mstone-4.2.tar.gz
If you installing multiple operating system types, unpack each operating system archive file the same way. This adds the binaries for each supported operating system.
To create an initial configuration for a Mstone test run, perform the following steps:
Go to the mstone directory. (e.g. cd /usr/netscape/mstone). All mstone commands must be run from this directory.
Run ./setup config. This will display the LICENSE and ask if you agree to it.
Answer yes to set up test clients.
Enter the test client machine names.
Configure e-mail or subversion protocols as needed.
Now run ./setup to copy messages and executables to test clients machines. If there are problems with configuration or access permissions, correct them and re-run setup.
If you add/change client machines or change test execution parameters, run ./setup again to copy the necessary files to each test client machine. See ??? for full configuration information.
For accurate test results it is important that the testmaster and all of the testclient machines show the same time on their system clocks. If at all possible, the times should be identical to less than a second.
To check for time consistency run ./checktime.
Checktime uses the client configuration contained in the machine configuration file (general.tbd). The will ssh to each machine and report the time. Because the ssh step takes some time, there might be some variation between the first and last machine, but adjacent machine should be within a second of each other. When in doubt, run checktime again to verify the results.
If there are discrepancies of more than one seconds between machines you need to adjust the system clocks.
The best way to synchronize system clocks is to use the Network Time Protocol (NTP). You can either synchronize once using ntpdate or setup continuous tracking with a ntpd server. These utilities are available for all modern operating systems.
If you do not have an NTP utility, Mstone provides the timesync mode for setup. The timesync utility will only work with operating systems where date supports setting time to the second. You must run ./setup timesync as root and have root ssh permissions on each client machine.
Go the mstone directory. (e.g. cd /usr/netscape/mstone). All mstone commands must be run from this directory.
If you have additional testclient machines, synchronize them as described in Synchronizing Client Machines.
You may need to re-run ./setup to push the mstone binaries and data files to each client machine.
Run the mstone command using the syntax:
./mstone testname [options]
where testname is the test workload file. For example:
./mstone popsmtp
View the results by copying the displayed URL into a browser.
You can specify command line options as needed. Running mstone with no arguments will show the options.
Options specified on the command line override any corresponding configuration parameters in a configuration file. For example, if the configuration file sets the test run time to two hours, but you only want to make a brief trial run of 10 minutes, you would enter this:
./mstone popsmtp -t 10m
mstone command line options describes the command line options you can use with the mstone command.
|
Option |
Description |
|
-a addgraphs |
Specify additional graph sections. |
|
-b banner |
Specifies
the banner (title) to be displayed as part of test results, where
banner is the actual text you want displayed enclosed by
quotation marks. For example: |
|
-c configfile |
Back compatibility with Mstone 4.0/4.1. |
|
-d |
Debug mode. Large amounts of information is output to the stderr log. |
|
-l load |
Specify the number of clients to use. |
|
-n ‘notes’ |
Specify additional notes (‘comments’ in the CONFIG section). |
|
-r ramptime |
Specifies the ramp time, where ramptime is a number and time unit designation. For example, to set a ramp time of 10 seconds: -r 10s |
|
-s sysconfig |
Specify the system data file. |
|
-t time |
Specifies the length of time the test is to run, where time is a number and time unit designation. For example, to set a test run of 20 minutes: -t 20m |
|
-v |
Be more verbose during report processing. |
|
-w workfile |
Specify additional workload files. |
|
PARAM=value |
Set attribute PARAM in the CONFIG section. |
Mstone will update the test report periodically during the test run. The report is first generated about 15 seconds after the test starts and it will then display a URL for accessing the report. It will then update approximately every minute until the test completes. You can also force an update at any time by running the process command. For example:
./process
To stop a test run before it has completed, type Control-C. This will signal mstone to initiate a clean shutdown and generate a report of the test so far. This process usually takes several seconds, but can take up to 5 minutes (if the server is not responding).
All of the results files from each Mstone test run are placed in a time-stamped subdirectory of the results directory. For example, the results of a test run on July 14, 2007 at 14:31 (24 hour clock) are stored in a subdirectory named: results/20070714.1431.
Mstone displays the results of a test run as an HTML page named results.html. There is one results file in each test run subdirectory.
An HTML-formatted index of all test runs is created and maintained in the results/index.html file. This file contains links to the results.html files from every test run.
The top portion of a typical results file is shown in Example Mstone test result page (top portion). (Your results will vary depending on how you configured your test run.)
Example Mstone test result page (top portion)
In this illustration:
The timestamp in the title “Netscape Mstone Results 19991215.2009” gives the time of the test run. In this case, December 15, 1999, at 20:09 hours.
The headline “Wmap/Imap/Pop/Http/Smtp torture test” is the text that was specified for the title variable in the primary configuration file (testname.wld).
The headline “Netscape Messaging Server 4.2” is the text that was specified for the comments variable in the primary configuration file (testname.wld).
Under the line are general information about the test and links to additional supporting files.
Then a series of tables report the test results for each protocol.
Following the test result tables, the results file displays the tests results as a series of graphs.
Mstone breaks each client session into timers. Each timer tracks these variables:
|
Tries |
A try is an attempt to perform an operation. The exact meaning of a try depends on the protocol and the timer. |
|
Errors |
An error indicates that a try did not succeed and the connection will be dropped. A line in the stderr log will indicate the cause of the error. |
|
Bytes written |
See explanation below. |
|
Bytes read |
See explanation below. |
|
Time |
This is the average transaction time for each try. |
|
TMin |
This is the minimum time for a transaction. A TMin of exactly 0 means that no transactions were performed. |
|
TMax |
This is the maximum time for a transaction. A TMax of exactly 0 means that no transactions were performed. |
|
TStd |
This is the standard deviation of the transaction time. |
The bytes written and read represent data bytes for the retrieve and submit timers, and protocol bytes for all others. For readability, values may be displayed in Kilobytes (1024 bytes), Megabytes (1048576 bytes), or Gigabytes (1073741824 bytes).
The number of successful operations is tries minus errors.
Note that actual precision of the time values depends on the precision of the system clock and on statistical sampling. The times are displayed in seconds unless other units are shown.
Each row in the report is a timer for a specific aspect of the protocol test. The one exception is “Total”, which is a sum of the other timers. Details of how each timer is used in each protocol are found in . Here is the current list of possible timers:
|
Total |
The total timer contains a sum of all the other timers. |
|
Connect |
The connect timer is the establishment of the TCP connection to the server. |
|
Banner |
The banner timer includes the message sent from the server when a connection is established. |
|
Login |
The login timer includes sending the user ID and password and having it verified. |
|
Command |
The command timer records all other protocol exchanges between the client and server. |
|
Submit |
The submit timer records message data sent to the server. |
|
Retrieve |
The retrieve timer records messages received from the server. |
|
Headers |
The headers timer records non-message HTML pages received from the server. |
|
Logout |
The logout timer records normally closed connections. |
|
Idle |
The idle timer records time that the client spent sleeping between requests to the server. |
Some stages may be bypasses depending on the protocol and workload. For example, the SMTP protocol will only do a “Login” if useEHLO and useAuthLogin are enabled and the server supports AUTHLOGIN.
There are also some special timers. The block timer counts the number of workload command blocks that have been executed. This can help indicate problems with the workload files.
The mstone results tables contain two sections: The first section reports all the tries, errors, bytes read, bytes written, and the average time value per operation. The second section lists all the operation rates. These are the values reported in the first table divided by the number of minutes that the test was run.
You can also extract additional information from the data that Mstone collects. Summaries of all the information are available in two additional formats for further processing.
A text version of the results is also produced in results/timestamp/results.txt. This is the same information as the HTML report, but in a machine friendly format. All byte counts are in bytes. All time values are in seconds. The protocol name and timer are listed on each line of the result tables. The operation rate entries end in ‘/m’ (for per minute).
A comma separated value summary for each client is generated in results/timestamp/time-*.csv. The CSV files can be imported into most spreadsheets and databases. Each file contains the sum of all client’s timers over the duration of the test for one protocol.
Configuration files describe both how the test will be run (e.g. what are the client machines and client counts) and the test itself. A number of test configurations are supplied and you can add more test configurations as needed. All configuration files are stored in the conf directory.
Each configuration files is structured into sections using common XML notation. A section begins with a type tag and ends with a matching closing tag. Each section can appear multiple times.
Within a section, each line specifies an attribute name and then its value. Some attributes can appear more than once. A ‘#’ (pound sign) indicates a comment to the end of the line. Lines can be continued by ending in a ‘\’ (back slash). Initial whitespace is ignored (except for line continuations).
Examples of all section types and all attributes are in conf/sample.wld. Here is an example CONFIG section demonstrating the line formatting:
<CONFIG>
title \ # title of this test
POP reads
# blank
lines and comment lines don’t matter
#telemetry 1 #
uncomment this to enable telemetry logs
</CONFIG>
All the details of test execution are performed on the ‘testmaster’ machine. The following section types control test execution: CONFIG, CLIENT, PRETEST, MONITOR, POSTTEST, and GRAPH. All other section types are protocol tests.
Configuration files can include other
configuration files using
<include conf/filename.wld>. This
allows all the general configuration information to be kept in one
file and not to have to be repeated in each test workload. A section
must be completely contained in a single file.
There are different kinds of configuration files:
General. The general configuration file specifies parameters that apply to all your Mstone tests. Each test includes conf/general.wld to define common parameters. The mstone utilities setup, cleanup, checktime, and timesync also read conf/general.wld for test client machines and test account information. See General Configuration File (general.wld).
Workload. The workload configuration file specifies the protocols to be tested, the attributes for each protocol, and the test messages to be used. By default, workload configuration files use the filename extension .wld. Mstone provides sample workload files for different kinds of tests: imap.wld, pop.wld, smtp.wld, popimapsmtp.wld, and popsmtp.wld. See Workload Configuration File (testname.wld).
System Data. The system data file specifies any information that you wish to have appended to test results. See System Data File.
Times are specified in configuration commands (or on the command line) by a number followed by one of these time unit designations:
s for seconds
m for minutes
h for hours
d for days
No spaces are permitted between the number and the time unit. The default unit is milliseconds (but there is no suffix to explicitly specify this). For example, '4m' specifies a time of four minutes.
Sizes are specified in configuration commands by a number followed by one of these unit designations:
b for bytes (default)
k for kilobytes (multiply by 1024)
m for megabytes (multiply by 1024x1024)
No spaces are permitted between the number and the size unit.
In addition to taking on constant values, some parameters can now be sampled from probability distributions: startDelay, idleTime, blockTime, loopDelay, numRecips, size, headers, mime, leaveMailOnServer, bandwidth, latency.
The following distributions are currently implemented
Uniform (~unif(a, b))
Normal (~normal(a, b))
Log-Normal (~lognormal(a, b))
Weibull (~weib(a, b, c))
Exponential (~exp(a))
Constant (~const(a))
Binomial (~binomial(a))
Sampled values may also be constrained by a minimum and maximum value. This is particularly useful for e.g. the normal and lognormal distributions. To specify minimum and maximum constraints, append ':[min,max]' to the distribution. See the getdist utility for a way to experiment with random distributions. Note that constraints will create a "lump" at the ends of the generated distribution, and may change the distribution's characteristics (e.g. mean).
Unit suffixes (e.g. "s", and "m") apply to random variable parameters as well as constant values. For unitless parameters, the suffixes corresponding to the random variable's units will still be recognized (i.e. the values will be multiplied by the appropriate factors). This might be considered as a "bug" as much as a "feature."
The general configuration file specifies general configuration parameters that apply to all Mstone tests across this testbed. The general configuration file is named conf/general.wld.
The initial general.wld file contains the following lines:
<Default>
server servername
smtpMailFrom from-address
addressFormat addressformat
loginFormat loginformat
passwdFormat password
numAddresses N
numLogins
N
firstAddress N
firstLogin N
</Default>
<CLIENT
HOSTS=localhost>
</CLIENT>
The workload configuration file specifies the services and commands to be tested, their weights, and the test message files to be used. There are sample workload files for different kinds of tests: imap.wld, pop.wld, smtp.wld, popimapsmtp.wld, popsmtp.wld, and wmap.wld. You can create additional workload files as necessary.
For example, the operative portions of the sample popimapsmtp.wld file look like this:
<include conf/general.wld>
<CONFIG>
# test specific config
title
POP, IMAP, SMTP combined load
clientCount 100
# set an appropriate client count
</CONFIG>
<SMTP
HOSTS=client1>
file en-1k.msg
weight
100
blockTime 1s
</SMTP>
<POP3
HOSTS=client1>
weight 50
blockTime
1s
#leaveMailOnServer 1
</POP3>
#
Since IMAP connections are much longer lived than POP or SMTP,
#
run them on a separate host for clearer results.
<IMAP4
HOSTS=client2,client3>
weight 100
loopTime 60s
numLoops 10
</IMAP4>
This example illustrates most of the features of a test workload. The first line includes the general configuration file to get all the specifics of the system being tested. This allows the test to be independent of most of the system details. We’ll see in a minute that we still have to put some specific machine names in for this unusual test.
Next there is a CONFIG section for the test. Since CONFIG is a special section, the attributes override the values in the general configuration file. This makes the general configuration act as test defaults. The command line can further override any value in CONFIG at test run time. Note that a final copy of the test with all attributes specified will be saved with the test report.
Now a note about the pop-imap-smtp test and why it is tricky. The SMTP portion brings new mail into the system. The SMTP connections are short lived. The POP section reads and deletes all the mail in a mailbox. Again, connections are short lived. The IMAP section logs in, and then spends 10 minutes watching for new mail. This large difference in execution times means that mixing IMAP with POP and SMTP will produce difficult to understand results. A short lived section like SMTP might be waiting 10 minutes to run again. This makes it hard to predict delivery rates.
To make things more predictable, the HOSTS attribute has been used to assign different sections to specific machines. The short-lived protocols (SMTP and POP) have been assigned to client1 and the longer-lived IMAP section has been assigned to client2 and client3. Note an alternate way to do this is to set useGroups in the CONFIG section and then group names would have been used instead of machine names. This would keep all machine names out of the test workload.
In addition to splitting sections by machine, the weight section has been used to control the balance between SMTP and POP. The SMTP section which is weighted 100 and will run twice as often as the POP section which is weighted 50. Note that weights only matter for sections running on the same machine.
The CONFIG section is used to control the test startup and report generation. Nothing in CONFIG directly affects a particular protocol test.
There is a general CONFIG section in conf/general.wld and an additional CONFIG in each test. All CONFIG sections will be merged into one when the test is run. Later values will override earlier ones.
Every parameter in CONFIG can be overridden from the command line using either a predefined switch, or an ATTRIBUTE=value definition. For example: mstone pop -b ‘Initialize store’ would run the pop test with a title (banner) of ‘Initialize store’.
CONFIG section attributes describes the various attributes you can use in the CONFIG section.
|
Attribute |
Description |
|
clientCount N |
Specifies how many clients to use for the test. Same as command line switch ‘-l clients’. |
|
comments string |
Specify the command for the test. This will appear on the third line of the report. Same as command line switch ‘-n notes’. |
|
maxBlocks N |
Specifies
the maximum number of blocks to allow before terminating the
test. If there is only one client, then this number of blocks
will be run. |
|
maxErrors N |
Specifies the maximum number of total errors to allow before terminating the test. |
|
noevents |
If 1, don't use the event based implementation. |
|
rampTime duration |
Specifies the duration of the test load ramp up. This is part of the test time. Defaults to 0. Same as command line switch ‘-r time’. |
|
sysConfig file |
Specifies the System Data file to be used for the ‘Details’ portion of the report. Can be overridden by the command line switch ‘-s filename’. |
|
telemetry N |
If
1, record a log of all transactions. This log is written into the
/var/tmp directory on each client machine. |
|
time duration |
Specify the test duration. Can be overridden by the command line switch ‘-t time’. |
|
title string |
Specify the title of the test. This will appear on the second line of the report. Can be overridden by the command line switch ‘-b title’. |
|
useGroups N |
If 1, use the group name set for each client, instead of its hostname. |
Unless the "noevents" parameter in the CONFIG section is set to 1, mstone will run in "event-queue" mode. Instead of creating one thread per client, it will multiplex clients over a much smaller number of threads. This is intended to increase the tool's scalability. It works by releasing the thread any time a protocol block would sleep due to blockTime, idleTime, or loopDelay.
The number of threads required will approximately equal the maximum number of concurrent operations. Also note that this change does not reduce the number of required file descriptors.
If mstone was build with USE_EVENTS set to off , then “noevents” has no effect.
The CLIENT section defines one or more machines to be used to run the test. The machines are specified using the HOSTS=client1,client2 section attribute. There are often multiple CLIENT sections.
Mstone is designed to simulate very large client counts. In order to do this, it can use multiple client machines, multiple processes on each machine, and multiple threads in each process.
Normally, you specify the number of clients using clientCount or the -l count command line parameter (that's 'l' as in 'load'). Mstone then figures out how to assign those clients across machines. If clientCount is lower than the number of CLIENT sections, then some sections will not be used. The machines used, the process counts, and the thread counts are all shown when the test is started.
Because machines may have differing capabilities, and you may need to define the limits of what each machine can handle, the maximum processes and thread counts will depend on operating system, CPU count, memory, and other system loads. The CLIENT section defines the capabilities of the machine, so that an appropriate client count can be assigned to it during test execution. If more than one machine is specified for a CLIENT section, then all machines will run the same client sub-count.
The example below shows two machines that should always run identical loads. The operating system is SunOS5.6 (i.e. Solaris 2.6). This tells Mstone not to assign more than 500 threads per process or more that 2000 clients total.
<CLIENT
HOSTS=client1,client2>
arch SunOS5.6
maxThreads 500
maxClients 2000
</CLIENT>
CLIENT section attributes describes the various attributes you can use in the CLIENT section.
|
Attribute |
Description |
|
arch string |
Specify the operation system version for this client. By default, this is the same as the testmaster machine. The subdirectories under build define the supported arch types. For example: Linux2.6_x86. |
|
command string |
Specify the Mstone client command to run. Defaults to ./mailclient. |
|
group string |
Specify a group label for this client (only used if useGroups is defined). |
|
maxClients N |
Specifies the maximum number of clients to assign to this machine. |
|
maxThreads N |
Specifies the maximum number of threads to run in a single client process. |
|
processes N |
Specifies the number of processes to run. If this is set, then Mstone will try to pick a thread count so that the right clientCount is run. |
|
tempDir string |
Specify the directory to run from on this machine. |
|
threads N |
Specifies the threads to run. If processes and threads are both set, then this machine will run processes*threads clients regardless of what clientCount is set to. |
Note that the machine name specified in the HOSTS= section attribute must be consistent with any other HOSTS= sections. Just because client1 and client1.example.com resolve to the same machine, does not mean that Mstone understands that they are the same machine.
The PRETEST and POSTTEST sections defines commands to be run before the test starts or after it finishes, respectively. This is often used to check system state (e.g. disk usage, server configuration, etc).
The MONITOR section defines commands to be run during the test. This is often used to monitor system load. Mstone inspects the command string to figure out how to shutdown the monitor at the end of the test. If the string does not contain %c, then it will be shutdown by a signal when the first client exits. If the string does contain %c, then it will be allowed to run to completion. For example: vmstat %f %c.
Specific machine(s) may be specified using the HOSTS=client1 section attribute. If HOSTS is not set, it will run on each client machine used for the test.
The results of PRETEST and POSTTEST will be linked to the final report.
PRETEST section attributes describes the various attributes you can use in the PRETEST section.
|
Attribute |
Description |
|
command command |
Specify the command to run. If there is a %f in the string, it will be replaced by the reporting frequency used my Mstone (in seconds). If there is a %c in the string, it will be replaced by the number of reports needed for the test duration. For example: df -k |
|
name string |
The pattern used to name the file: CLIENT-log-NAME.txt |
|
process command |
Process the raw data with this command to create a cleaned up report for the report. The default is to reference the command output directly. |
|
rsh command |
Specify the remote shell command used to run command. For example: ssh -u mailuser. |
The process command specifies a script to clean up the results from the test. It is called like this: conf/clean_vmstat.pl infile outfile duration frequency width height. Where duration and frequency are in seconds, and width and height are in pixels. See conf/clean_vmstat.pl for an example of how to process a monitor script that runs vmstat. See conf/clean_ifconfig.pl for a more complex example that determines how many bytes have passed through a system's network interface.
The GRAPH section defines each graph to be generated for the final report. If no GRAPH sections are defined, then Mstone will use a pre-defined set of graphs. Additional graphs can be added after the test is complete, by running ./process timestamp -a moregraphs.wld.
In addition to the protocol timers, there is an internally generated “timer” called “connections”. This contains the current server connection count, taking into account connects, logouts, and errors. To graph this timer, specify a variable of “connection”.
GRAPH section attributes describes the various attributes you can use in the GRAPH section.
|
Attribute |
Description |
|
field string |
Specify the field in each timer to be used for the graph. For example: “Time”. If the timer does not contain fields (e.g. blocks or connections), then field should not be set. |
|
label string |
Specify the label to be used on the left side of the graph (the other axis is always time). For example: “Login time”. |
|
title string |
Specify the title to appear above the graph. For example: “Time to get fully logged in”. |
|
variables string |
Specify the timers to be added together to form the graph. For example: “connect, banner, login”. |
The DEFAULT section defines attributes that may be used by all the protocol sections. Note that attributes in DEFAULT that are never used are silently ignored. If there is more than one DEFAULT section (typically in multiple workload files), they will be merged into one when the test is run.
For example: the conf/general.wld file might contain the following default section:
<DEFAULT>
server mailserver.example.com
loginFormat
test%ld
passwdFormat mozilla
numLogins 1000
firstLogin 0
</DEFAULT>
The system data file specifies additional data to be added to the test report. That is, at the end of each test run the information in the system data file is appended to the HTML-formatted test results report. See the sample file is conf/sample.html. The system data file typically describes the system configuration under test, but you can use this file to add any information you wish to the test report.
The system data file is a block of HTML-formatted text that will be inserted into the HTML test report. This is usually used to provide detailed information about the system under test.
A typical system data file looks like this:
<PRE>
<B>
Mailhost.domain.com </B>
Netscape Messaging
Server 4.1 (2/18/99)
E6000
26x250MHz
UltraSPARC-II
6.5GB RAM
A3500 w/ 60x9GB
7200 RPM
store: 5 x (10x9 RAID-0)
queue:
1 x (10x9 RAID-0)
</PRE>
Sample test messages for use by SMTP are stored in the data subdirectory. You can create additional message files as needed. Each file should be a properly formatted message. Mstone does not use or alter anything in the message. Run ./setup to copy data/*.msg and data/*.dat files to each testclient.
Mstone implements a series of protocol tests. The following sections detail what is done in each protocol test. Timers are shown in ALL CAPS. Attributes are shown in italics.
This top level loop is use to run the protocol tests:
Select a section using a random number and the weight of each section.
Run the protocol initialization.
IDLE for idleTime.
Repeat
this part numLoops times (or until test ends)
{
Run the protocol loop body.
IDLE for loopDelay
}
Run the protocol conclusion.
IDLE for blockTime
Note that an error will cause the section to be aborted and the connection closed. However, the final IDLE for blockTime will still be performed. If the test time is up, then all IDLE calls will return immediately and no more loops will be performed.
General protocol section attributes describes attributes that all protocol sections support.
|
Command |
Description |
|
HOSTS=name |
Command section attribute specifying that the commands are to be run only on specific testclient(s). For example, <IMAP4 HOSTS=client1,client3> specifies that this IMAP4 command is to be run on machines client1 and client3. Note that no spaces or tabs are allowed. |
|
blockTime D |
Specifies the minimum time for the entire block to take. The delay is taken even if an error was encountered. Defaults to 0. |
|
idleTime D |
Specifies the minimum delay from starting the connecting to starting the loop. Defaults to 0. |
|
loopDelay D |
Specifies minimum time between loops (e.g. download checks). Defaults to 0. |
|
numLoops N |
Specifies the number of loops to perform before closing the connection. See the protocol descriptions for what is done in each loop. Defaults to 1. |
|
startDelay D |
Specifies startup delay in milliseconds (typically a distribution). |
|
timeout T |
Timeout for network operations (default 60s). |
|
weight N |
Specifies the relative frequency of Mstone running that command. See below for details. Defaults to 100. |
Commands in workload files are assigned a weight factor that determines the relative frequency of the command’s occurrence. For example, if the numeric weights you have assigned to all commands called by the test add up to 25, and a particular command has a weight of 15, then that command has a 60 percent chance of being chosen every time a command is executed, while a command with a weight of 5 has only a 20 percent chance of being chosen. By adjusting the weight factors for the different commands, you can balance them according to your needs.
Weight commands use the syntax:
weight nn
where nn is a number representing the weight factor of that command, or command block. For example:
weight 9
assigns the numeric weight of nine to that command. The default weight is 100.
Note that weights only affect the balance on one machine. If some sections are only run on specific machines, then only the sections on that machine matter.
Wherever loginFormat is used, the first %ld will be replaced by a user number and the second %ld will be replaced by a domain number. The user number is controlled by firstLogin, numLogins, and sequentialLogins. The domain number is controlled by numDomains, firstDomain, and sequentialDomains. Wherever passwordFormat is used, the first %ld will be replaced by the user number.
Wherever addressFormat is used, the first %ld will be replaced by a user number and the second %ld will be replaced by a domain number. The user number is controlled by firstAddress, numAddresses, and sequentialAddresses. The domain number is controlled by numDomains, firstDomain, and sequentialDomains.
The purpose sequentialDomains, sequentialLogins, and sequentialAddresses attributes allow a range of numbers to be covered in a uniform and predictable way. However, the exact interpretation of these attributes may not be obvious when multiple machines are used.
By default, domain, login, and user numbers are picked randomly from the range specified by the appropriate first_ and num_ attributes. The range is divided among all the threads within a process so that there is an efficient sharing of the total range.
When sequential mode is specified, then the numbers proceed in ascending sequence within the range assigned to each thread. If there is more than one process, then each process will cover the same total range. Multiple machines can be assigned non-overlapping total ranges to properly distribute a desired overall test range.
For example: if you specify one machine, two threads, and a range of 100; then one thread will start at 0 and go to 49, and the other thread will start at 50 and go to 99.
The IMAP protocol stores mail on the mail server and allows message management without downloading the whole message. Clients are usually connected for long periods of time and periodically check for new messages.
describes the various attributes you can use in the IMAP section.
|
IMAP section attributes |
|
|
Attribute |
Description |
|
bandwidth N |
Transfer at most N bytes per second. |
|
firstDomain N |
Specify the first domain number to use. The domain number will be inserted into the second %ld in the loginFormat. Defaults to 0. |
|
firstLogin N |
The first (lowest) login number. The login number will be inserted into the first %ld in the loginFormat. Defaults to 0. |
|
latency N |
Set the network latency to N milliseconds. |
|
leaveMailOnServer N |
If 1, don’t delete messages after reading them. |
|
loginFormat format |
Specifies the login string. The first %ld is replaced by the login number, the second %ld (if present) is replaced by the domain number. For example: test%ld or server%ld@dom%ld.com. |
|
loopDelay time |
Specifies how often to check for new mail. The default is every 10 minutes (10m). For example, ‘5s’ means check for mail every five seconds. |
|
numDomains N |
Specifies the number of domains. Defaults to 0. |
|
numlogins N |
Specifies the number of logins. |
|
passwdFormat password |
Specifies the password string. If the string contains %ld, it will be replaced by the login number. For example: test%ld. |
|
portNum N |
Specifies a non-default port number for a service. Defaults to the standard port for IMAP4. |
|
sequentialDomains N |
If 1, domain numbers should proceed in order. By default, use random domain numbers. |
|
sequentialLogins N |
If 1, logins should proceed in order. By default, use random login numbers. |
|
server name |
The name of the mail server being tested. For example: mailhost.example.com. |
|
sslcert filename |
Read the client's cert from filename, which should be in PEM format. |
|
sslkey filename |
Read the client's private key from filename, which should be in PEM format. |
|
ssltunnel {0|1} |
Turn SSL tunneling on or off (default: off). |
|
usetls {0|1} |
Turn STARTTLS/STLS on or off (default: off). Note: starttls and ssltunnel are mutually exlcusive. |
CONNECT by TCP to portNum on server.
Read the BANNER.
LOGIN with loginFormat and passwordFormat. Get response.
IDLE for idleTime.
Repeat
this part numLoops times (or until test ends)
{
COMMAND: select INBOX folder. Get response.
For
each new messages
{
COMMAND: Fetch the message size. Get response.
RETRIEVE message. Get data.
COMMAND: Send noop. Get response.
COMMAND: Mark message as deleted and seen. Get response.
}
COMMAND: Expunge deleted messages. Get response.
IDLE for loopDelay
}
LOGOUT
IDLE for blockTime
The POP protocol transfers messages to the client for viewing and storage. The client connects long enough to download (and usually delete) messages and then disconnects.
describes the various attributes you can use in the POP3 section.
|
POP3 section attributes |
|
|
Attribute |
Description |
|
bandwidth N |
Transfer at most N bytes per second. |
|
firstDomain N |
Specify the first domain number to use. The domain number will be inserted into the second %ld in the loginFormat. Defaults to 0. |
|
firstLogin N |
The first (lowest) login number. The login number will be inserted into the first %ld in the loginFormat. Defaults to 0. |
|
latency N |
Set the network latency to N milliseconds. |
|
leaveMailOnServer N |
If 1, don’t delete messages after reading them. |
|
loginFormat format |
Specifies the login string. The first %ld is replaced by the login number, the second %ld (if present) is replaced by the domain number. For example: test%ld or server%ld@dom%ld.com. |
|
numDomains N |
Specifies the number of domains. Defaults to 0. |
|
numlogins N |
Specifies the number of logins. |
|
passwdFormat password |
Specifies the password string. If the string contains %ld, it will be replaced by the login number. For example: test%ld |
|
portNum N |
Specifies a non-default port number for a service. Defaults to the standard port for POP3. |
|
sequentialDomains N |
If 1, domain numbers should proceed in order. By default, use random domain numbers. |
|
sequentialLogins N |
If 1, logins should proceed in order. By default, use random login numbers. |
|
server name |
The name of the mail server being tested. For example: mailhost.example.com. |
|
sslcert filename |
Read the client's cert from filename, which should be in PEM format. |
|
sslkey filename |
Read the client's private key from filename, which should be in PEM format. |
|
ssltunnel {0|1} |
Turn SSL tunneling on or off (default: off). |
|
usetls {0|1} |
Turn STARTTLS/STLS on or off (default: off). Note: starttls and ssltunnel are mutually exlcusive. |
CONNECT by TCP to portNum on server.