Quick Start Guide

1. Introduction

AutoPerf is a load testing tool for Web applications. It works in two main modes: capacity analysis mode, and profiling mode. In this section, we will tell you how to quickly get started with running a Web performance test in either of these modes. For a detailed explanation of the various ways of using AutoPerf, and its inner workings and algorithms, see the user manual.

2. Quick Start for Capacity Analysis

In the capacity analysis mode, AutoPerf runs a set of load experiments at different load levels, with the main aim of determining the capacity of the server in terms of the maximum number of users it can support. 

The steps to do this are as follows:

  1. Your Web application to be tested should be deployed on servers and ready to accept HTTP requests.

  2. Choose a client machine for load generation – this should be a separate machine.

  3. Download and Install the AutoPerf Master on the client, and the Profiler (the slave) on each of the server machines.

  4. Configure the client.

  5. Start the server profiler agents.

  6. Start the Master load generator on the client.

2.1 Configuring the client

The client configuration in capacity analysis mode requires editing the following files. These files should be kept in the current working directory from which autoperf command is invoked. Otherwise their path may be specified as described in "File and Directory Selection" part of the section 2.2.2.

  • input.xml: This describes the URLs, the think times, and provides server information.

  • CBMG.txt - This file describes the probabilistic navigation pattern of a user of this website.

2.1.1 Configuring input.xml

The input.xml provided in the distribution, is an example configuration file required for capacity analysis of the WebCalendar application. The application consists of two Servers - a Web Server and a Database Server. The input.xml describes the various URLs that are part of the user session, and provides server information.

Read the comments in the given sample file and edit it to customize it to the application you are testing. Place this in your current working directory. You may rename this file when using the "-fi" switch. Please "File and Directory Selection" part of the section 2.2.2

2.1.2 Configuring CBMG.txt

This file contains the Consumer Behaviour Model Graph (CBMG) – i.e. the probabilistic navigation pattern of the users. This is described by specifying the probability that a user will issue a transaction X after issuing a transaction Y. This is specified in a tabular form in this file. The cbmg.txt file for the example webcalendar application is provided, and has the following matrix.

  Entry ViewEntryByDay ViewEntryByWeek ViewEntryByMonth Exit
Entry 0 0.4 0.4 0.2 0
ViewEntryByDay 0 0.4 0.2 0.1 0.3
ViewEntryByWeek 0 0.3 0.3 0.3 0.1
ViewEntryByMonth 0 0.3 0.4 0.2 0.1
Exit 0 0 0 0 0

The transaction names here correspond to the names provided in the input.xml file and in the same order as specified in the Load tag. The Entry transaction is a reserved keyword used to denote the initial URLs that a user may issue, with the given probabilities. The Exit transaction is a reserved keyword used to denote the probability of exiting a user session. (E.g. this matrix shows that the user may start a session with a ViewEntryByWeek request, with probability 0.4, then go to ViewEntryByMonth with probability 0.3, and then exit with probability 0.1.). This file can be renamed to <ApplicationName>.txt. E.g webcal.txt. The name of the file should be given in the <cbmgfilename> tag in the input.xml

Edit the cbmg.txt file to specify the user navigation pattern for your website. Ensure that

  • Row sums are equal to 1; except for the Exit state.
  • All the entries in the first column are 0s.
  • The last row has all zeros.

2.2 Running AutoPerf

Once the above two files are configured, user can run AutoPerf by initiating Profiler and Client Load Generator as follows:

2.2.1 Profiler

For each host machine which runs any of the servers:

  • Login and open a shell on the host machine and navigate to the profiler code base <AutoPerf>/AutoPerf-Profiler/.

  • For each server process that you want to profile, initiate the profiler as:

./prof -d <port-number>

The port number given here should be same as that given in the configuration file - “input.xml” in the <NodeInfo> tag. The number of Profiler instantiations should be equal to the number of processes that needs to be profiled.

2.2.2 Client Load Generator

On the client side:

  • Login and open a shell on the host machine.
  • Initiate only one instance of the autoperf from any directory as

autoperf [OPTIONSPATTERN [FILE...]

autoperf [OPTIONS] [-fi FILE] [-fc FILE] [-fm FILE]

OPTIONS

The options corresponding to a Mode of Operation. For capacity analysis mode, following option is required:

-c

Other options, depending on the mode of Operation are discussed in the User Manual.

File and Directory Selection

Absence of the file pattern option(s) will make AutoPerf pick the files from the current working directory ("${PWD}") from which autoperf command was invoked with standard file names "input.xml", "cbmg.txt", "mll.txt". Otherwise one or more options can be given along with autoperf command. File names given along with the file pattern options can be in non-standard format. Please see case 2 in Examples.

  • -fi input.xml

Instructs autoperf to pick the input.xml file

  • -fc cbmg.txt

Instructs autoperf to pick the cbmg.txt file

The input files for inputype="file" (discussed here) have to be present in the same path as input XML file.

EXAMPLES

1. AutoPerf runs in capacity analysis mode and look for the relevant files (with standard names) from the current working directory.

autoperf -c

2. AutoPerf runs in capacity analysis mode and take the "input2.xml" from the ${PWD}/1 folder and the absence of the other optional switches imply that the files will be picked from the current working directory (if required).

autoperf -c -fi 1/input2.xml

3. AutoPerf looks for "input.xml" in the ${PWD}/1 folder, "cbmg.txt" file in ${PWD}/2 folder and "mll.txt" file in the current working directory.

autoperf -c -fi 1/input.xml -fc 2/cbmg.txt

This will start the load generation. You will see some trace output being written to the screen. Note that in this mode, Autoperf automatically selects a set of load levels at which to run the algorithm. Thus you may see load levels oscillating between high and low levels (they will not be in strictly increasing order).

2.3 Capacity Analysis mode summary output 

  • Duration of load generation - Time for which the URLs were fired on the profiler from the load generator for a particular load level.
  • Total number of urls fired on the server - The total number of URLs fired are determined by "Throughput convergence" and "Service Time Granularity" algorithms (in capacity analysis mode).
  • Number of URLs that timed out - Indicates the number of URLs that got timed out. The time out value is a parameter from the Config.properties file. If a particular URL's response is not received in that particular time period, it is considered as Timed Out.
  • Throughput - Throughput is measured as ((Total Number of URLs Fired - Total URLs timed out) / (Duration of the load generation))
  • Node - Represents the IP address of the host.
  • Process - Represents the name of the process whose service demand is being reported. It is a parameter in "input.xml" file.
  • CPU Utilisation - The average value of CPU utilisation observed for the period during which the test was run.

The following are the per transaction resource consumption details. In the profiling mode, these correspond to the transaction being profiled. In the capacity analysis mode, this is an average over all the transactions, with respect to the transaction mix generated (the mix is determined by the CBMG). 

Per process, per transaction detail:

  • Service Demand (in ms) - This is the average time that this process used the CPU for to service a single transaction . 
  • Virtual Memory Usage - Indicates the maximum amount of non-swapped physical memory that a task has used (in kilobytes) during its execution.

Per host detail. The following are calculated per host, and represent resource consumption as a result of execution of a single transaction. 

  • Number of disk blocks read 
  • Number of disk blocks written
  • Number of error-free network packets read
  • Number of error-free network packets written

3. Quick start for Profiling mode

In the profiling mode, the purpose of running a load generation experiment is to find the per transaction server resource consumption details of the Web application.  E.g., when we  “profile"  the ViewEntryByWeek transaction, this implies finding details such as CPU execution time required by this request in every server process that it uses, the size of the Virtual Memory used by the server process, network bytes read and written, and the disk bytes read and written by each host machine when it executes one such request.

Profiling information is primarily useful for creating a queuing model of the system being tested.

3.1 Configuring the client

Only the input.xml file is required to be configured for operation in profiling mode. The input.xml file provided shows configuration given for profiling transactions “ViewEntryByWeek” and “ViewEntryByMonth” for the Webcalendar application. Read the comments in the file and edit the file to customize it to your application.

3.2 Running AutoPerf

Once the input.xml file is configured, the user can run AutoPerf by initiating Profiler and Client Load Generator as follows:

3.2.1 Profiler

For each host machine which runs any of the servers:

  • Login and open a shell on the host machine.

  • For each server process that you want to profile, initiate the profiler as:

./prof -d <port-number>

The port number given here should be same as that given in the configuration file - “input.xml” in the <NodeInfo> tag. The number of Profiler instantiations should be equal to the number of processes that needs to be profiled.

3.2.2 Client Load Generator

On the client side:

  • Login and open a shell on the host machine.
  • Initiate only one instance of the autoperf from anywhere as

autoperf [OPTIONSPATTERN [FILE...]

autoperf [OPTIONS] [-fi FILE] [-fc FILE] [-fm FILE]

OPTIONS

The option for profiling mode is as follows:

-p

Other options, depending on the mode of Operation are discussed in the User Manual.

File and Directory Selection format is same as discussed in the Capacity Analysis Mode.

EXAMPLE

AutoPerf looks for "input.xml" in the ${PWD} folder

autoperf -p

3.3 Profiling mode output 

  • Duration of load generation - Time for which the URLs were fired on the profiler from the load generator for a particular load level.
  • Total number of urls fired on the server - The total number of URLs fired are determined by "Throughput convergence" and "Service Time Granularity" algorithms (in capacity analysis mode).
  • Number of URLs that timed out - Indicates the number of URLs that got timed out. The time out value is a parameter from the Config.properties file. If a particular URL's response is not received in that particular time period, it is considered as Timed Out.
  • Throughput - Throughput is measured as ((Total Number of URLs Fired - Total URLs timed out) / (Duration of the load generation))
  • Node - Represents the IP address of the host.
  • Process - Represents the name of the process whose service demand is being reported. It is a parameter in "input.xml" file.
  • CPU Utilisation - The average value of CPU utilisation observed for the period during which the test was run.

The following are the per transaction resource consumption details. In the profiling mode, these correspond to the transaction being profiled. In the capacity analysis mode, this is an average over all the transactions, with respect to the transaction mix generated (the mix is determined by the CBMG). 

Per process, per transaction detail:

  • Service Demand (in ms) - This is the average time that this process used the CPU for to service a single transaction . 
  • Virtual Memory Usage - Indicates the maximum amount of non-swapped physical memory that a task has used (in kilobytes) during its execution.

Per host detail. The following are calculated per host, and represent resource consumption as a result of execution of a single transaction. 

  • Number of disk blocks read 
  • Number of disk blocks written
  • Number of error-free network packets read
  • Number of error-free network packets written

Refer to the detailed manual for a more comprehensive discussion of AutoPerf features.

 

Copyright © 2013 Vikas Goel. All rights reserved