Web Testing

From iMacros
Revision as of 11:01, 26 April 2007 by Hm (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Response Time Measurements

(Related example macro: Demo-Stopwatch) (Related example script: get-response-times.vbs)

The STOPWATCH command in iMacros allows you to measure the time that elapses between the first occurrence of the command in a macro ( = stopwatch on) and the second occurrence ( = stopwatch off). By using different identifiers in the STOPWATCH command you can create up to 100 independent measurement points in your macro.

In order to do web site response measurements you need to insert the STOPWATCH statements manually after you recorded your macro. For accurate measurements it is important to set the browsers replay speed to FAST so no artificial delays are added.

iMacros response time measurements always reflects the true user experience as they are measured using a real browser. Therefore response times measured by iMacros include loading times for browser plug-ins such as the Macromedia Flash Player or the Java runtime.

By default the measured times are saved to the Downloads\ directory of your iMacros installation (e.g. C:\Program Files\iMacros\Downloads\). The default file name is macroName_stopwatch.csv. You can instruct iMacros to save the data to a custom file name by setting the built-in variable !FILESTOPWATCH.

The values are comma separated (CSV format) so they can be viewed with any text editor, imported directly in Microsoft Excel or viewed by any other software you use. Additional information about the date and time of the measurements and the calling macro will be added to the response times.

In this example we measure response times of different parts of the iOpus homepage:

VERSION BUILD=4230323  
SET !FILESTOPWATCH mydata.csv     
STOPWATCH ID=total 
URL GOTO=http://www.iopus.com/iim/       
STOPWATCH ID=1   
TAG POS=1 TYPE=A ATTR=HREF:http://www.iopus.com/iim/compare 
STOPWATCH ID=1 
STOPWATCH ID=store    
TAG POS=1 TYPE=A ATTR=TXT:US$<SP>149    
TAG POS=1 TYPE=INPUT:SUBMIT FORM=NAME:order ATTR=NAME:ORDER_PRODUCT_NOW&&VALUE:Order<SP>Now   
TAG POS=1 TYPE=A ATTR=HREF:http://www.iopus.com/store 
STOPWATCH ID=store 
STOPWATCH ID=total 

This macro will create the following data in the "mydata.csv" file - obviously the response times will be different when you replay this macro. The format of the file is:

YYYY/MM/DD, HH:MM:SS, Macro name, ID, time (s) 
"2004/08/3","11:56:23","mymacro","1","1.272" 
"2004/08/3","11:56:32","mymacro","store","8.943" 
"2004/08/3","11:56:32","mymacro","total","10.21" 

For more information please see the Tips for Accurate Web Response Time Measurements.

Automating Response Time Measurements

(Related example script: get-response-times.vbs )

If you want to automate reponse time measurements it is likely that you will call iMacros from another application. Instead of writing the response time to a log file the data can be sent to your application via the Scripting Interface. Simply use the extract feature. The last recorded response time value is stored in the built-in !STOPWATCHTIME variable. You can use this variable as follows to transfer the response time data to the calling script or program:

VERSION BUILD=4230323  

SET !FILESTOPWATCH NO

STOPWATCH ID=total 
URL GOTO=http://www.iopus.com/iim/iim/demo/v4    
STOPWATCH ID=1   
TAG POS=1 TYPE=A ATTR=HREF:http://www.iopus.com/iim/compare 
STOPWATCH ID=1 
SET !EXTRACTADD {{!STOPWATCHTIME}} 
STOPWATCH ID=store    
TAG POS=1 TYPE=A ATTR=TXT:US$<SP>149    
TAG POS=1 TYPE=INPUT:SUBMIT FORM=NAME:order ATTR=NAME:ORDER_PRODUCT_NOW&&VALUE:Order<SP>Now   
TAG POS=1 TYPE=A ATTR=HREF:http://www.iopus.com/store 
STOPWATCH ID=store 
SET !EXTRACTADD {{!STOPWATCHTIME}} 
STOPWATCH ID=total 
SET !EXTRACTADD {{!STOPWATCHTIME}} 

Thus, the data is added to the !EXTRACT variable. Its contents can be obtained via the "iimGetLastExtract" command. In the above example it contains the three recorded response times (Example string):

1.272[EXTRACT]8.943[EXTRACT]10.21[EXTRACT] 

Note: "SET !FILESTOPWATCH NO" instructs iMacros not to create a response time log file. This is useful if you intend to return the values via the Scripting Interface.

Tips for Accurate Web Response Time Measurements

Sample measurements with a modified Demo-Stopwatch macro running on the iOpus.com Dallas monitoring server
  • Add a CLEAR statement to your macro. This way you can make sure that the browser cache is cleared before each run. Otherwise iMacros might read the web pages from the cache and not the web server, which would most likely result in lower response times. Whether the cache is actually used depends on the Internet Explorer settings.
  • Run the measurements as a loop and average several runs. The "internet speed" can fluctuate from minute to minute even on a fast connection. Therefore differences between each measurement run are normal. To get stable results it is good practice to average several runs. Several common programs such as Microsoft Excel can create averages automatically for you.
  • If you compare results between different PC's, please keep in mind that the accuracy depends on the accuracy of the PC clock. This applies to all software that does time measurements on a PC.
  • Under normal conditions the processor speed does not influence the measured response times. Only if the PC is so slow that the web page rendering of the browser is slowed down will the CPU speed have an influence on the measured response time. iMacros response time measurements always reflect the true user experience as they are measured using a real browser and the original browser plug-ins such as Macromedia Flash Player or SUN Java runtime.