The renewal maintenance has officially ended for Progress iMacros effective November 30, 2023.
This Wiki site will also no longer be moderated from the Progress side.
Thank you again for your business and support.
Sincerely, The Progress Team
Difference between revisions of "Web Testing"
Line 122: | Line 122: | ||
1. TAG command | 1. TAG command | ||
− | 2. EXTRACT command | + | 2. TAG... EXTRACT command |
3. IMAGESEARCH command | 3. IMAGESEARCH command | ||
Revision as of 19:30, 8 July 2008
Response Time Measurements
(Related example macro: Demo-Stopwatch) (Related example script: get-response-times.vbs)
The STOPWATCH command in iMacros allows you to measure the time that elapses between the first occurrence of the command in a macro ( = stopwatch on) and the second occurrence ( = stopwatch off). By using different identifiers in the STOPWATCH command you can create up to 100 independent measurement points in your macro.
In order to do web site response measurements you need to insert the STOPWATCH statements manually after you recorded your macro. For accurate measurements it is important to set the browsers replay speed to FAST so no artificial delays are added.
Demo Movie: Performance Web Testing with STOPWATCH
iMacros response time measurements always reflects the true user experience as they are measured using a real browser. Therefore response times measured by iMacros include loading times for browser plug-ins such as the Macromedia Flash Player or the Java runtime.
By default the measured times are saved to the Downloads\ directory of your iMacros installation (e.g. C:\Program Files\iMacros\Downloads\). The default file name is macroName_stopwatch.csv. You can instruct iMacros to save the data to a custom file name by setting the built-in variable !FILESTOPWATCH.
The values are comma separated (CSV format) so they can be viewed with any text editor, imported directly in Microsoft Excel or viewed by any other software you use. Additional information about the date and time of the measurements and the calling macro will be added to the response times.
In this example we measure response times of different parts of the iOpus homepage:
VERSION BUILD=4230323 SET !FILESTOPWATCH mydata.csv STOPWATCH ID=total URL GOTO=http://www.iopus.com/imacros/ STOPWATCH ID=1 TAG POS=1 TYPE=A ATTR=HREF:http://www.iopus.com/imacros/compare STOPWATCH ID=1 STOPWATCH ID=store TAG POS=1 TYPE=A ATTR=TXT:US$<SP>149 TAG POS=1 TYPE=INPUT:SUBMIT FORM=NAME:order ATTR=NAME:ORDER_PRODUCT_NOW&&VALUE:Order<SP>Now TAG POS=1 TYPE=A ATTR=HREF:http://www.iopus.com/store STOPWATCH ID=store STOPWATCH ID=total
This macro will create the following data in the "mydata.csv" file - obviously the response times will be different when you replay this macro. The format of the file is:
YYYY/MM/DD, HH:MM:SS, Macro name, ID, time (s)
"2007/08/3","11:56:23","mymacro","1","1.272" "2007/08/3","11:56:32","mymacro","store","8.943" "2007/08/3","11:56:32","mymacro","total","10.21"
For more information please see the Tips for Accurate Web Response Time Measurements.
Automating Response Time Measurements
(Related example script: get-response-times.vbs )
If you want to automate reponse time measurements it is likely that you will call iMacros from another application. Instead of writing the response time to a log file the data can be sent to your application via the Scripting Interface. Simply use the extract feature. The last recorded response time value is stored in the built-in !STOPWATCHTIME variable. You can use this variable as follows to transfer the response time data to the calling script or program:
VERSION BUILD=4230323
SET !FILESTOPWATCH NO
STOPWATCH ID=total URL GOTO=http://www.iopus.com/iim/iim/demo/v4 STOPWATCH ID=1 TAG POS=1 TYPE=A ATTR=HREF:http://www.iopus.com/iim/compare STOPWATCH ID=1 SET !EXTRACTADD {{!STOPWATCHTIME}} STOPWATCH ID=store TAG POS=1 TYPE=A ATTR=TXT:US$<SP>149 TAG POS=1 TYPE=INPUT:SUBMIT FORM=NAME:order ATTR=NAME:ORDER_PRODUCT_NOW&&VALUE:Order<SP>Now TAG POS=1 TYPE=A ATTR=HREF:http://www.iopus.com/store STOPWATCH ID=store SET !EXTRACTADD {{!STOPWATCHTIME}} STOPWATCH ID=total SET !EXTRACTADD {{!STOPWATCHTIME}}
Thus, the data is added to the !EXTRACT variable. Its contents can be obtained via the "iimGetLastExtract" command. In the above example it contains the three recorded response times (Example string):
1.272[EXTRACT]8.943[EXTRACT]10.21[EXTRACT]
Note: "SET !FILESTOPWATCH NO" instructs iMacros not to create a response time log file. This is useful if you intend to return the values via the Scripting Interface.
Tips for Accurate Web Response Time Measurements
- Add a CLEAR statement to your macro. This way you can make sure that the browser cache is cleared before each run. Otherwise iMacros might read the web pages from the cache and not the web server, which would most likely result in lower response times. Whether the cache is actually used depends on the Internet Explorer settings.
- Run the measurements as a loop and average several runs. The "internet speed" can fluctuate from minute to minute even on a fast connection. Therefore differences between each measurement run are normal. To get stable results it is good practice to average several runs. Several common programs such as Microsoft Excel can create averages automatically for you.
- If you compare results between different PC's, please keep in mind that the accuracy depends on the accuracy of the PC clock. This applies to all software that does time measurements on a PC.
- Under normal conditions the processor speed does not influence the measured response times. Only if the PC is so slow that the web page rendering of the browser is slowed down will the CPU speed have an influence on the measured response time. iMacros response time measurements always reflect the true user experience as they are measured using a real browser and the original browser plug-ins such as Macromedia Flash Player or SUN Java runtime.
- On very complex web applications it can happen that a STOPWATCH command around a URL GOTO command correctly measures the time for a page to open, but stops the time measurement when the page is still loading and displaying "Loading". In such a case, please use a keyword check (e. g. with a TAG or IMAGESEARCH command to check that the page is loaded completely. This way you can measure the exact time from page access, to the completion of the page load.
Example: Instead of
STOPWATCH ID=Firstpage URL GOTO=http://www.iopus.com/imacros/demo/v6/flash/ STOPWATCH ID=Firstpage
Please use:
STOPWATCH ID=Firstpage URL GOTO=http://www.iopus.com/imacros/demo/v6/flash/ IMAGESEARCH IMAGE=flashEnter.bmp CONFIDENCE=90 STOPWATCH ID=Firstpage
Testing AJAX websites
Ajax-driven or JavaScript-heavy sites often require different strategies than when testing a regular site: elements appear, disappear and change at any time. Because of its built-in "intelligence" iMacros can automate many AJAX based websites using standard HTML based TAG commands. And it can automate absolutely all AJAX elements with DirectScreen Technology. This allows in-browser testing of even the most complicated AJAX elements, including sliders and drag & drop activity.
Change User Agent
Related example: Set-User-Agent.vbs (VBS script)
Every time you access a web site the browser you use sends a string to the web server containing information about your operating system and the browser you are using. This string might, for example, look like this:
Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)
Sometimes it is desirable to pretend to be a different user agent because some web sites change in behaviour or appearance depending on the user agent. iMacros can simulate all user agent strings with the -useragent command line switch. The command line switch can also be used in iimInit command of the the Scripting Interface:
iret = iim1.iimInit ("-useragent ""Nokia6230/2.0+(04.43)""")
If your user agent contains spaces, please use double quotes ("") around it.
You can see the current user agent of iMacros at http://www.iopus.com/imacros/demo/v6/user-agent.htm
In iMacros for Firefox please use the following command inside the macro:
SET !USERAGENT "New User Agent here".
So with iMacros for Firefox the user agent can be changed during the macro runtime. With Internet Explorer, the user agent is defined when Internet Explorer is started.
Web Testing FAQs
Q: How can I search for a specific keyword on a web page?
When parsing a web site you want to find out if a certain word exists on a web page to trigger further action (like saving the web page, printing the page or running the next macro).
There are many applications of the keyword search feature. For example tracking your search engine ranking, observing a (financial) news web site, get feedback on a failed login, watching web pages for changes or even monitoring trademark infringements automatically.
iMacros offers three options for keyword search (keyword assert):
1. TAG command 2. TAG... EXTRACT command 3. IMAGESEARCH command
Please see below for more details on each option. We also created the "Demo-Keyword-Check.iim" example macro.
Keyword Search with TAG
The TAG command searches for a HTML tag inside the web page. If found, it clicks on it. But TAG can also search for any kind of text on a web page. Thus, any TAG command is automatically a keyword assert command. For example
TAG POS=1 TYPE=STRONG ATTR=TXT:*iMacros*
searches for the word iMacros enclosed by the STRONG HTML tag. If you want to search for text regardless of the format, i.e. the enclosing HTML tag, please use the wildcard character * instead of a HTML tag:
TAG POS=1 TYPE=* ATTR=TXT:*iMacros*
This command is the same as above, but this time the command ignores the formatting of the keyword. This command takes longer, as iMacros needs to scan the complete page, not only certain html tags.
You can also "invert" the keyword search and generate an error if the keyword is found on the page:
TAG POS=1 TYPE=* ATTR=TXT:*iMacros* CONTENT=EVENT:FAIL_IF_FOUND
As the event suggests the TAG command will now generate an error if it succeeds.
Keyword Verification by Extracting Text
You can use the EXTRACT feature and make the keyword the extraction anchor. The general idea is to try and extract a certain piece of information and then check if the extraction result is the Extraction Anchor Not Found (#EANF#) message.
Example: We want to find out if the words "Order completed" are displayed on a web page. If yes, we want to print the page. To search the web page for the test phrase, create a macro called mysearch, which only has two lines:
VERSION BUILD=6140125 TAG POS=1 TYPE=* ATTR=TXT:*Order<SP>completed* EXTRACT=TXT
In this example we are searching the web page for the first occurrence (POS=1) of the keyword "Order completed". If the message #EANF# is returned then the keyword was not found as the keyword is the data extraction anchor. If the keyword was found then the EXTRACT command returns the complete text of the found HTML tag. In our example this could be "Software Order completed".
To print the web page create a macro called print_this. It has only two lines:
VERSION BUILD=6140125 PRINT
To connect both macros together create a small Windows script:
set iim1= CreateObject ("imacros") iret = iim1.iimInit() iplay = iim1.iimPlay("mysearch") extracted_text = iim1.iimGetLastExtract(1) 'test if keyword appeared on website. If iplay = 1 Then If instr (extracted_text, "#EANF#") > 0 Then MsgBox ("Sorry, keywords not found") Else iplay = iim1.iimPlay("print_this") End If End If If iplay < 0 Then MsgBox "Error!" End If
Note: The same procedure can be used to look for several keywords on a page, for example "cat", "dog" and "mouse":
The solution is to use several EXTRACT commands. So in the macro use:
TAG POS=1 TYPE=* ATTR=TXT:*mouse* EXTRACT=TXT TAG POS=1 TYPE=* ATTR=TXT:*cat* EXTRACT=TXT TAG POS=1 TYPE=* ATTR=TXT:*dog* EXTRACT=TXT
In the script you can look at each array element to see if they keyword was found:
iplay = iim1.iimPlay("wsh-extract-rate") If iplay = 1 Then If iim1.iimGetLastExtract(1) <> "#EANF#" Then MsgBox "Keyword CAT found!" If iim1.iimGetLastExtract(2) <> "#EANF#" Then MsgBox "Keyword DOG found!" If iim1.iimGetLastExtract(3) <> "#EANF#" Then MsgBox "Keyword MOUSE found!" End If
Keyword Search with IMAGESEARCH
You can use the IMAGESEARCH command to look for keywords or keyimages. Thus, instead of a text you supply an image of the word:
Example (Image of the word "iMacros"): imacros
The advantage of this command is that it works on any type of web page, including Flash, Java or ActiveX controls.
For more details please see the Image Recognition Plugin chapter.
Q: I want to test that certain images will show up on my site when a page is loaded.
(a) Check that an image is present
The Save Item function includes an image checkpoint. If a image is missing, an error will occur. Example:
CLEAR TAG POS=1 TYPE=IMG ATTR=TXT:*bee.jpg CONTENT=EVENT:SAVEITEM TAG POS=1 TYPE=IMG ATTR=TXT:*shark.jpg CONTENT=EVENT:SAVEITEM
This macro will download two images. If one image is missing on the web page the macro stops with an "image not found" error. To make sure that the missing image is not stored in the browser cache it is recommended that you use the CLEAR command before this test.
This technique is also recommended if you are automating or testing web sites with complex frames. To make sure a specific frame is completely loaded do the above test for a specific image in this frame. iMacros will wait at the TAG.....EVENT:SAVEITEM command until this image appears.
The method is easily implemented if you have information about the image such as its own URL, the link URL or the alternative text. If all you have is the image itself use the IMAGESEARCH command of the Image Recognition Plugin.
(b) Check the content of an image
Please use the IMAGESEARCH command if you need to verify the content of an image.
Q: Does iMacros support checkpoints?
Yes! Please see the Checkpoints page for more details.
Q: Does iMacros support multiple checkpoints in one macro?
By design, a macro stops when an error occurs (see checkpoints). To verify multiple checkpoints there are two strategies:
1. Split the task in several sub macros, with one checkpoint per macro. A macro can be very small, and even contain only one line. Example:
i = iim1.iimPlay ("Checkpoint1") if i < 0 then msgbox ("Checkpoint 1: Macro reports error.") i = iim1.iimPlay ("CODE:TAG POS=2 TYPE=A ATTR=TXT:*Download* ") if i < 0 then msgbox ("Checkpoint 2: Text not found.") i = iim1.iimPlay ("CODE:IMAGESEARCH IMAGE=result.bmp CONFIDENCE=90") if i < 0 then msgbox ("Checkpoint 3: Image not found.")
2. If you only need to verify text you can also use the Data Extraction features of iMacros can return the information to your script, and then process the result according to your business requirements.
Q: Is there any way to verify if any javascript errors have been thrown?
Yes! You can run a regression test and ensure no errors came out of Javascript. To log Javascript error messages, please make sure that the display of Javascript error message is enabled. To enable it, uncheck the "Disable Script Debugging" option in the Internet Explorer Settings Dialog. These settings are also used by the iMacros Browser.
For more details please see the Page Errors chapter and ONERRORDIALOG.
Q: What effect does iMacros itself have on web application response measurements?
The overhead of iMacros itself is very thin. On a 3 GHz Pentium each macro step takes only about 20 ms (0.02s) of processing time. If you compare this to the average page response time the delay caused by iMacros itself is typically much less than 2%. Note that a few commands need more processor time, for example the CLEAR, IMAGESEARCH or IMAGECLICK commands.
Since the iMacros Browser is an IE-compatible browser realistic viewer experience results are guaranteed. For example, your performance measurement results include the browser rendering time.
Q: How can I set up a 24 hours a day, 7 days a week (non-stop) operation with iMacros?
Many applications of iOpus iMacros require continuous operation of the software. Examples are the use of iMacros as part of payment gateway, extracting large volumes of information or web testing applications in general.
Problem: iMacros has the ability to remotely control the web browser and a wide variety of third-party browser plug-ins, such as Java virtual machine (Java applets), Adobe Acrobat Reader (PDF viewer), Macromedia Shockwave (Flash applets) and many others. By design most of these controls are not intended for 24x7 operations and running them repeatedly for several days can lead to undesirable effects, such as increased memory consumption ("memory leaks").
Solution: The Scripting Interface is a compact and reliable component that controls the iMacros browser and thus, all its plug-ins. The Scripting Interface was designed to automate web testing and at the same time to compensate for problems of the Windows Internet components used by the iMacros browser and third-party browser plug-ins.
The image shows stable CPU and memory usage over time running the sample code listed below with the "Demo-Flash" macro. The left part is a task manager snapshot taken on day 1 and the right part is taken on day 30. The snapshots were taken on a Windows Server 2003 with 2 GHz CPU and 512 MB Ram.
A periodic restart is not required. However, as with server application in general you should make sure that the application restarts correctly after the machine is rebooted or reset.
If you need to repeat a certain macro for example five million times do not start a single browser instance and use the Play (Loop) button. Instead use the Scripting Interface for the loop. In VBS code this looks like:
'Sample code A Set iim1= CreateObject ("imacros") For m = 1 to 5000000 iret = iim1.iimInit() iret = iim1.iimPlay("macro1") iret = iim1.iimExit() Next
This command sequence closes the browser after each macro and thus avoids browser memory leaks. Even if the browser or one of its plug-ins "hangs" or"crashes", the Scripting Interface is not influenced by this and will open a fresh browser instance and continue to work. However, closing and re-opening the browser (as the above example does) takes a few seconds on an average PC. So if processing speed is important for your application you can split this task in two loops and use iimExit only, for example, every 1000th loop to save time.
'Sample code B Set iim1= CreateObject ("imacros") For m = 1 to 5000 iret = iim1.iimInit() For n = 1 to 1000 iret = iim1.iimPlay ("macro1") Next iret = iim1.iimExit() Next