The renewal maintenance has officially ended for Progress iMacros effective November 30, 2023.
This Wiki site will also no longer be moderated from the Progress side.
Thank you again for your business and support.
Sincerely, The Progress Team
Difference between revisions of "Image Recognition"
Line 41: | Line 41: | ||
== Create Images for IMAGESEARCH == | == Create Images for IMAGESEARCH == | ||
− | iMacros V7.10 and higher includes the new [[Image Recognition Wizard]] with allows very easy image creation and testing.[[File:Image recognition wizard.png|center|thumb| | + | iMacros V7.10 and higher includes the new [[Image Recognition Wizard]] with allows very easy image creation and testing.[[File:Image recognition wizard.png|center|thumb|450px|The new [[Image Recognition Wizard]]. Create and test input images in seconds.]] |
Revision as of 23:25, 12 November 2010
Image Recognition PlugIn
The Image Recognition PlugIn is part of the iMacros business editions. It adds two new, extremely powerful commands. The use of image recognition is recommended if you need to test web sites with non-html based functions, such as Macromedia Flash (Shockwave) or Java applets. Unlike DirectScreen commands the IMAGESEARCH and IMAGECLICK commands do not rely on the coordinates of an element but on its visual appearance. Thus buttons and input boxes can be found even if they move around on the screen.
The Image Recognition menu entries allow you to create and test input images for the new image recognition commands.
Image Recognition does not only work with "real" images, but also with text, buttons or any other element that is inside the web browser window!
Demo movie: Web Testing on Disney.com
Search and Click Images/Buttons
The commands IMAGESEARCH and IMAGECLICK are active after the Image Recognition Plugin installation.
Search for an image on a website
The command to search certain images on a website is called IMAGESEARCH and has two parameters, IMAGE and CONFIDENCE. The command works by searching for the appearance of an image on the web site, which is saved locally on your hard drive. The parameter IMAGE sets the image that the Plugin searches for. The parameter CONFIDENCE tells the Plugin how close the image must match the image on the website. A confidence level of 100 means that the input image and the image on the website must be 100% identical - an exact match; otherwise the IMAGESEARCH command generates an ERROR. A confidence level of less than 100, e.g. 95, allows for some deviation in the color of the image, or a change in the image background or foreground colors.
Click (or send text) to a certain area of the website (defined by an image)
The command to click or send content to an image on a web site is called IMAGECLICK. This command has three parameters, IMAGE, CONFIDENCE and CONTENT. The command works by searching for the appearance of an image on the web site, which is saved locally on your hard drive. If this image, supplied by the IMAGE parameter, is found, the CONTENT is send to it. The parameter CONFIDENCE has the same function as in the IMAGESEARCH command.
The IMAGECLICK command is a combination of IMAGESEARCH and DS CMD=CLICK. The following example shows the connection:
IMAGESEARCH IMAGE=pic1.bmp CONFIDENCE=95 DS CMD=CLICK X={{!IMAGEX}} Y={{!IMAGEY}} CONTENT=
The internal variables !IMAGEX and !IMAGEY are the X/Y coordinates of the middle of the position where the "pic1.bmp" image was found. They are set by the IMAGESEARCH command.
Some comments on the CONFIDENCE parameter
A confidence level of 100 means that the input image must match the image found on the web page 100%. The images are identical on the pixel level. This also means that no deviations in size, color or shape are allowed. The search algorithm used for CONFIDENCE=100 is very fast. It is recommended that use 100 if you do not expect any changes in the image. Typical search times are less than a second on a PC with a 2.5 GHz CPU.
A confidence level of less than 100 means that the input image does not have to match the image found on the web page exactly. Deviations in color or shape are allowed. The lower the confidence level, the more deviations are allowed. If you simply want to ignore color changes, a CONFIDENCE level of 95 is recommended. If you lower the confidence level too much, iMacros will detect the wrong image as an acceptable match. For typical web testing applications, levels between 50 and 99 have proven to be useful. Since, the search algorithm uses advanced image recognition technology it is not as fast as a CONFIDENCE=100 search. Also, the search time increases with input image size. Typical search times are between 1-3 seconds on a PC with a 2.5 GHz CPU.
Create Images for IMAGESEARCH
iMacros V7.10 and higher includes the new Image Recognition Wizard with allows very easy image creation and testing.
The information in this chapter is for iMacros V7.05 and older:
To create a new image for use with the IMAGESEARCH or IMAGECLICK commands, you can use any screenshot tool and any image editor.
The following is a suggestion of a good and efficient method using the MSPAINT software. MSPAINT is a simple image editor that is part of the Windows operating system by default.
1. Surf to the website that you want to automate or test using the iMacro Browser.
2. Go to ImageSearch => Create Image File.
iMacros will then take a screenshot of the current web page and open it in MSPAINT.
In our example, we want to automate the Flash calculator, shown at http://www.iopus.com/iim/demo/v4/flash/ and create an image for the button with the number 8.
3. In MSPAINT, choose the "Select" tool (see the red circle in the figure).
4. Use the Select tool and mark an area around the button 8.
5. After you have marked the area around button 8, right-click the mouse and select the "Copy To..." command.
6. A file dialog box opens. By default, we recommend that you store the images in the datasources folder of iMacros.
7. Select a name for the selected area that you want to save, e.g. button8.bmp. You must store the image in "24-bit Bitmap" format.
8. It's that easy; you are finished! You can now point the IMAGE parameter of any IMAGESEARCH or IMAGECLICK command to point to the just created button8.bmp
Example: Automate Flash Chat Web Applet
The diagram shows you that only three images and three lines of macro code are required to automate input into a flash (or Java) based chat applet. The picture below indicates that the procedure to create images for IMAGESEARCH has been done for the three elements.
This example includes a verification that the input is received.
The macro for this application is:
'Fill text into input box IMAGECLICK POS=1 IMAGE=inputbox.bmp CONFIDENCE=95 CONTENT=Hello<SP>Heidelberg 'Click "send" button IMAGECLICK POS=1 IMAGE=sendbutton.bmp CONFIDENCE=95 'Now test if the applet works => look for text "Heidelberg" in applet window IMAGESEARCH POS=1 IMAGE=text.bmp CONFIDENCE=95
If you add STOPWATCH commands to this macro, it can be used to measure the performance of the Flash (or Java) applet.