The renewal maintenance has officially ended for Progress iMacros effective November 30, 2023.
This Wiki site will also no longer be moderated from the Progress side.
Thank you again for your business and support.
Sincerely, The Progress Team
Difference between revisions of "Image Validation"
(14 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
− | == Image Validation [[Image:IMacros-icon.png|iMacros Browser]] [[Image:Ie-icon.png|IE Plug-in]]== | + | == Image Validation [[Image:IMacros-icon.png|iMacros Browser]] [[Image:Ie-icon.png|IE Plug-in]] [[Image:Ff-icon.png|Firefox]] [[Image:Cr-icon.png|Chrome]] [[Image:NGIM-icon.png|iMacros 2021.0]] == |
+ | |||
+ | [[Image:Imagesearch.png|frame|Image Search Menu]] | ||
+ | |||
+ | Powerful image validation support (sometimes also called visual search or pattern recognition) is not part of all iMacros editions (see [[Features Comparison]]). It adds two new, extremely powerful commands. The use of image validation is recommended if you need to test web sites with non-html based functions, such as Macromedia Flash (Shockwave) or Java applets. Unlike [[DS|DirectScreen]] commands the [[IMAGESEARCH]] and [[IMAGECLICK]] commands do not rely on the coordinates of an element but on its visual appearance. Thus buttons and input boxes can be found even if they move around on the screen. | ||
− | |||
− | |||
− | The Image Validation menu entries allow you to create and test input images for the new image | + | The Image Validation menu entries allow you to create and test input images for the new image validation commands. |
Image Validation does not only work with "real" images, but also with text, buttons or any other element that is inside the web browser window! | Image Validation does not only work with "real" images, but also with text, buttons or any other element that is inside the web browser window! | ||
− | Demo movie: [ | + | Demo movie: [https://forum.imacros.net/_uploads/IR/ImageRecognition.htm Web Testing on Disney.com] |
− | == Image | + | == Image Validation Wizard == |
− | [[File:Image recognition wizard.png]] | + | [[File:Image recognition wizard.png|1000 px]] |
+ | |||
+ | * The Image Validation wizard can be launched from an instance of the iMacros Browser. It can also be launched from the iMacros Browser or IE, while recording is in progress. Note however, the wizard cannot be launched from Firefox or Chrome | ||
* The wizard opens with a screenshot of the page which is currently open in the browser. | * The wizard opens with a screenshot of the page which is currently open in the browser. | ||
Line 41: | Line 45: | ||
* The found matches are shown on the screenshot numbered according to their position. The position numbers are the same as iMacros uses in IMAGESEARCH POS=... | * The found matches are shown on the screenshot numbered according to their position. The position numbers are the same as iMacros uses in IMAGESEARCH POS=... | ||
− | * A more detailed information about the matches appears on the Wizard Feedback box [[File:IRFeedbackBox.png| | + | * A more detailed information about the matches appears on the Wizard Feedback box [[File:IRFeedbackBox.png|250px]] |
* The buttons "Test Command(s)" and "Add Command(s)" test the macro command as in "Suggested Command(s)", in the browser window, or add it to the macro which is currently being recorded. | * The buttons "Test Command(s)" and "Add Command(s)" test the macro command as in "Suggested Command(s)", in the browser window, or add it to the macro which is currently being recorded. | ||
Line 47: | Line 51: | ||
== Search and Click Images/Buttons == | == Search and Click Images/Buttons == | ||
− | The commands IMAGESEARCH and IMAGECLICK are active after the Image | + | The commands IMAGESEARCH and IMAGECLICK are active after the Image Validation Plugin installation. |
=== Search for an image on a website === | === Search for an image on a website === | ||
Line 55: | Line 59: | ||
=== Click (or send text) to a certain area of the website (defined by an image) === | === Click (or send text) to a certain area of the website (defined by an image) === | ||
− | The command to click or send content to an image on a web site is called IMAGECLICK. This command has three parameters, IMAGE, CONFIDENCE and CONTENT. The command works by searching for the appearance of an image on the web site, which is saved locally on your hard drive. If this image, supplied by the IMAGE parameter, is found, the CONTENT is send to it. The parameter CONFIDENCE has the same function as in the IMAGESEARCH command. | + | The command to click or send content to an image on a web site is called [[IMAGECLICK]]. This command has three parameters, IMAGE, CONFIDENCE and CONTENT. The command works by searching for the appearance of an image on the web site, which is saved locally on your hard drive. If this image, supplied by the IMAGE parameter, is found, the CONTENT is send to it. The parameter CONFIDENCE has the same function as in the [[IMAGESEARCH]] command. |
− | The IMAGECLICK command is a combination of [[IMAGESEARCH]] and [[DS]] CMD=CLICK. The following example shows the connection: | + | The [[IMAGECLICK]] command is a combination of [[IMAGESEARCH]] and [[DS]] CMD=CLICK. The following example shows the connection: |
IMAGESEARCH IMAGE=pic1.bmp CONFIDENCE=95 | IMAGESEARCH IMAGE=pic1.bmp CONFIDENCE=95 | ||
Line 64: | Line 68: | ||
[[Image:Imageclick.png|left|frame|Fig 1: Coordinates returned by IMAGESEARCH]] | [[Image:Imageclick.png|left|frame|Fig 1: Coordinates returned by IMAGESEARCH]] | ||
− | The internal variables !IMAGEX and !IMAGEY are the X/Y coordinates of the middle of the position where the "pic1.bmp" image was found. They are set by the IMAGESEARCH command. | + | The internal variables [[!IMAGEX]] and [[!IMAGEY]] are the X/Y coordinates of the middle of the position where the "pic1.bmp" image was found. They are set by the [[IMAGESEARCH]] command. |
=== Some comments on the CONFIDENCE parameter === | === Some comments on the CONFIDENCE parameter === | ||
A confidence level of less than 100 means that the input image does not have to match the image found on the web page exactly. Deviations in color or shape are allowed. The lower the confidence level, the more deviations are allowed. If you simply want to ignore color changes, a CONFIDENCE level of 95 is recommended. If you lower the confidence level too much, iMacros will detect the wrong image as an acceptable match. For typical web testing applications, levels between 50 and 99 have proven to be useful. Since, the search algorithm uses advanced image recognition technology it is not as fast as a CONFIDENCE=100 search. Also, the search time increases with input image size. Typical search times are between 1-3 seconds on a PC with a 2.5 GHz CPU. | A confidence level of less than 100 means that the input image does not have to match the image found on the web page exactly. Deviations in color or shape are allowed. The lower the confidence level, the more deviations are allowed. If you simply want to ignore color changes, a CONFIDENCE level of 95 is recommended. If you lower the confidence level too much, iMacros will detect the wrong image as an acceptable match. For typical web testing applications, levels between 50 and 99 have proven to be useful. Since, the search algorithm uses advanced image recognition technology it is not as fast as a CONFIDENCE=100 search. Also, the search time increases with input image size. Typical search times are between 1-3 seconds on a PC with a 2.5 GHz CPU. | ||
− | |||
− | |||
== Example: Automate Flash Chat Web Applet == | == Example: Automate Flash Chat Web Applet == | ||
Line 94: | Line 96: | ||
* [http://wiki.imacros.net/FAQ#Q:_What_factors_can_cause_a_macro_that_uses_DirectScreen_or_image_recognition_to_fail_on_different_machines.3F What factors can cause the image recognition or DirectScreen to fail on one machine whereas it is working on others?] | * [http://wiki.imacros.net/FAQ#Q:_What_factors_can_cause_a_macro_that_uses_DirectScreen_or_image_recognition_to_fail_on_different_machines.3F What factors can cause the image recognition or DirectScreen to fail on one machine whereas it is working on others?] | ||
− | * [ | + | * [https://forum.imacros.net/viewtopic.php?t=9543 Which technologies are supported in tray/silent mode] |
Latest revision as of 10:56, 16 April 2021
Image Validation
Powerful image validation support (sometimes also called visual search or pattern recognition) is not part of all iMacros editions (see Features Comparison). It adds two new, extremely powerful commands. The use of image validation is recommended if you need to test web sites with non-html based functions, such as Macromedia Flash (Shockwave) or Java applets. Unlike DirectScreen commands the IMAGESEARCH and IMAGECLICK commands do not rely on the coordinates of an element but on its visual appearance. Thus buttons and input boxes can be found even if they move around on the screen.
The Image Validation menu entries allow you to create and test input images for the new image validation commands.
Image Validation does not only work with "real" images, but also with text, buttons or any other element that is inside the web browser window!
Demo movie: Web Testing on Disney.com
Image Validation Wizard
- The Image Validation wizard can be launched from an instance of the iMacros Browser. It can also be launched from the iMacros Browser or IE, while recording is in progress. Note however, the wizard cannot be launched from Firefox or Chrome
- The wizard opens with a screenshot of the page which is currently open in the browser.
- If you check the box "Only visible page area", a new screenshot is taken containing only the part of the page which is currently visible on the browser.
- After you are satisfied with the screenshot, you can either choose an image file which was previously saved, or you can select a piece of the screenshot.
- To select an image for further search, the "select picker" has to be enabled . Now just hold down the left mouse button at the top left corner of the desired selection and drag it until the bottom right corner.
- As soon as you release the left mouse button, the selected area is shown on the right panel.
- The check box "Zoom to fit" performs a zoom of the selected image either in or out until it fits in that space. The aspect ratio is kept.
- The new image is not yet saved to disk and cannot yet be searched on the page screenshot. Save it first. The Wizard suggests a file name based on the current time, but you can change that name, for that just edit the text box containing the image name.
- A mouse click over the screenshot cancels any previous selection or drawn search results.
- When the "Save & Search" button is pressed a search is automatically done. Also every slider movement (to adjust the confidence threshold) triggers a search.
- If you choose an image previously saved in the file name drop-down, a search is automatic performed as soon as you close the drop down.
- If you change the confidence threshold with the keyboard arrow keys (finer changes!), do not forget to click over the slider button to trigger a new search.
- The found matches are shown on the screenshot numbered according to their position. The position numbers are the same as iMacros uses in IMAGESEARCH POS=...
- The buttons "Test Command(s)" and "Add Command(s)" test the macro command as in "Suggested Command(s)", in the browser window, or add it to the macro which is currently being recorded.
Search and Click Images/Buttons
The commands IMAGESEARCH and IMAGECLICK are active after the Image Validation Plugin installation.
Search for an image on a website
The command to search certain images on a website is called IMAGESEARCH and has two parameters, IMAGE and CONFIDENCE. The command works by searching for the appearance of an image on the web site, which is saved locally on your hard drive. The parameter IMAGE sets the image that the Plugin searches for. The parameter CONFIDENCE tells the Plugin how close the image must match the image on the website. A confidence level of 100 means that the input image and the image on the website must be 100% identical - an exact match; otherwise the IMAGESEARCH command generates an ERROR. A confidence level of less than 100, e.g. 95, allows for some deviation in the color of the image, or a change in the image background or foreground colors.
Click (or send text) to a certain area of the website (defined by an image)
The command to click or send content to an image on a web site is called IMAGECLICK. This command has three parameters, IMAGE, CONFIDENCE and CONTENT. The command works by searching for the appearance of an image on the web site, which is saved locally on your hard drive. If this image, supplied by the IMAGE parameter, is found, the CONTENT is send to it. The parameter CONFIDENCE has the same function as in the IMAGESEARCH command.
The IMAGECLICK command is a combination of IMAGESEARCH and DS CMD=CLICK. The following example shows the connection:
IMAGESEARCH IMAGE=pic1.bmp CONFIDENCE=95 DS CMD=CLICK X={{!IMAGEX}} Y={{!IMAGEY}} CONTENT=
The internal variables !IMAGEX and !IMAGEY are the X/Y coordinates of the middle of the position where the "pic1.bmp" image was found. They are set by the IMAGESEARCH command.
Some comments on the CONFIDENCE parameter
A confidence level of less than 100 means that the input image does not have to match the image found on the web page exactly. Deviations in color or shape are allowed. The lower the confidence level, the more deviations are allowed. If you simply want to ignore color changes, a CONFIDENCE level of 95 is recommended. If you lower the confidence level too much, iMacros will detect the wrong image as an acceptable match. For typical web testing applications, levels between 50 and 99 have proven to be useful. Since, the search algorithm uses advanced image recognition technology it is not as fast as a CONFIDENCE=100 search. Also, the search time increases with input image size. Typical search times are between 1-3 seconds on a PC with a 2.5 GHz CPU.
Example: Automate Flash Chat Web Applet
The diagram shows you that only three images and three lines of macro code are required to automate input into a flash (or Java) based chat applet. The picture below indicates that the procedure to create images for IMAGESEARCH has been done for the three elements.
This example includes a verification that the input is received.
The macro for this application is:
'Fill text into input box IMAGECLICK POS=1 IMAGE=inputbox.bmp CONFIDENCE=95 CONTENT=Hello<SP>Heidelberg 'Click "send" button IMAGECLICK POS=1 IMAGE=sendbutton.bmp CONFIDENCE=95 'Now test if the applet works => look for text "Heidelberg" in applet window IMAGESEARCH POS=1 IMAGE=text.bmp CONFIDENCE=95
If you add STOPWATCH commands to this macro, it can be used to measure the performance of the Flash (or Java) applet.