Difference between revisions of "FAQ"

From iMacros
Jump to: navigation, search
(Q: How do I extract information from a table with variable length and/or more than one page?)
(Q: How do I extract information from a table with variable length and/or more than one page?)
Line 529: Line 529:
 
  End Sub
 
  End Sub
  
See also: [http://forum.iopus.com/viewtopic.php?f=7&t=10390&start=15#p33539 Yellow Pages example]
+
=== How do I loop through multiple pages of results? ===
 +
 
 +
Please see this forum thread: [http://forum.iopus.com/viewtopic.php?f=7&t=10390&start=15#p33539 Yellow Pages example]
  
 
=== Q: How do I extract data from a hidden input field? ===
 
=== Q: How do I extract data from a hidden input field? ===

Revision as of 18:51, 29 April 2011

We separated the Frequently Asked Questions we got over the years into Getting started, Installation, Creating macros, Data extraction and Web testing related questions. Some topics in this section are duplicated topics that proved to be hard to find in the online help and the user manual, while others are solutions for a specific task.


Contents

Getting started

Q: What is the functional difference between Internet Explorer with iMacros turned on, and the iMacros browser?

Technically speaking, the iMacros Browser is a "giant wrapper" around the Microsoft Internet Explorer low-level components, so iMacros uses exactly the same code to render a web page as Internet Explorer does.

But the iMacros Browser has extra functions that are not available in the IE add-on:

If you are undecided whether to use IE with iMacros or the iMacros Browser, we recommend to use the iMacros Browser. It is a dedicated application for web automation, data extraction and web testing. But note that you can fully remote control the iMacros Browser, IE and Firefox. So you can always switch easily between the different browsers. To switch, you only need to change the iimInit parameter. No changes in your macro and script code are required.

Background information (for experts): The IExplore.exe of Internet Explorer is only a small application that is instantiated when Internet Explorer is loaded. This executable application uses Internet Explorer components to perform the navigation, history maintenance, favorites maintenance, HTML parsing and rendering, and so on, while it supplies the toolbar and frame for the stand-alone browser. IExplorer.exe directly hosts the Shdocvw.dll component as web browser. The Shdocvw.dll component is more frequently referred to as the WebBrowser Control. The iMacros Browser uses the same Shdocvw.dll component for its web browser.

iMacros for Firefox does not use any IE components. It runs directly inside the Mozilla Firefox web browser and automates the original Firefox web browser. Just like Internet Explorer and the iMacros Browser, Firefox can be remote controlled via the iMacros Web Scripting Interface.

Q: The web page I am accessing requires Internet Explorer (IE). Is this a problem?

No. iMacros is available as Internet Explorer add-in, so you can run all macros directly inside the real Microsoft Internet Explorer browser.

But you can also use the iMacros Browser because the iMacros browser is 100% Internet Explorer compatible. It comes with all Internet Explorer functions and Plugins, including Cookies, ActiveX Controls, AJAX, JavaScript, Silverlight and Adobe Flash.

If you use iMacros for web testing automation, this means that you can run all IE related tests in the iMacros Browser, too. The iMacros Browser will give you exactly the same rendering and error messages as Internet Explorer.

Q: Are there conditional statements like if... then...else in the iMacros macro language?

(Short link to this FAQ)

The iMacros language itself is designed as a descriptive language (similar to HTML) and does not contain conditional statements. We did not add such statements as we do not think that our customers should have to attend week-long seminars just to learn yet another proprietary scripting or programming language. In other words, iMacros commands refer to web page elements, so any programming logic must be put into a script that then uses iMacros to automate the website.

So we created the very powerful Scripting Interface that allows you to use iMacros with every Windows Scripting or programming language on the planet. Examples are VBS, VB, VBA, VB.NET, Perl, Java, Foxpro, C, C++, C#, ASP, ASP.NET, PHP and many more. These languages are used by millions of computer users, are reliable and very well documented. We also added a command line interface for use with batch files and task scheduler.

Basically what you do is:

 ReturnCode = iim1.iimPlay ("YourMacro1")
 if ReturnCode = 1 then
  'Do something
  MsgBox "OK!"
 else
   'Do something else
   ReturnCode = iim1.iimPlay ("YourMacro2")
 end if 

This example uses the Visual Basic syntax, but you can use any programming or scripting language!.

More information about the iMacros language and its relation to HTML is available at TAGs_and_HTML. Examples on how to use the iMacros Scripting Interface are available in sample code chapter.

Related FAQ: How can I do calculations in a macro?

Q: How do I know whether I should use just a macro or if I need to also use a script for my task?

Please see this article for some tips on deciding whether you need to use pure iMacros or the iMacros Scripting Interface (i.e. iMacros together with a scripting or a programming language).

Installation

Q: I create macros for my clients. Do you have a free player?

Yes, you can purchase an Unlimited Users Player License key in our Online Store. This license allows you to ship macros and programs based on iMacros along with a copy of iMacros itself to your users without royalty payment. Technically speaking, this means that if iMacros is unlocked with the iMacros Player license key, your users have all the features of the Scripting Edition available except the ability to record new macros. For more details please see here.

Q: How can I automatically install iMacros?

General information can be found under Distributing iMacros. But basically there are two options:

(1) If you want to distribute the files with your own setup file, please visit our website [1] for an installation script source code. You can then modify this installer for your own needs or simply use it to learn how the software is installed.

(2) You can run the original setup file using command line switches to control the behavior of the installer.

Q: Is there a network installation available for the install of the player?

You can create network installs either by using the command line options provided by the installer (see Distributing the iMacros Setup) or by creating your own, customized setup. In the later case you can even preconfigure all settings, including the license key (see Creating a Custom Install)

Q: What if the iMacros icon does not appear in the IE toolbar?

If you have previously customized your Internet Explorer toolbar the new iMacros icon does not appear by default. You need to add it to the Internet Explorer toolbar manually:

Internet Explorer View

1. To add the icon, open your Internet Explorer and in the menu select View > Toolbars> Customize





Customize Toolbar

2. Select the iMacros icon from the list of "Available Toolbar Buttons" and add it.


Tip: For IE9 specific tips see IE9 Nags. For more general IE tips see IE Nags.



3. Done! The icon appears in the toolbar.

Note: This behavior is a feature of Internet Explorer - and not "bug" with the iMacros software. Microsoft's philosophy behind this seems to be that users who customized their browser toolbar once do not want new icons added automatically.

Q: How to change a iMacros Player installation to a regular license?

Some third party distributions of iMacros Player (BPO 5000, for instance) do not have a place to enter the license key on the Help menu item. In order to change to a regular installation of iMacros, the solution is either

(1) re-install with option "Clean install" checked, or

(2) delete iim.ini manually (typically located in "C:\Documents and Settings\All Users\Documents\iMacros")

When you restart iMacros, the option to enter a license key is restored and you can unlock iMacros full version with your license key

Q: Is the license key tied to one physical machine?

The iMacros license is NOT hardware locked. You can un-install the software on one machine and move it to another machine without problems as often as you need.

The Unlimited User Player license package allows an *unlimited* number of users and installations of the iMacros Player. This includes the public distribution of your solution without royalty payment.

Creating Macros

How to

Q: A link changes every time I visit the web page ("session ID"), how can I replay the macro without error?

By default, iMacros tries to define a an HTML element by its ID or text. If these attributes do not exit, it uses the URL of the element (typically a link). The problem: Some websites create links with a "session id" that is different each time you log in. This is no problem for iMacros.

There are two options to work with such links:

If you want or need to use the URL, edit the macro manually after you recorded it and replace the id (or any other changing part of the string) with the wildcard *. This method is described in detail in Fine Tune TAG Commands.

Related Forum posts:

(tinyurl to above FAQ: http://tinyurl.com/6j4pjh)

Q: How can I print a selected frame?

If the page uses frames and you want to print only a specific frame, select this frame with DS CMD=CLICK first before you use PRINT. An example macro can be found in the PRINT chapter of this manual.

Q: Some websites require me to enter a graphical "security code" or Turing number. I want the login macro to stop at this point and enter the characters manually. How do I set the focus to a input field for manual entry?

To set the focus to a certain input field (or any other element on the screen), please use the DS CMD=CLICK command.

Alternately, you could use the PROMPT command to pause the script and request the desired information from your user. It would look something like this:

Macro Code

  VERSION BUILD=5100314      
  TAB T=1     
  TAB CLOSEALLOTHERS     
  URL GOTO=https://s226.lanxtra.com/servlet/SLogin?template=/0/en/sloginsc.vm&login=true     
  SIZE X=801 Y=602    
  TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:form1 ATTR=NAME:memberNumber CONTENT=TEST
  PROMPT Please<SP>enter<SP>the<SP>value<SP>of<SP>the<SP>Random<SP>Code !VAR1 
  TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:form1 ATTR=NAME:imageword CONTENT={{!VAR1}}
  TAG POS=1 TYPE=INPUT:SUBMIT FORM=NAME:form1 ATTR=NAME:&&VALUE:Next<SP>-->  


This will pop up a box with the selected text and will not continue until the user clicks ok.

Q: Is it possible to create macros that will run on the page displayed in the web browser, rather than using the URL GOTO command to direct it to a web page at the beginning of a macro. The pages I want to run macros on are all set out identically so one macro should work on all the pages.

The URL command is inserted by default every time you start recording a macro. After recording a macro, edit the macro source and manually remove the URL GOTO command. If you want to run the same macro on different pages, yet not all pages are exactly the same, use SET !ERRORIGNORE YES to simply skip TAG commands that cannot be executed.

Q: How can I read and write from a database?

Database integration is very easy with iMacros thanks to its powerful Scripting Interface. iMacros can connect to any Windows database, for example Access or SQL Server, but also MySQL. The Scripting Interface can be used from any Windows programming or scripting language like Excel VBA, Foxpro, Perl, C, C++, Java with no problems.

Please see the following example scripts: "extract-2-file.vbs" (uses wsh-extract-and-fill-part1 and wsh-extract-and-fill-part1 macros) "extract-2-database.vbs" (store information from a website in a database) "file-2-web.vbs", "database-2-web.vbs" (reads a text file or database and submits the content to a website)

Q: How do I make the macro continue (and not stop), if somewhere in the macro I get a timeout or error?

Please add the command SET !ERRORIGNORE YES to your macro.

Q: How can I do calculations in a macro?

For very simple calculations, iMacros offers the ADD command.

But instead of doing calculations in the macro, you can perform the calculations inside a script (e.g. Visual Basic Scripting, VBS). This allows you to use all kinds of more sophisticated calculations, loops (e.g. for....next loops) or conditional logic (e.g. if/then) for automating your web tasks. You can then use iimSet of the Scripting Interface to send values to a macro and the EXTRACT command to return values from a macro to the script.

The file Random-Numbers.vbs demonstrates how to use a script to generate random numbers and modify the macro based on the outcome. Or see the file extract-and-fill.vbs on how to extract a value, subtract a number from it, and re-submit the new value to the website.

Note that VBS files are not started from within the iMacros Browser. You can locate the file (e.g. via Explorer) and then simply double-click on the VBS file. Windows will then start to process the VBS file. The iMacros Browser is started by the VBS file via the iimInit command automatically.

More information about VBS files can be found in our online Windows Scripting Host tutorial or in this manual.

Q: How do I link several macros together or run one after the other?

There are several method to combine different macros.

Copy & Paste iMacros are simple text files, so you can easily combine (merge) several macros by editing the text files. The .iim macro files can be viewed and edited with an editor (e.g. Notepad), see here.

Batch files If all you need is to run several macros, one after the other, you can call them using the command line interface in a batch file. The -noexit command line switch may prove helpful in this context. Example batch files can be found in the Examples/Batch Files folder of your iMacros installation.

Scripting Interface You can use the iimPlay command to run one macro after the other. The following macro continues exactly at the same position where the other macro stopped as the browser window remains open between each macro. Remember to remove the URL GOTO... command from the following macros, if you want it to continue exactly where the other macro left off. To close a browser window, you can use iimExit. The documented example script combine-macros.vbs can be found in the Examples/Windows Scripting Host folder of your iMacros installation.

Q: If someone needs to make that decision meaning that the task would be partially automated, is there a way to make iMacros stop until a users enters a value and continue afterwards?

You can add the PROMPT command to the macro to make iMacros ask for a variable (e.g. the file name) or simply wait for a click on OK. Or use PAUSE.

Q: How can I create a macro that can select one from a series of radio buttons? I want to avoid setting up separate macros for each selection.

You can use variables to select a radio button. For example, instead of using one of these three lines:

TAG POS=1 TYPE=INPUT:RADIO FORM=ACTION:/cgi-bin/Info.cgi ATTR=NAME:lunch CONTENT=yes
TAG POS=1 TYPE=INPUT:RADIO FORM=ACTION:/cgi-bin/Info.cgi ATTR=NAME:lunch CONTENT=no
TAG POS=1 TYPE=INPUT:RADIO FORM=ACTION:/cgi-bin/Info.cgi ATTR=NAME:lunch CONTENT=maybe

You can use:

TAG POS=1 TYPE=INPUT:RADIO FORM=ACTION:/cgi-bin/Info.cgi ATTR=NAME:lunch CONTENT={{choice}}

and set {{choice}} to either "yes", "no" or "maybe". Please note that this value might be case sensitive. This depends on the website that processes this input.

To assign a value to the variable you can use either the SET command, the -var_varname varvalue command line argument, or the iimSet command of the Scripting Interface.

Q: How to display the content of a variable?

For testing and debugging macros it can be useful to display the content of some variables. You can divert the PROMPT command from its intended use for this purpose. If you want to display the value of the two variables, !COL2 and MY_VARIABLE, use the following command

PROMPT {{!COL2}} {{!VAR1}} {{MY_VARIABLE}} 

This will open an input dialog for !VAR1. This is the main purpose of a PROMPT command, but for our current use of the command this can be ignored. The input dialog box displays the value of {{!COL2}} as dialog box text and the value of {{MY_VARIABLE}} as input box default value. This allows you to check if these variables are populated correctly.

Q: How to add missing steps?

As iMacros macros are simple text files, you can copy steps from one macro to another, cf. http://wiki.imacros.net/Insert_Missing_Steps

Q: How to scroll down a page?

DS clicks work only in the visible part of the browser window. So it can be required to scroll a certain part of the page into view. This can be done with a simple TAG command that selects an element around this area. The TAG command not only selects the element, it also scrolls the page so that the element is now visible above the fold. For more information please see the Testing Java, Flash and Silverlight Applets workshop slides.

Q: How to test AJAX?

Please see the following forum posts:

Q: Can a macro start another macro?

A macro itself can not run another macro, but there are two powerful options to achieve the same result:

  • Use the command line interface via batch files.
  • Use the Scripting Interface: This is by far the most powerful and flexible method. With the iimPlay command you can start one macro after the other, and also combine that with logging and error checking. Please see the Combine-Macros.vbs demo script. It starts three different macros in a row.

Q: How to create nested loops?

For simple loops you can use the LOOP button and the !LOOP variable. However, in many applications you need more complex solutions and nested loops. All this can be done in iMacros via the Web Scripting interface.

We created some examples using the popular VB Script language, but please keep in mind you can do the same from any programming or scripting language:

Q: How can I do string manipulation?

String manipulation can be done by sending the text back to the calling script, manipulate it there, and then use it in the next macro:

 i = iim1.iimPlay ("GetText")
 s = iim1.iimGetLastExtract (1)
 
 'Manipulate s here!
 s = Mid(s, 3, 3)
 
 'Send s back to next macro
 i = iim1.iimSet ("myvar", s)
 'The macro UseNewText can use the new value with the {{myvar}} variable now
 i = iim1.iimPlay ("UseNewText")

Q: How to pass values from one macro to the next?

Please see Can you pass the coordinates from a macro to another?.

Q: How do I avoid following a link or "clicking" a button using the TAG command?

If you want to use a button or a link as reference, you should tag it with TAG ... EXTRACT=TXT, to avoid following the link or "pushing" the button. In that case, do not forget to use SET !EXTRACT NULL, to clear the extract variable before the real extract.

Q: How do I click a button on the browser toolbar, select a browser menu item, or interact with another browser plug-in?

iMacros can only automate interactions within the web page itself, and not with the browser's menu items, toolbars or other plug-ins.

Q: How can I tag an element in a frame if the frame number is different each time?

Please see the forum topic Tag elements in frames with changing frame numbers

Q: Is there any particular command for right click mouse and choosing a menu item from the list when automating a Flash application?

iMacros cannot currently automate menu selections from a context-sensitive pop-up menu (for example, the menu that pops up when you right-click a Flash/Adobe/Java applet).

Q: How can I fill a TinyMCE editor field?

You can actually control TinyMCE simply using it's JavaScript interface and the URL command. This line should do the trick:

URL GOTO=javascript:void(tinyMCE.editors['EDITOR_NAME'].setContent('EDITOR_CONTENTS'));

where EDITOR_NAME is name of the textarea the editor is associated with and EDITOR_CONTENTS is what you want to set the contents to (remember to keep the iMacros syntax - <SP> instead of space etc.).

Example macro:

VERSION BUILD=7031111 RECORDER=FX
URL GOTO=http://tinymce.moxiecode.com/tryit/full.php
WAIT SECONDS=5
URL GOTO=javascript:void(tinyMCE.editors['content'].setContent('My<SP>content<SP>filled<SP>by<SP>iMacros'));

(should open the TinyMCE example page and after 5 seconds change the editor content to My content filled by iMacros)

Related forum thread: TinyMCE Editor

Issues during Replay

Q: Does iMacros work with every web site?

The macros recorded with iMacros work on almost all sites. Even if the standard click methods fail (for example on Flash of Java applets) the DirectScreen command can be used in almost all cases. If you think you have found a web site where iMacros does not work please let us know. We are very interested to hear of such cases and will assist you in finding a solution.

Q: We have a macro that goes to a web site and issues a query. The result is a variable list of links that all need to be clicked and the result saved for each link. Can this be accomplished using the tool?

Basically what you need is a macro that first navigates to your website, runs the query and then loops over the result. The solution is to split the task in two macros. The first macro (macro1) navigates to the site and runs the query. The second macro (macro2) "loops" over the result until all items are processed. Both macros are combined with a small script. Note that you have to manually remove the URL GOTO... in macro2. Thus the iMacros Browser continues macro2 exactly at the position where macro1 stopped. As an example, please see the "combine-macros.vbs" example script.

Q: Does the macro script wait for the page to fully finish loading?

Yes, if a command inside a macro triggers a page (re-)load iMacros automatically waits until the page is completely loaded before it continues with the next command.

On a few very complex web pages with frames the macro may continue before everything is loaded completely. In this case you can tell iMacros manually to wait for another "Page loaded..." signal by adding the WAIT SECONDS=#DOWNLOADCOMPLETE# command to the macro.

One solution that will always work is to use the IMAGESEARCH command to search for an image. iMacros will wait until this image appears (or the timeout value is reached). This approach looks at the website as if it were an image - just as a human would do! Thus this approach works with every web site, regardless of which web technology is used (HTML, Frames, ASP.NET, Java applet, Flash applet,....).

The same issue is also discussed under an different angle in the Web Testing FAQ - How to check if an image is already loaded?.

Q: But I don't want my macro to wait for the page to fully finish loading. How do I tell it not to wait?

In this case, you can simply set a low value for the page timeout (!TIMEOUT_PAGE) and tell iMacros to ignore the timeout error by using SET !ERRORIGNORE YES. For example:

SET !TIMEOUT_PAGE 1
SET !ERRORIGNORE YES
URL GOTO=...
' Reset values to defaults
SET !ERRORIGNORE NO
SET !TIMEOUT_PAGE 60

Q: Why is a certain input box never recorded?

1. If all input boxes except one or two are recorded correctly, the problem could be that these boxes have the same internal name. Typically, this occurs with input boxes that are used for "From" and "To" or "Password" and "Re-enter password" type of inputs. If (and only if) both fields have the same internal name iMacros can not fill the fields correctly during replay. Solution: Manually edit the macro and use POS=2 to reach the second element - the "To" selection in the following example.

URL GOTO=http://www.flybmi.com/bmi/en-gb/index.aspx 
SIZE X=800 Y=600
'Fill FROM field
TAG POS=1 TYPE=SELECT FORM=NAME:IBEMagnet_IBEMagnet__ctl0 ATTR=NAME: CONTENT=$*Dublin* 
'Fill TO field
TAG POS=2 TYPE=SELECT FORM=NAME:IBEMagnet_IBEMagnet__ctl0 ATTR=NAME: CONTENT=$*Frankfurt*
TAG POS=1 TYPE=INPUT:IMAGE FORM=NAME:IBEMagnet_IBEMagnet__ctl0 ATTR=NAME:&&VALUE:

2. If you click on input boxes and no TAG commands at all are created, you might have encountered a Flash- or Java-based input box (inside a Flash- or Java-Applet) instead of a standard HTML input box. In this case, change the ClickMode to WinClick to record the input.

Q: Why is the dialog "XYZ" not handled by iMacros? [XYZ can be any browser dialog, for example the print, download or save dialog]

This issue can have several causes. Here are some questions in order to diagnose this problem:

Dialog is not recognized during Recording

· During recording, an iMacros dialog should appear under the Web Browser dialog. If the iMacros dialog does not appear, please make sure that the option "Use IM Dialog Manager" is checked in the Browser tab of the Options dialog.


· If this option is checked and the dialog is still not recognized, please play the "Demo-Javascript", Demo-Download" or "Demo-SaveAs" macro. Are the dialogs in these macros handled correctly? If not, try reinstalling the software or contact tech support.


· Does the dialog appear before you start recording? This could happen, for example, with a login dialog. To work around this, start recording at a different page, and enter your real web site address. You can always edit the macro after recording and remove the first URL.


· If the dialog is still not recognized: Is the dialog in our list of supported dialogs? If yes, try adding the commands manually to the macro and test if everything works correctly during replay.


2. Dialog is recorded, but not managed during Replay

· If the dialog appears at the end of the macro, it can be that the macro stops shortly before the dialog appears and thus cannot "catch" it. To work around this, add a WAIT SECONDS=3 command after the command that triggers the dialog to make sure iMacros is still active when the dialog appears.

Q: How do I make the IM Browser appear as native IE (Internet Explorer)?

Open the iMacros IE Plug-in or the iMacros Browser and open the Options dialog. In the Options dialog, select the Browser tab and choose "Stealth mode (Identify as native Internet Explorer)". In this mode the iMacros Browser appears to a web server exactly the same as Internet Explorer.

See also FAQ#Q: The web page I am accessing requires Internet Explorer (IE). Is this a problem?

Q: How to authenticate at a proxy server?

Problem: The issue is that we have username /password to get to internet websites which means we are authenticated at the proxy server. iMacro fails to wait asking for username /password and just goes ahead and says since proxy is not authenticated we get the error from proxy server .

Solution: Please make sure that you have the ONLOGIN command before the first URL GOTO= command.

 ONLOGIN USER=user PASSWORD=password
 URL GOTO=http://www.iopus.com

Q: Why does my output file start with three weird characters?

Those extra codes at the beginning of the file are the Byte Order Mark (BOM) and identify the file as being UTF-8 encoded.

Extracting Data

Q: I use the EXTRACT command and get the message "Extraction anchor not found". What is the solution?

The reason for this message is that the extraction anchor, given mainly by the ATTR parameter of the TAG command, could not be found on the currently active web site.

Here are some tips on creating good extraction anchors:

Make sure that you selected the appropriate part for the anchor. It should only contain HTML code that does not change during each session or the changing parts need to be replaced by the wildcard *.

Incorrect: This anchor does not work because it contains a changing value (the dollar amount).

TAG POS=12 TYPE=TD ATTR=TXT:Price<SP>US$<SP>3.50 EXTRACT=TXT

Correct: Replaced changing part with asterisk *

TAG POS=12 TYPE=TD ATTR=TXT:* EXTRACT=TXT

To test your anchor please make sure you click on "test_EXTRACT_tag" after iMacros suggests an anchor element or you have manually created one. Most extraction issues are detected with the test feature.

Experiment with the POS attribute. POS indicates which of a number instances of the same anchor on a web page should be extracted.

If the information you want to extract is inside a framed web site you need to click inside the frame that contains the information you want to extract before opening the Extraction Wizard. This generates the FRAME command and marks the frame as active for the extraction.

Q: I require macros that can extract simple text data from a web page and then insert the extracted information back into either the same web page or another web page. I do not need to store this data in any way. In effect all I need to do is copy and paste the text data.

Extracted data is saved in the !EXTRACT variable. You can assign the extracted value to a variable and use it again at a later point in your macro.

TAG POS=29 TYPE=TD ATTR=TXT:* EXTRACT=TXT
SET !VAR1 {{!EXTRACT}}
...
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:f1 ATTR=NAME:n1 CONTENT={{!VAR1}}

Q: How do I extract information from a table with variable length and/or more than one page?

This task occurs often e.g. when you extract data from a database (search engines, flight schedules, news service). Depending upon your keyword you might get zero, one, two or 500 entries back. To extract this data you need to divide your macro into two separate macros as described here. Your second macro consists of just one extraction tag for one line of the table, e.g.:

TAG POS={{mypos}} TYPE=TD ATTR=TXT:<SP>noWrap>*EUR* EXTRACT=TXT

After replaying the first macro, which navigates to the results, you need to loop through the results by changing the POS parameter, i.e. the mypos variable from 1 to 999. This line would be the content of the "macro_extract" macro in the source code sample below. It extracts the EUR price on each line.

Once you get the #EANF# message back instead of a result you know that you reached the end of the table on the current page.

If necessary, you can now run another macro to click on a "NEXT" link for another page of results. If no "NEXT" link is found then you know that you are done.

Here is a Visual Basic source code snippet, that shows how the different macros are used:

'Read keywords
For i1 = 1 To 999 '<<<<<<<<< LOOP Keywords until all words processed (=> Read_Line returns "ERROR")
    sKeyword = Read_Line(mFileInput, i1) 'Read keyword from a text file
    If sKeyword = "ERROR" Or Len(sKeyword) < 2 Then
        Exit For
    End If
    iRet = iim1.iimDisplay("Search: " + sKeyword)
    iRet = iim1.iimSet("-var_search", sKeyword)
    iRet = iim1.iimPlay("macro_search")
    sData = iim1.iimGetLastExtract()

    iRet = iim1.iimDisplay("Extract: " + sKeyword)
    
    For i2 = 1 To 999 '<<<<<<<<<<<<<LOOP "next" links
    'Loop all NEXT links until error => no more NEXT links to process)
        
     For i3 = 1 To 999 '<<<<<<<<<<<<<<< LOOP table lines and extract them
        'Loop the table rows until an error occurs => no more data on this page
            
           DoEvents
            iRet = iim1.iimSet("-var_mypos", CStr(i3))
'           iRet = iim1.iimDisplay(sKeyword + "P:" + CStr(i2) + "L:" + CStr(i3))
            iRet = iim1.iimPlay("macro_extract")
            sData = iim1.iimGetLastExtract()
            If iRet = 1 and len (sData) > 0 Then
                s1 = iim1.iimGetLastExtract(1)
                s2 = iim1.iimGetLastExtract(2)
                s3 = iim1.iimGetLastExtract(3)    
                'Check if data found, and save it to a file
                i = InStr(s0, "#EANF#")
                If i <= 0 Then             
                 sLine = sKeyword + sSep + CStr(i2) + sSep + CStr(i3) + sSep + s1 + sSep + s2 + sSep + s3
                 Call Write_Line(mFileOutput, sLine)
                Else
                    Exit For 'next page
                End If
            Else
                Exit For 'next page
            End If 'iRet
        Next  'table rows loop
        
        iRet = iim1.iimDisplay(sKeyword + " Page: " + CStr(i2))
        iRet = iim1.iimPlay("macro_next")
        If iRet < 0 Then
            Exit For 'next keyword
        End If
        
    Next 'NEXT links loop

 Next 'Keyword list loop

iRet = iim1.iimDisplay("Extraction completed")


Note: The support functions used in this example are:

Function Read_Line(sFile, iline) As String
 Dim sValue
 Dim i As Integer
 Dim bFound As Boolean
 bFound = False

 Open sFile For Input As #1
 i = 1
 Do While (Not EOF(1) And bFound = False)
     Input #1, sValue
     If i = iline Then bFound = True
     i = i + 1
 Loop
 Close #1

 If bFound = False Then
     sValue = "ERROR"
 End If

 Read_Line = sValue
End Function

Public Sub Write_Line(sFile, sLine)
  Open sFile For Append As #2
  Print #2, sLine
  Close #2
End Sub

How do I loop through multiple pages of results?

Please see this forum thread: Yellow Pages example

Q: How do I extract data from a hidden input field?

Sometimes web forms contain internal data that is passed between pages in hidden input fields. Since you can not see these fields you can not record extracting visually, but you can edit the recorded macro afterwards and manually create a TAG...EXTRACT command for the hidden field. How this is done is described in the extract chapter and at the TAGs and HTML chapter.

Q: How do I extract text from a (Javascript) message box?

You only need to add:

SET !EXTRACTDIALOG YES

to your macro and the text of a message box is extracted. The text is processed the same way as standard extracted text. Examples of Javascript dialogs can be found in the demo macros "Demo-JavascriptDialog" and "Demo-OfflineDialogs".

Q: Sometimes multiple items inside one cell are separated by <br> instead of creating individual cells for the data. How do I extract data separated with <br>?

To work around this problem use the TYPE=HTM extraction. It preserves all HTML tags inside the text so that the extraction result can be separated later. See also the "Demo-Extract" example macro for more details.

Q: How do I extract the page URL?

The URL of the current page is stored in the internal variable {{!URLCURRENT}}. You can use this value inside commands like any other variable, or extract it by assigning the value to the EXTRACT command:

SET !EXTRACTADD {{!URLCURRENT}}

Q: How do I extract the content of an Input Box?

If you already recorded a macro, check if the extraction TAG command contains the "CONTENT" parameter. If if does, delete it.

Creating the extraction command semi-manually (without the Extraction Wizard):

  • When recording, click the input box(which is recorded as a TAG command)
  • After recording:
    • Add "EXTRAXT=TXT" to this TAG
    • Delete the CONTENT parameter ("CONTENT=...") from this TAG

The result then looks like

TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:form1 ATTR=ID:TF2 EXTRACT=TXT