Custom Solutions

From iMacros
Jump to: navigation, search

In case you want us at iMacros to develop a solution for you, here's some hints on how to quickly get the project going.


Getting in touch

Getting a free quote is easy: just contact us using the form at http://imacros.net/support/professional-services

However, the more information we get right from the start, and the more detailed the description is, the quicker we can work on estimating complexity (and thus: costs) of the project.

Project description

What, then, is relevant information?

We will illustrate the necessary parts with a sample project: fetching data from a real estate property site.

URLs to access

As all macros work on web pages, we need to know the URLs of all relevant sites.

If these are accessible with a login only, please provide a (test) login.

Steps to be automated

Keep in mind that we are not as familiar with the site you want to automate, as you are. That's why we need detailed information about

  • what links to follow
  • to enter what text into what fields
  • what checkboxes to check, what radiobuttons to select
  • what, exactly, to extract, if extraction is part of the macro (e.g. "extract last name, city and ZIP", and not "get address")
  • ...

We recommend using a tool like Snagit to provide a series of annotated screenshots for visually showing each step in addition to providing a written description.

Input-/Output samples

If the project relies on e.g. web queries or on forms to fill, we need a sample of input files that

  • tell us about the input layout (i.e. how many columns, what kind of data in what column)
  • lets us test the site using this input data. Note that we need valid input data for testing the site!

With respect to the output, we need a sample, too, as it

  • gives us more information about what exactly to extract
  • tells us how much "formatting" the "raw" extracted data will need

The default format for input/output is CSV files that Excel / OpenOffice.Org et al. can import/export. However, almost every other format is possible, too. Just tell us what you need:

  • XML
  • SQL datbases
  • custom text files
  • ...

What programming/scripting language to use

In most cases of automation, VBS scripts are sufficient.

But (e.g. in case you need a "real" user interface) other scripting/programming languages are possible, too, like:

  • C#
  • Java
  • ...

In case you want a user interface, please specify what elements of input/output (e.g. free text input, dropdown boxes, information about progress) the interface should show.

Error handling

When automating websites, several kinds of errors may occur:

  • Disconnection
  • Server error
  • Invalid actions
  • "Empty" query results

Please specify how to handle them, e.g.

  • Restart script on disconnection
  • Check whether actions were performed correctly
  • What to do with empty result: ignore? Save as "empty"? ...

Sample Project Description

Here's how the necessary information would look like in a sample project (which is described with full source code here: VBS_looping). The aim of this project is to automate the extraction of real-estate data from a certain website.

Note that iOpus/Ipswitch Inc. is not affiliated with any of the companies that show up in the following sample project description.

URLs to access

The solution would start at http://realestate.nytimes.com/

Initial site.png

Steps to be automated

  • On start page: select "New York" and "Manhattan", then click "Go"
    Query1.png
  • on next page: select "All Manhattan" and click "Search"
    Query2.png
  • on results page: select "List View" to get the list of data that is to be extracted
    Select list view.png
  • On this list, we need to extract the following data from each of the properties listed:
    • Street
    • District (given below street)
    • Kind of property (same line as district)
    • Price
    What to extract.png
  • And we need to extract all results on the first three pages of result (accessible via the "Next" link")
    Loop through pages.png

Input-/Output samples

There is no input (it is always New York/Manhattan that is worked on).

Extraction file format is CSV and here's a sample of the result file.

"795 FIFTH AVENUE", "Upper East Side", "Co-op", "$70,000,000"
"EAST 68TH STREET", "Upper East Side", "House", "$64,000,000"


What programming/scripting language to use

The script is to be run nightly without user interaction. Thus, scheduling a VBS script is enough for this solution.

Error handling

The following features are needed:

  1. Report success/failure to log file
  2. If any of the steps above fail, indicate that the script ended with error.
  3. On error, raise pop-up window