Visual Web Ripper And Setting Start Urls

By Author: Tracy Morgan
Total Articles: 20
Comment this article

Visual Web Ripper can handle the start URLs for data extraction projects easily. When extracting data from a website you must set the beginning URL so the web scraper knows where to begin. This is not always easy with basic web extractors because some sites will have login forms. However setting the starting URL with Visual Web Ripper is usually easy and painless. There are a few things to remember though.
The start URL is the first page from which you want to begin extracting data. This may be the site root or it may be further inside the site. As long as there is no session URL involved (requiring login), you can simply start a root template, and use the editor to navigate to the site by putting in the URL you want to start in the built-in URL bar. If a log-in is required, then the template will need to begin at the login page, submit the required values and then navigate to the data extraction start page. All of this can be done from the editor interface without scripting.
Users can then proceed to create a project template based on the start URL and extract as little or as much of the data available as their ...
... needs dictate.
On the other hand, there may be multiple start URLs required for a specific project or site. Then the Visual Web Ripper can be configured to pull static URLs from a CSV file or a database. This is a more advanced modification to the template but far more powerful and, in many cases, an incredible time saver. This capability is just one of the ways in which Visual Web Ripper outperforms other data extraction programs.
Once the template is complete, the start URL defined and everything saved, the project creator can then watch the data extraction process from the editor screen, elect to run the project in the agent and have it automatically shut off after completion, or run the project at a pre-scheduled time. Any necessary script calls can be integrated into the template from the developer’s menu and the project will run just as programmed.
The more knowledge of basic scripting a user has, the more power is unlocked with the Visual Web Ripper. However, even users with only basic knowledge can run flawless projects on prescheduled intervals using the editor and the visual project setup.
For more information about Data Extraction Please visit www.visualwebripper.com