123ArticleOnline Logo
Welcome to 123ArticleOnline.com!
ALL >> Computer-Programming >> View Article

Straightforward Tips For Efficient Web Crawling Utilizing Selenium Python

By Author: Siyaram ray
Total Articles: 387

Be that as it may, first let me quickly present you with python's selenium module on the off chance that, on the off chance that you are not acquainted with it:

It is really a python authoritative for the API of Selenium Web Drivers. For instance, you will have the capacity to helpfully get to the API of Selenium Web drivers like Firefox, Chrome, and PhantomJS and so on.
Utilizing selenium training in Bangalore this module, you can utilize web driver API to reproduce a wide range of activities that you can perform on a run of the mill Web Browser! i.e. tap on catches of sites, scroll and explore through pages, type something in information boxes, submit frames, use intermediaries, even execute custom Javascript on pages and some more! All these stuff utilizing only python content! Quite cool! Isn’t that so?

Presently, let’s bounce directly to the primary tip:

Tip No 1: Crawl sites without stacking Images

As we are discussing, mechanized contents these contents will run hundreds or thousands time. In this way, consistently (maybe, milliseconds?) check. The majority of the advanced powerful sites have loads of pictures. At the point when a page loads, selenium stacks every one of the components in it including those pictures!

Thus, despite the fact that we don't interface much with those pictures when we are trying site functionalities. Selenium still loads them! The beneficial thing is there are approaches to stack pages without stacking pictures in selenium!

PhantomJS Web Driver Load Web Page without Images:

driver = webdriver.PhantomJS(service_args=['- - stack images=no'])

Chrome Web Driver Load Pages without Images:

chromeOptions = webdriver.ChromeOptions()
prefs = {'profile.managed_default_content_settings.images':2}
chromeOptions.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(chrome_options=chromeOptions)

Tip No 2: Scrape Websites utilizing Disk Cache

Reserving the benefits frequently prompts quicker page loads. In present day internet browser, circle reserving decreases page stacking time amazingly. You can take the upside of this on selenium web driver also!. You should simply set the design before instatement of web driver. This essentially stores every one of the sites resources like css, js in the plate stockpiling for quicker stacking. Accommodating, when you stack various pages of a similar site

Note: Obviously, while computerizing your tests, you can't (and shouldn't) store the benefits that has influences on the information you need to test! For instance, in the event that you are trying a dynamic site where the information is stacked utilizing resources state javascript at that point, circle reserve may even make your tests old!
Tip No 3: Use Javascript for looking over

While communicating with page components exceptionally tapping on catches, if the component you are searching for isn't unmistakable in the view port. Selenium raises an exemption telling that component isn't noticeable. Lets lean toward Javascript for looking over the component into view and, at that point sit tight for a bit utilizing time.sleep() so the parchment impact goes off. Furthermore, at that point trigger the snap… basic!

Tip No 4: Scrolling things in a drop down utilizing Select

Let’s state, you are endeavoring to choose a choice from a select component with colossal number of alternatives state 20+. For this situation, you can't choose the thing in normal way. You should initially find the select component utilizing driver. At that point, discover every one of the alternatives. Channel through every one of them to locate the suitable choice you are searching for. From that point forward, make it noticeable and tap on it! Luckily, selenium has a class called 'Select' which will assist you with doing the above assignment in significantly progressively simpler way!

Tip No 5: Properly shut the driver utilizing quit

It is imperative to legitimately close the driver subsequent to completing the mechanization uniquely, on the off chance that you run your contents occasionally! When you conjure your python computerization content to get things done for you, it utilizes extra assets/forms for the selenium web drivers. After the python content completions execution it doesn't discharges those extra assets on the off chance that you don't instruct it to do as such! One approach to do it is to utilize driver.close().

Be that as it may, driver.close() doesn't generally stop the web driver that is running in foundation. What's more, on the off chance selenium course in Bangalore that you are doing things utilizing numerous tabs of the selenium internet browser then it closes just the present one! Driving others open.

You can utilize driver.quit() rather, as it shuts every one of the tabs of the selenium internet browser . The Selenium Web driver occurrence is additionally gets executed after this! Continuously limit the quantity of solicitations you make to the Web Server. Attempt to diminish it however much as could reasonably be expected.

More About the Author

best institute for selenium training in Bangalore – Infocampus. Infocampus is an excellence training center for selenium courses in Bangalore. At Infocampus, selenium training is focused on practical and real time examples. Contact Infocampus at 8884166608 or visit http://infocampus.co.in/best-selenium-testing-training-center-in-bangalore.html for complete details on selenium training.

Total Views: 214Word Count: 788See All articles From Author

Computer Programming Articles

1. Selenium Testing With Python And Why It Makes Sense
Author: Siyaram ray

2. Though Rudi Says The Rocket League Crew Is Cognizant
Author: lolgaonline

3. Basic Python Training
Author: zenitech

4. For Selenium Testing - Which One Is Best For You Either Java Or Python
Author: Siyaram ray

5. Avail Smart Video Surveillance Software From Reliable Company For Better Output
Author: Devline

6. Benefits Of Selenium - Test Automation Tool
Author: Siyaram ray

7. Designing And Execution Of Selenium Test Automation Framework
Author: Siyaram ray

8. Microsoft Access Queries And How Important They Are In Your Database
Author: Ben Beitler

9. Восхождение На Вулкан Ринджани. Ломбок. Индонезия (rinjani)
Author: Armando Kozak

10. The Future Of Innovation In The Era Of Artificial Intelligence
Author: Charlotte

11. Product Engineering Services From Enlume Inc
Author: Srikanth reddy

12. Www Mcafee Com Activate | Mcafee Com/activate | Is Mcafee Antivirus Not Functioning Properly?
Author: SUPPORT MCAFEE

13. 123.hp Setup | 123.hp Com/setup | 123.hp Setup Officejet
Author: setup desk

14. Best Software Training And Placement Company In Hyderabad
Author: hema sri

15. Professional Private Investigator Milwaukee Services To Unravel The Truth
Author: Milwforensics

Login To Account
Login Email:
Password:
Forgot Password?
New User?
Sign Up Newsletter
Email Address: