python with selenium

Install selenium in the windows+anaconda3 environment:

 Install selenium:

Use command line commands: conda or pip
+ conda install selenium
+ pip install selenium

Download drivers:

  • Selenium requires the driver to interact with the selected browser, for example, ChromeDriver
  • Download addresses of different browsers.
Browser Driver download address
Chrome Https://sites.google.com/a/chromium.org/chromedriver/downloads
Edge Https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/
Firefox Https://github.com/mozilla/geckodriver/releases
Safari Https://webkit.org/blog/6900/webdriver-support-in-safari-10/

Driver storage path:

Take Chrome as an example. Place chromedriver.exe in the Scripts directory under the anaconda installation path, for example, D:\Anaconda3\Scripts

Finally, check whether the installation is successful

You can open the browser and prove that the installation is successful.

from selenium import webdriver

from selenium.webdriver.common.keys import Keys

driver = webdriver.Chrome()

Troubleshooting:

  • After installing selenium, matplotlib cannot be used; use pip/Conda uninstall matplotlib touninstall and reinstall matplotlib
  • The version of chromedriver.exe should match the version of Chrome. Enter chrome://version/ in your browser to view the browser version. The following is the version correspondence.

 

  • ChromeDriver v2.40 (2018-06-07) ———-Supports Chrome v66-68
  • ChromeDriver v2.39 (2018-05-30)——–Supports Chrome v66-68
  • ChromeDriver v2.38 (2018-04-17) ———-Supports Chrome v65-67
  • ChromeDriver v2.37 (2018-03-16) ———-Supports Chrome v64-66
  • ChromeDriver v2.36 (2018-03-02) ———-Supports Chrome v63-65
  • ChromeDriver v2.35 (2018-01-10) ———-Supports Chrome v62-64

 

simple code analysis:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get("http://www.python.org")
assert "Python" in driver.title
elem = driver.find_element_by_name("q")
elem.clear()
elem.send_keys("pycon")
elem.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source
driver.close()

Code interpretation:

  • Webdriver provides web drivers including Firefox, Chrome, and IE.
  • Keys provides special button functions on the keyboard, such as Return, F1, ALT, etc.
from selenium import webdriver

from selenium.webdriver.common.keys import Keys
  • Instantiate a Chrome driver
driver = webdriver.Chrome()
  • The driver.get() method will use the browser to open the specified URL, and the script will block until the URL is opened.
driver.get("http://www.python.org")

  • Make sure the title of the driver. get page contains the string “Python.”
assert "Python" in driver.title
  • Get the element of the attribute name=q (here is an input box for input)
elem = driver.find_element_by_name("q")
  • Empty the contents of the input
elem.clear()
  • Enter the string “pycon” in input
elem.send_keys("pycon")
  • Press the RETURN button to complete the search function.
elem.send_keys(Keys.RETURN)
  • Determine if the result is “No results found.”
assert "No results found." not in driver.page_source
  • Turn off the driver (browser)
driver.close()

Chrome headless mode and operation browser:

Usually, the crawler doesn’t need to open a browser. You only need to use the browser’s kernel so that you can use Chrome’s headless mode.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--headless')
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get("http://www.google.com")
driver.close()

Operate browser:

from selenium import webdriver
import time

driver = webdriver.Chrome()
driver.get("http://www.google.com")

Full screen:

driver.fullscreen_window()

Maximize window:

driver.maximize_window()

Minimize window:

driver.minimize_window()

 

Get window position
Get window size
Get window position and size at the same time

print(driver.get_window_position())
print(driver.get_window_size())
print(driver.get_window_rect())
{'x': 10, 'y': 10}
{'width': 1050, 'height': 708}
{'height': 708, 'width': 1050, 'x': 10, 'y': 10}

Set window position
Set window size
Set the window position and size at the same time.

driver.set_window_position(30,30)
driver.set_window_size(500,500)
driver.set_window_rect(10,10,1050,708)

Element positioning:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome()
driver.get("http://www.python.org")

Position the element by id and name:

• Get the input box by id or name

elem = driver.find_element_by_id("id-search-field")

elem = driver.find_element_by_name("q")

Position the element by tag name or class name:

elem = driver.find_element_by_tag_name("input")

elem = driver.find_element_by_class_name("search-field")


Position the element through the CSS selector:

elem = driver.find_element_by_css_selector("#id-search-field")

Locating elements through XPath:

elem = driver.find_element_by_xpath("//input[@id='id-search-field']")


Position the element via a text link:

elem = driver.find_element_by_link_text("Docs")
elem.click()
time.sleep(5)
driver.back()

Position the element via a partial text link

elem = driver.find_element_by_partial_link_text("Doc")
elem.click()
time.sleep(5)
driver.back()

operation of the positioned element:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
import time
driver = webdriver.Chrome()
driver.get("http://www.python.org")

Left mouse click:

driver.find_element_by_link_text("Downloads").click()
time.sleep(2)
driver.back()

Right mouse click:

elem = driver.find_element_by_class_name("python-logo")
ActionChains(driver).context_click(elem).perform()

Double click with the mouse:

elem = driver.find_element_by_class_name("python-logo")
ActionChains(driver).double_click(elem).perform()

Mouse zoom:

elem = driver.find_element_by_class_name("python-logo")
target = driver.find_element_by_name("q")
ActionChains(driver).drag_and_drop(elem,target).perform()

.


Keyboard typing:

Enter characters and special buttons (such as carriage return)

elem = driver.find_element_by_name("q")
elem.clear()
elem.send_keys("selenium")
elem.send_keys(Keys.RETURN)
time.sleep(2)
driver.back()

Key combination

elem = driver.find_element_by_link_text("Django")
print(elem.text)
# ctrl+c
elem.send_keys(Keys.CONTROL,"c")
input_ = driver.find_element_by_name("q")
input_.clear()
input_.send_keys(Keys.CONTROL,"v")
input_.send_keys(Keys.RETURN)
time.sleep(2)
driver.back()

Submit form:

elem = driver.find_element_by_name("q")
elem.clear()
elem.send_keys("selenium")
elem.submit()
time.sleep(2)
driver.back()

Get text content:

elem = driver.find_element_by_tag_name("code")
print(elem.text)
# Python 3: Fibonacci series up to n
>>> def fib(n):
>>> a, b = 0, 1
>>> while a < n:
>>> print(a, end=' ')
>>> a, b = b, a+b
>>> print()
>>> fib(1000)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
Get the attributes of the tag
elem = driver.find_element_by_name("q")
print(elem.get_attribute("id"))
id-search-field


Get a set of elements and auto fill username and password:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

Previously when we positioned the element, we used the find_element_ method. If we want to get a set of elements, we can use the find_elements_ method.

chrome = webdriver.Chrome()
chrome.get("https://google.com/")
inputs = chrome.find_elements_by_tag_name("input")
for input_ in inputs:
if input_.get_attribute("type") == "radio" and input_.get_attribute("value")=="2":
input_.click()
time.sleep(5)
chrome.close()

Customize the username and password

chrome = webdriver.Chrome()
chrome.get("https://mail.eaxmple.com/")
inputs = chrome.find_elements_by_tag_name("input")
for input_ in inputs:
if input_.get_attribute("type") == "text":
input_.send_keys("username")
if input_.get_attribute("type") == "password":
input_.send_keys("password")

Leave a Reply

Your email address will not be published. Required fields are marked *