SeleniumHere a simple Selenium example where we go to fill then submit a form from data previously converted in XML.

I choose XML because this flat format is easy to grab with Python, and it can be created from an Excel file with Access for example. Morever, if the auto-submit crashs during the execution (it is possible even with a good code, depending on the website providing the form), Chromium will stop on the last record and you could remove the begining of the XML before to re-execute the same code (to avoid duplicates).

For those in a hurry (🧐) my code is here on GitHub. But for a full understanding just see below.

 

First you should download and unzip the Chrome driver according your browser version.

OK, now try to fill just one record hard coded:

  1. from selenium import webdriver
  2.  
  3. chrome_path = 'C:/Path/To/chromedriver.exe'
  4. browser = webdriver.Chrome(chrome_path)
  5.  
  6. browser.get('https://www.example.com/path/to/the/form.html')
  7.  
  8. firstname = browser.find_element_by_id('input_id_firstName')
  9. lastname = browser.find_element_by_name('input_name_lastname')
  10. email = browser.find_elements_by_class_name('input_class_email')
  11.  
  12. firstname.send_keys('Edward')
  13. lastname.send_keys('Wilson')
  14. email.send_keys(Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.')
  15.  
  16. browser.find_element_by_id('submit_id').click()

in lines 6, 7 and 8, I provide 3 examples to get the inputs from CSS id, name or class (find your own CSS tags with F12).

The line 14 submits the form. As the button, use click() to manage checkboxes.

OK, it works. Now let's go to submit this kind of data:

<dataroot xmlns:od="urn:schemas-microsoft-com:officedata" generated="2021-05-04T12:22:38">
<MyXML>
<title>Mrs</title>
<FirstName>Greta</FirstName>
<LastName>Thunberg</LastName>
<Email>Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.</Email>
<JobTitle>Superhero</JobTitle>
<Country>Sweden</Country>
</MyXML>
...

Just check you are able to iterate throught the XML:

import requests
import xml.etree.ElementTree
 
r = requests.get('C:/Path/To/Your/XML/MyFile.xml')
root = xml.etree.ElementTree.fromstring(r.content)
 
for people in root.findall('poi'):
    my_firstname = people.find('FirstName').text
    my_lastname = people.find('LastName').text
    print(my_firstname, my_lastname)

Now just add you Selenium code in the loop, with the parse function from xml.etree.ElementTree:

from selenium import webdriver
import xml.etree.ElementTree as ET
 
chrome_path = 'C:/Path/To/chromedriver.exe'
browser = webdriver.Chrome(chrome_path)
 
tree = ET.parse('C:/Path/To/Your/XML/MyFile.xml')
root = tree.getroot()
 
for people in root.findall('poi'):
    xml_FirstName =people.find('FirstName').text
    xml_LastName = people.find('LastName').text
    xml_Email =people.find('Email').text
 
    browser.get('https://www.example.com/path/to/the/form.html')
 
    firstname = browser.find_element_by_id('input_id_firstName')
    lastname = browser.find_element_by_name('input_name_lastname')
    email = browser.find_elements_by_class_name('input_class_email')
 
    firstname.send_keys(xml_FirstName)
    lastname.send_keys(xml_LastName)
    email.send_keys(xml_Email)
 
    browser.find_element_by_id('submit_id').click()

Cookies management

Sometimes you should manage the cookies, here an example (line 17):

  1. from selenium import webdriver
  2. import xml.etree.ElementTree as ET
  3.  
  4. chrome_path = 'C:/Path/To/chromedriver.exe'
  5. browser = webdriver.Chrome(chrome_path)
  6.  
  7. tree = ET.parse('C:/Path/To/Your/XML/MyFile.xml')
  8. root = tree.getroot()
  9.  
  10. for people in root.findall('poi'):
  11. xml_FirstName =people.find('FirstName').text
  12. xml_LastName = people.find('LastName').text
  13. xml_Email =people.find('Email').text
  14.  
  15. browser.get('https://www.example.com/path/to/the/form.html')
  16.  
  17. try:
  18. browser.find_elements_by_class_name('cookieButton')[0].click()
  19. except Exception:
  20. pass
  21.  
  22. firstname = browser.find_element_by_id('input_id_firstName')
  23. lastname = browser.find_element_by_name('input_name_lastname')
  24. email = browser.find_elements_by_class_name('input_class_email')
  25.  
  26. firstname.send_keys(xml_FirstName)
  27. lastname.send_keys(xml_LastName)
  28. email.send_keys(xml_Email)
  29.  
  30. browser.find_element_by_id('submit_id').click()