Auto submit a form with Python, Selenium and Chromium

Détails: 15 décembre 2023; 7033

December 15, 2023: warning: the article below is from an old Selenium version. Another article with an example using the last Selenium version (4.16) is available here (go to the bottom)! We do not need the Chromedriver anymore with last Selenium version.

Here a simple Selenium example where we go to fill then submit a form from data previously converted in XML.

I choose XML because this flat format is easy to grab with Python, and it can be created from an Excel file with Access for example. Morever, if the auto-submit crashs during the execution (it is possible even with a good code, depending on the website providing the form), Chromium will stop on the last record and you could remove the begining of the XML before to re-execute the same code (to avoid duplicates).

For those in a hurry (🧐) my code is here on GitHub. But for a full understanding just see below.

First you should download and unzip the Chrome driver according your browser version.

OK, now try to fill just one record hard coded:

from selenium import webdriver
 
chrome_path = 'C:/Path/To/chromedriver.exe'
browser = webdriver.Chrome(chrome_path)
 
browser.get('https://www.example.com/path/to/the/form.html')
 
firstname = browser.find_element_by_id('input_id_firstName')
lastname = browser.find_element_by_name('input_name_lastname')
email = browser.find_elements_by_class_name('input_class_email')
 
firstname.send_keys('Edward')
lastname.send_keys('Wilson')
email.send_keys(Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.')
 
browser.find_element_by_id('submit_id').click()

in lines 6, 7 and 8, I provide 3 examples to get the inputs from CSS id, name or class (find your own CSS tags with F12).

The line 14 submits the form. As the button, use click() to manage checkboxes.

OK, it works. Now let's go to submit this kind of data:

<dataroot xmlns:od="urn:schemas-microsoft-com:officedata" generated="2021-05-04T12:22:38">
<MyXML>
<title>Mrs</title>
<FirstName>Greta</FirstName>
<LastName>Thunberg</LastName>
<Email>Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.</Email>
<JobTitle>Superhero</JobTitle>
<Country>Sweden</Country>
</MyXML>
...

Just check you are able to iterate throught the XML:

import requests
import xml.etree.ElementTree
 
r = requests.get('C:/Path/To/Your/XML/MyFile.xml')
root = xml.etree.ElementTree.fromstring(r.content)
 
for people in root.findall('poi'):
    my_firstname = people.find('FirstName').text
    my_lastname = people.find('LastName').text
    print(my_firstname, my_lastname)

Now just add you Selenium code in the loop, with the parse function from xml.etree.ElementTree:

from selenium import webdriver
import xml.etree.ElementTree as ET
 
chrome_path = 'C:/Path/To/chromedriver.exe'
browser = webdriver.Chrome(chrome_path)
 
tree = ET.parse('C:/Path/To/Your/XML/MyFile.xml')
root = tree.getroot()
 
for people in root.findall('poi'):
    xml_FirstName =people.find('FirstName').text
    xml_LastName = people.find('LastName').text
    xml_Email =people.find('Email').text
 
    browser.get('https://www.example.com/path/to/the/form.html')
 
    firstname = browser.find_element_by_id('input_id_firstName')
    lastname = browser.find_element_by_name('input_name_lastname')
    email = browser.find_elements_by_class_name('input_class_email')
 
    firstname.send_keys(xml_FirstName)
    lastname.send_keys(xml_LastName)
    email.send_keys(xml_Email)
 
    browser.find_element_by_id('submit_id').click()

Cookies management

Sometimes you should manage the cookies, here an example (line 17):

from selenium import webdriver
import xml.etree.ElementTree as ET
 
chrome_path = 'C:/Path/To/chromedriver.exe'
browser = webdriver.Chrome(chrome_path)
 
tree = ET.parse('C:/Path/To/Your/XML/MyFile.xml')
root = tree.getroot()
 
for people in root.findall('poi'):
    xml_FirstName =people.find('FirstName').text
    xml_LastName = people.find('LastName').text
    xml_Email =people.find('Email').text
 
    browser.get('https://www.example.com/path/to/the/form.html')
 
    try:
        browser.find_elements_by_class_name('cookieButton')[0].click()
    except Exception:
        pass
 
    firstname = browser.find_element_by_id('input_id_firstName')
    lastname = browser.find_element_by_name('input_name_lastname')
    email = browser.find_elements_by_class_name('input_class_email')
 
    firstname.send_keys(xml_FirstName)
    lastname.send_keys(xml_LastName)
    email.send_keys(xml_Email)
 
    browser.find_element_by_id('submit_id').click()

Dropdown management

There are different possible contexts. Examples with the article management from a Joomla!4 backend, categories, access level and tags fields.

from selenium.webdriver.support.ui import Select
 
# LISTE CATEGORIE
categorie_list = browser.find_element_by_xpath('.//div[@class="choices__inner"][contains(., "Accueil")]')
categorie_list.click()
# Pour choisir News
browser.find_element_by_id('choices--jform_catid-item-choice-8').click()
 
# LISTE ACCESS
access_list = Select(browser.find_element_by_id('jform_access'))
access_list.select_by_value('8')
 
# LISTE TAGS
tags_list = browser.find_element_by_xpath('.//input[@placeholder="Saisir ou sélectionner des tags"]')
tags_list.click()
# Pour choisir Workshop
browser.find_element_by_id('choices--jform_tags-item-choice-1').click()
# Pour choisir SIG
browser.find_element_by_id('choices--jform_tags-item-choice-6').click()

Catch a input button

browser.find_element_by_xpath(".//input[@value='Login' and @type='submit']").click()

Catch a input button (2)

sub = browser.find_element_by_link_text('Save')

Sleep

import time
time.sleep(3)

Click a div

clic = browser.find_element_by_xpath(".//div[contains(., 'France') and @class='dropdown__item soft-half cursor-pointer tt-suggestion tt-selectable']")
clic.click()

Click a button

clic = browser.find_element_by_xpath(".//button[@class='sc-112xqu1-0 jcLAzz']")
clic.click()

Drop text

from selenium.webdriver.common.keys import Keys
browser.find_element_by_name('live_location[address]').send_keys(Keys.CONTROL + 'a')
browser.find_element_by_name('live_location[address]').send_keys(Keys.DELETE)

Menu