Automating Web Tasks with Python and Selenium

Using Selenium IDE to perform several automated web tasks; these tasks need to be scheduled to run at specific times each day.

1. Environment Configuration

1. Install Node.js

  • 1) Install using the exe file
  • 2) Set up “Environment Variables” in Windows

Set the path for Node.js and npm

  • The installation package comes with npm management tool

3) Set npm mirror

Aliyun NPM mirror: <https://npm.aliyun.com>

Tencent Cloud NPM mirror: <https://mirrors.cloud.tencent.com/npm/>

Huawei Cloud NPM mirror: <https://mirrors.huaweicloud.com/repository/npm>

2. Install Python

  • 1) Install using the exe file
  • 2) Set up environment variables in Windows Set the path for Python
  • The installation package comes with pip management tool
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple # You can set it to a domestic mirror source

3. Install Selenium

pip install selenium

4. Install ChromeDriver

  • 1) Download ChromeDriver using npm
  • 2) Python configuration Copy the ChromeDriver exe installation package from the npm package to the Python folder
  • 3) Browser settings Enter and press enter in the address bar: chrome://flags/#extensions-on-chrome-urls. Change the option from Disabled to Enabled.
  • 4) Set environment variables in Windows Set the path for chromedriver.exe

5. Install selenium-side-runner

This is used to run side files (after converting the side files generated in the browser through Selenium IDE into py files, selenium-side-runner is no longer needed)

npm install -g selenium-side-runner

2. Execution

1. Method to execute py files

python auto.py

2. Method to execute side files

  • Specify the file to be executed and the address of the browser to be used
selenium-side-runner C:\Downloads\auto.side -c "goog:chromeOptions.binary='C:\Program Files\Google\Chrome\Application\chrome.exe'"
  • (Specify the browser name)
"browserName=chrome"
  • Run Chrome browser without GUI
selenium-side-runner -c "browserName=chrome goog:chromeOptions.binary='C:\Program Files\Google\Chrome' chromeOptions.args=[disable-infobars,no-sandbox,headless]" --base-url <http://xxx.xx.xx.xx> test.side

3. Python Scheduling Syntax

1. Use schedule and time dependencies

import schedule
import time

2. Define two tasks

def job1():
    print("Executing scheduled task 1...")
    auto1()
    print("Task 1 executed...")
def job2():
    print("Executing scheduled task 2...")
    auto2()
    print("Task 2 executed...")

3. Execute at scheduled times every day

# Execute at scheduled times every day
schedule.every().day.at("12:00").do(job1)
schedule.every().day.at("14:00").do(job2)
# Or execute the task every 10 minutes
# schedule.every(1).minutes.do(job1)
# End
while True:
    schedule.run_pending()
    time.sleep(1)

Click the above WeChat public account link to directly follow!

Leave a Comment