Web Scraping

Simple Python Code for Office Automation: Copy and Use

by
Simple Python Code for Office Automation: Copy and Use

The following are commonly used code scenarios and core library recommendations for Python in office automation, organized based on multiple practical cases: 1. Excel Data Processing Data Cleaning and Merging import pandas as pd def merge_excel(folder_path): all_data = [] for file in Path(folder_path).glob('*.xlsx'): df = pd.read_excel(file, skiprows=2) df['Source'] = file.name all_data.append(df) return pd.concat(all_data).dropna().to_excel("merged.xlsx") Features: Automatically … Read more

Fearless Against Anti-Scraping? This Open Source HTTP Client Stands Strong!

by
Fearless Against Anti-Scraping? This Open Source HTTP Client Stands Strong!

Hello everyone, I am Octopus Cat. Those who have worked with web scraping know that the HTTP request step is often the “key to success or failure”. If you are just requesting some public APIs, Go’s net/http standard library is completely sufficient. However, once you encounter websites with anti-scraping detection, the situation becomes complicated: Some … Read more

Python: Scrape Any Website in Seconds with Just One Line of Code

by
Python: Scrape Any Website in Seconds with Just One Line of Code

Source: Internet If you are looking for the most powerful Python scraping tool, look no further! This one line of code will help you get started immediately.ScrapeasyScrapeasy is a Python library that makes it easy to scrape web pages and extract data from them. It can be used to scrape data from a single page … Read more

Creating an Executable Software with Python: It’s Really Easy with This Library

by
Creating an Executable Software with Python: It's Really Easy with This Library

Beginners in Python may find developing an executable software quite complex, but it is not. The process of converting <span>.py</span> to <span>.exe</span> files is straightforward. You can even develop a fully functional executable software in a single day using Python, thanks to the dedicated executable packaging library PyInstaller, which can package Python scripts into executable … Read more

Practical Python Web Scraping: Sunshine GaoKao Data Extraction

by
Practical Python Web Scraping: Sunshine GaoKao Data Extraction

Sunshine GaoKao Project Project Requirements Scrape basic information and admission guidelines from various universities (the admission guidelines should be stored in PDF format and entered into the database). Database Table Design id task_url status: 0 (not scraped), 1 (scraping), 2 (scraping completed), 3 (error), 4 (updating), 5 (data updated successfully), 6 (data not updated, remains … Read more

Requests: A Library for Simplifying HTTP Requests for Web Scraping and API Calls

by
Requests: A Library for Simplifying HTTP Requests for Web Scraping and API Calls

Click 【Follow + Collect】 to get the latestpractical code examples 1. Sending a Simple GET Request import requests # Define the URL for the request url = 'https://jsonplaceholder.typicode.com/posts/1' # Send a GET request response = requests.get(url) # Check the response status code if response.status_code == 200: # Print the JSON data from the response print(response.json()) … Read more

Basic Usage of the urllib Module in Python Web Scraping

by
Basic Usage of the urllib Module in Python Web Scraping

Hello everyone, I am Mengmeng, your learning assistant.~ In the last article, we introduced the HTTP protocol, which includes the request and response parts. A request is when the client sends request information to the server, and after receiving the request, the server processes it and returns a response. From the above image and the … Read more

asyhttp: A Remarkable Python Library!

by
asyhttp: A Remarkable Python Library!

Hey friends, today let’s talk about the asyhttp module in Python. This tool is like giving wings to web requests, making them incredibly fast. It is suitable for scenarios that require handling multiple web requests simultaneously, such as web scraping and data fetching. Using it can double your program’s efficiency, it’s simply fantastic. Now, let … Read more

30 Classic Python Web Scraping Projects with Source Code!

by
30 Classic Python Web Scraping Projects with Source Code!

This resource package gathers 30 carefully selected practical Python web scraping projects, ranging from easy to difficult, covering the core knowledge points of scraping technology and various application scenarios. Whether you are a beginner just starting out or a developer looking to deepen your skills, these projects will help you quickly enhance your abilities in … Read more

The Useful HTML Parsing Library Parsel in Python

by
The Useful HTML Parsing Library Parsel in Python

Parsel is a lightweight and efficient web data extraction library in the Python ecosystem, developed based on lxml (a high-performance XML/HTML parser). It supports CSS selectors, XPath expressions, and regular expressions, making it flexible for various web data extraction scenarios (either used independently or in conjunction with the Scrapy framework). Advantages 1. Multi-selector support: Compatible … Read more