1. Web Scraping
Requests: A Python HTTP library used for sending HTTP requests, supporting HTTP/1.1 and HTTP/2, suitable for scraping simple web pages.
BeautifulSoup: An HTML/XML parsing library used to parse web content and extract needed information.
Scrapy: A Python web scraping framework that allows for rapid development of efficient scraping programs, supporting distributed scraping, automatic rate limiting, automatic retries, and more.
Selenium: An automation testing tool that simulates user interaction with a browser, suitable for scraping pages that require login simulation or dynamic loading.
PyQuery: A jQuery-like library used to parse HTML documents, supporting CSS selectors. These libraries can be selected based on specific needs to achieve web downloading, parsing, data extraction, etc. Additionally, they can be combined with third-party libraries like pandas and matplotlib for data analysis and visualization.
Urllib2: An HTTP client library in the Python standard library used for sending HTTP requests, supporting cookies, proxies, etc.
PySpider: A distributed web scraping framework that allows for rapid development of efficient scraping programs, supporting dynamic fetching, data parsing, and data storage.
MechanicalSoup: A Python library that simulates browser operations, supporting automatic form filling and login.
RoboBrowser: A Python library that simulates browser operations, supporting automatic form filling and login, while also supporting JavaScript rendering.
Grab: A Python library used for scraping web content, supporting multithreading, proxies, cookies, and also XPath and CSS selector syntax. While these libraries are less commonly used, they each have their own features and advantages, and can be selected based on specific needs.
2. GUI Development
Tkinter: The standard GUI library included with Python, suitable for creating basic windows and controls, easy to learn.
PyQt (PyQt5, PyQt6): A Python GUI library used for creating complex windows and controls, supporting Qt Designer for visual design.
wxPython: A Python GUI library for creating cross-platform windows and controls, supporting Visual Studio for visual design.
Kivy: A Python GUI library used for creating cross-platform touchscreen applications, supporting multi-touch and animations.
PySide (PySide2, PySide6): A Python GUI library similar to PyQt, supporting Qt API for creating complex windows and controls.
3. Image Processing
Pillow: A Python image processing library used for opening, editing, and saving various types of image files, supporting operations like resizing, cropping, rotating, and applying filters.
OpenCV: An open-source computer vision library used to process images and video streams, supporting face recognition, object tracking, image enhancement, and more.
Scikit-image: A Python image processing library used for image segmentation, feature extraction, and filtering.
Matplotlib: A Python data visualization library used for plotting various types of charts, including histograms, scatter plots, and line charts, while also supporting image display and processing.
Pygame: A Python game development library used for creating 2D games and graphical interface programs, supporting image and audio processing.
4. Video Processing and Editing
OpenCV: An open-source computer vision library used for processing images and video streams, supporting video reading, writing, splitting, merging, transcoding, and more.
MoviePy: A Python video processing library used for video editing, merging, transcoding, and adding audio, while supporting various video formats and codecs.
PyAV: A Python library based on FFmpeg for video editing, audio editing, and transcoding, supporting various video and audio formats.
Pygame: A Python game development library used for creating 2D games and graphical interface programs, supporting video and audio processing.
FFmpeg-python: A Python FFmpeg library used for video transcoding, editing, and splitting, supporting various video formats and codecs.
Scikit-video: A Python video processing library used for video analysis, processing, and feature extraction, supporting various video formats and codecs.
imageio-ffmpeg: A Python library based on FFmpeg for video editing, transcoding, and writing, supporting various video formats.
5. Audio Processing
Pydub: A Python audio processing library used for audio editing, merging, transcoding, volume adjustment, fading in and out, while supporting various audio formats and codecs.
Python-sounddevice: A Python audio library used for audio recording, playback, and processing, supporting various audio formats and codecs.
Librosa: A Python audio processing library used for feature extraction, analysis, and processing, supporting various audio formats and codecs.
Aubio: A Python audio processing library used for feature extraction, processing, and analysis, supporting various audio formats and codecs.
FFmpeg-python: A Python FFmpeg library used for audio transcoding, editing, and splitting, supporting various audio formats and codecs.
6. Data Processing
NumPy: A Python numerical computing library used for array operations, linear algebra, Fourier transforms, random number generation, etc., is one of the foundational libraries for scientific computing and data processing.
Pandas: A Python data processing library used for data reading, cleaning, transformation, merging, and analysis, supporting various data formats and structures.
Matplotlib: A Python data visualization library used for plotting various types of charts, including histograms, scatter plots, and line charts, an important tool for data analysis and presentation.
Seaborn: A Python data visualization library used for plotting various high-level charts, including heatmaps, bar charts, density plots, etc., supporting data aggregation and grouped presentation.
Scikit-learn: A Python machine learning library used for data mining, classification, clustering, regression, etc., supporting various machine learning algorithms and models.
7. Databases
SQLite: A lightweight relational database suitable for small applications or mobile applications.
MySQL: A popular relational database suitable for large applications or web applications.
Oracle: A commercial-grade relational database suitable for large enterprise applications.
PostgreSQL: An open-source relational database suitable for highly scalable applications.
pymongo (MongoDB): A popular document database suitable for unstructured data storage and processing.
8. Web Development
Django: A Python web framework that supports rapid construction of high-performance, scalable web applications, suitable for building various types of web applications such as blogs and e-commerce sites.
Flask: A Python web framework that supports rapid construction of lightweight, flexible web applications, suitable for building small web applications and APIs.
Pyramid: A Python web framework that supports rapid construction of high-performance, scalable web applications, suitable for building various types of web applications such as blogs and e-commerce sites.
Tornado: A Python web framework that supports rapid construction of high-performance, scalable web applications, suitable for building real-time web applications and long-connection applications.
Bottle: A Python web framework that supports rapid construction of small, lightweight web applications, suitable for building APIs and micro web applications.
9. Neural Learning and AI Development
TensorFlow: An open-source deep learning framework from Google, supporting various neural network models, such as convolutional neural networks and recurrent neural networks, used for deep learning tasks in image processing, speech recognition, natural language processing, etc.
Keras: A high-level neural network API that runs on top of TensorFlow, Theano, Microsoft Cognitive Toolkit, etc., supporting rapid construction of deep learning models like convolutional and recurrent neural networks.
PyTorch: An open-source deep learning framework from Facebook, supporting both dynamic and static computation graph modes, used for deep learning tasks in image processing, speech recognition, natural language processing, etc.
CNTK: An open-source deep learning framework from Microsoft, supporting various neural network models, suitable for deep learning tasks in image processing, speech recognition, natural language processing, etc.
Theano: A Python numerical computing library that supports automatic differentiation and GPU acceleration, used for building deep learning models like convolutional and recurrent neural networks.
Scikit-learn: A Python machine learning library supporting various algorithms like classification, regression, clustering, etc., used for data mining and predictive analysis.
NLTK: A Python natural language processing library, supporting various NLP algorithms like tokenization, part-of-speech tagging, named entity recognition, etc., used for text mining, information extraction, machine translation, etc.
Gensim: A Python text processing library supporting various text processing algorithms like word vectors and topic models, used for text classification, topic analysis, etc.
10. Packaging
setuptools: A Python packaging tool used for packaging Python modules and applications, supporting multiple packaging formats including source packages, binary packages, and Windows installers.
distutils: The built-in Python packaging tool used for packaging Python modules and applications, supporting both source and binary packaging.
pyinstaller: A Python packaging tool that packages Python code into executable files, supporting multiple platforms and versions of Python interpreters, automatically handling dependencies.
cx_Freeze: A Python packaging tool that packages Python code into executable files or installers, supporting multiple platforms and versions of Python interpreters, automatically handling dependencies.
py2exe: A Python packaging tool that packages Python code into Windows executable files, supporting Python 2.x versions, automatically handling dependencies.
11. Excel Processing
OpenPyXL: A Python Excel processing library used for reading and writing Excel files, supporting various Excel file formats, including .xlsx, .xlsm, .xltx, .xltm, etc.
XlsxWriter: A Python Excel processing library used for creating and writing Excel files, supporting various Excel file formats, including .xlsx, .xlsm, etc.
Pandas: A Python data processing library used for reading and writing Excel files, supporting various Excel file formats, including .xlsx, .xls, and also supporting data processing and transformation.
xlrd: A Python Excel processing library used for reading Excel files, supporting various Excel file formats, including .xls, .xlsx, etc.
xlwt: A Python Excel processing library used for writing Excel files, supporting various Excel file formats, including .xls, etc.
12. WeChat
itchat: An open-source WeChat personal account interface that allows logging in, sending messages, and automatic replies through Python code.
wxpy: An excellent WeChat bot development library that allows logging in, sending messages, automatic replies, and friend management through Python code.
WeRoBot: A WeChat public account development framework that allows automatic replies, menu settings, and template message sending through Python code.
wechaty: An open-source WeChat bot framework that allows logging in, sending messages, automatic replies, and friend management through Python code.
wxpusher: A WeChat message pushing platform that allows sending messages to personal accounts and group chats through Python code, also supporting message templates and statistics.
13. Mouse and Keyboard Control
pyautogui: A Python automation control library used for simulating mouse and keyboard actions, such as moving, clicking, dragging the mouse, and pressing keyboard keys.
pynput: A Python library for keyboard and mouse control, used for simulating keyboard and mouse actions, such as moving, clicking, dragging the mouse, and pressing keyboard keys.
keyboard: A Python library for keyboard control, used for capturing keyboard events and simulating key events, supporting Windows, Linux, and macOS.
PyUserInput: Used to simulate user input events, such as keyboard keys and mouse movements and clicks. This library provides a simple way to automate the behavior of interactive software, simulating actual user actions to automatically perform specified tasks.
14. Game Controllers
pygame: A Python game development library that supports game controller control, used for handling controller input events, such as buttons, joysticks, triggers, etc.
pyvjoy: A Python virtual game controller control library used to simulate game controller input events, supporting Windows.
15. Peripheral Control
pyserial: A Python serial communication library used for controlling serial devices like Arduino and Raspberry Pi.
RPi.GPIO: A Python library for controlling Raspberry Pi GPIO, used for controlling GPIO peripherals like LEDs, motors, sensors, etc.
Adafruit_Python_GPIO: A Python GPIO control library that supports various GPIO boards like Raspberry Pi and BeagleBone.
pyusb: A Python USB control library used for controlling USB devices like cameras, printers, scanners, etc.
pybluez: A Python Bluetooth control library used for controlling Bluetooth devices like Bluetooth headsets and speakers.
16. Email
IMAPClient: A third-party Python library for connecting and operating IMAP servers, supporting searching, downloading, deleting, moving emails, etc.
pymailq: A third-party Python library for connecting and operating SMTP servers, supporting sending, queuing, and retrying emails.
poplib: A Python standard library for connecting and operating POP3 servers, supporting downloading and deleting emails.
smtplib: A Python standard library for connecting and operating SMTP servers, supporting sending emails.
yagmail: A third-party Python library for connecting and operating SMTP servers, supporting sending emails, easy to use.
17. SMS
twilio: A third-party Python library for sending and receiving SMS, supporting various SMS service providers like Twilio and Plivo.
yunpian-sdk: A third-party Python library for sending and receiving SMS, supporting Yunpian SMS service provider.
qcloudsms_py: A third-party Python library for sending SMS, supporting Tencent Cloud SMS service provider.
aliyun-python-sdk-dysmsapi: A third-party Python library for sending SMS, supporting Alibaba Cloud SMS service provider.