A Beginner’s Guide to Efficiently Using Makefile in Python Projects

Have you ever had to type a long string of commands just to run a test? Or do you find yourself manually executing several steps every time you start a project? If so, Makefile might be the savior you need.

To be honest, I started using Makefile in Python projects because I had used it in other programming languages before. At first, I was a bit hesitant—wondering how effective it would be in a Python project. Doesn’t Python have setup.py and poetry? But a few weeks later, I was completely won over by its practicality.

Why Use Makefile in Python Projects?

Although Makefile is a bit “old-fashioned”, its value in Python projects is surprisingly high:

• • Command Simplification: Turn those long and complex command-line operations into simple<span>make xxx</span>
• • Process Standardization: Ensure everyone on the team works in the same way, reducing the awkwardness of “it works on my machine”.
• • Automated Integration: Chain together operations like testing, deployment, and cleanup, completing them with a single command.

Some might say, “Doesn’t Python have its own tools?” That’s true, but the simplicity and flexibility of Makefile often allow you to achieve more with less effort. It doesn’t require you to learn the rules of a specific framework; you just need to write down the commands you want to execute.

Getting Started: Creating Your First Makefile

To start using Makefile, simply create a file named<span>Makefile</span> in the root directory of your project (note: no extension). Here’s a basic example:

# Dependency Installation
install:
   pip install -r requirements.txt

# Test Execution
test:
   pytest tests/

# Code Formatting
format:
   black .

# Environment Cleanup
clean:
   find . -type f -name "*.pyc" -delete
   find . -type d -name "__pycache__" -delete

# Service Start
run:
   python app.py

.PHONY: install test format clean run

This simple Makefile already covers most operations in daily development. It’s also very easy to use:

make install  # Install dependencies
make test     # Run tests
make format   # Format code

What does the last line<span>.PHONY</span> mean? It tells Make that these targets are not filenames, but task names. This way, even if you happen to have a file named “test” in your directory, the<span>make test</span> command will still work correctly.

Enhancing Your Makefile: Some Practical Tips

Using Variables for More Flexible Configuration

Imagine if your project name changes, or if you want to change the virtual environment directory. Would you have to modify it everywhere in the Makefile? Of course not; variables can solve this problem:

VENV_DIR = venv
PYTHON = $(VENV_DIR)/bin/python
PIP = $(PYTHON) -m pip
PROJECT_NAME = myproject
PYTEST_FLAGS = -v --cov=$(PROJECT_NAME)

venv-setup:
 python -m venv $(VENV_DIR)
$(PIP) install --upgrade pip

install:
$(PIP) install -r requirements.txt

test:
$(PYTHON) -m pytest $(PYTEST_FLAGS)

This way, when you need to change the project name or virtual environment path, you only need to change it in one place.

I had a colleague who spent half a day modifying various references in the Makefile because the project was renamed without using variables; I really don’t want to experience that again.

Combining Tasks: Completing Multiple Operations with One Command

Single tasks are useful, but combining them can truly enhance efficiency:

# Complete Development Process
all: install test run

# Deployment Process
deploy: clean install test
 @echo "Starting deployment..."
 rsync -avz --exclude 'venv' --exclude '.git' ./ user@server:/path/to/deploy/
 ssh user@server "cd /path/to/deploy && make restart-service"

Here, the<span>all</span> and<span>deploy</span> targets depend on other targets, ensuring that all necessary steps are executed in the correct order. The<span>@echo</span> before it indicates that the command itself will not be displayed, only the execution result.

Parameterized Tasks: Increasing Flexibility

Sometimes, you may want to decide the specific behavior of a task at runtime:

# Specify test file (usage example: make test-file FILE=tests/test_api.py)
test-file:
 pytest $(FILE) $(PYTEST_FLAGS)

# Create new application component (usage example: make new-component NAME=user_auth)
new-component:
 mkdir -p src/$(PROJECT_NAME)/$(NAME)
 touch src/$(PROJECT_NAME)/$(NAME)/__init__.py
 touch src/$(PROJECT_NAME)/$(NAME)/models.py
 @echo "Created new component: $(NAME)"

This method allows you to customize task behavior via command-line parameters without modifying the Makefile.

Engineering Practices: A Complete Example

Here’s a more complete engineering Makefile example that covers various aspects of modern Python projects:

# Environment Initialization
init:
 python -m venv venv
 . venv/bin/activate && pip install -r requirements.txt
 @echo "Environment initialization complete, please run: source venv/bin/activate"

# Development Dependency Installation
install-dev:
 pip install -r requirements-dev.txt
 pre-commit install

# Code Quality Check
lint:
 flake8 .
 mypy .
 black --check .
 isort --check-only --profile black .

# Test Report Generation
test-report:
 pytest --cov=src --cov-report=html tests/
 @echo "Test report generated at htmlcov/index.html"

# Security Audit
security:
 bandit -r src/
 safety check
 pip-audit

# Build Distribution Package
build:
 rm -rf dist/
 python -m build
 twine check dist/*

# Publish to PyPI
publish: build
 twine upload dist/*

# Generate Documentation
docs:
 sphinx-build -b html docs/source/ docs/build/html
 @echo "Documentation generated at docs/build/html/index.html"

.PHONY: init install-dev lint test-report security build publish docs

Considerations When Using Makefile

1. 1. Tab Indentation is Required: Makefile strictly requires the use of tabs for indentation; using spaces will lead to strange errors. This is the opposite of Python, so it may take some getting used to at first.
2. 2. Handling Environment Activation: Automatically handle virtual environment activation in tasks:

   task:
    ( 
      source venv/bin/activate; 
      python command.py; 
    )

3. 3. Windows Compatibility: Windows users need to install Make tools or use WSL. To improve compatibility, you can write:

   # Cross-platform file deletion
   ifeq ($(OS),Windows_NT)
    RM = del /Q
   else
    RM = rm -f
   endif
   
   clean:
    $(RM) *.pyc

4. 4. Adding Help Information: Provide guidance for newcomers:

   .PHONY: help
   help:
    @echo "Available commands:"
    @echo "  make install    - Install project dependencies"
    @echo "  make test       - Run test suite"
    @echo "  make format     - Format code"
    @echo "  make clean      - Clean temporary files"

Practical Application Scenarios

Makefile has many practical application scenarios in Python projects:

• • Continuous Integration: Call<span>make test</span> and<span>make lint</span> in the CI/CD pipeline to ensure that every commit meets project standards.
• • New Member Onboarding: New members can quickly set up the development environment by running<span>make init</span>, without having to read lengthy setup documentation.
• • Code Review: Use<span>make lint</span> to standardize code style checks, reducing style discussions during code reviews.
• • Secure Deployment: Execute<span>make security</span> to check for dependency security issues before going live, preventing unsafe code from being deployed to production.
• • Documentation Maintenance: Use<span>make docs</span> to automatically generate the latest documentation, keeping it in sync with the code.

Integration with Other Tools

Makefile can seamlessly integrate with other Python tools:

# Integrate with Poetry
install-poetry:
 poetry install

# Integrate with Docker
docker-build:
 docker build -t $(PROJECT_NAME):latest .

docker-run:
 docker run -p 8000:8000 $(PROJECT_NAME):latest

# Integrate with pre-commit
setup-hooks:
 pre-commit install
 pre-commit install --hook-type commit-msg

A Real Example: My Experience

In a data analysis project I participated in, team members often needed to run various data processing scripts, each with different parameters and dependencies. Initially, we had a lengthy document spanning over 10 pages explaining how to run these scripts.

After introducing Makefile, the situation changed completely. We created tasks like this:

process-daily:
 $(PYTHON) scripts/process_data.py --mode=daily --date=$(shell date --iso)

process-weekly:
 $(PYTHON) scripts/process_data.py --mode=weekly

generate-report: process-daily
 $(PYTHON) scripts/generate_report.py --output=reports/$(shell date +%Y%m%d).pdf
 @echo "Report generated"

New members no longer needed to read lengthy documentation; they could simply run<span>make generate-report</span> to complete all steps. The error rate significantly decreased, and efficiency improved dramatically.

Conclusion

Although Makefile is an “old relic”, its application value in Python projects should not be underestimated. It doesn’t require you to learn complex frameworks; you just need to write down the commands you want to execute, greatly improving development efficiency.

Of course, Makefile is not a panacea. For very complex build logic, you may need to consider specialized build tools; for simple personal projects, Makefile may seem a bit “heavy”. However, in small to medium-sized team projects, a well-designed Makefile can often become a significant driver of team efficiency.

If your project hasn’t yet adopted Makefile, consider starting with simple tasks. You might be surprised by the convenience it brings. Just like I did, from “Why use this old tool” to “Why didn’t I discover this treasure earlier”—this transformation often only takes the time of a<span>make</span> command.