Ansible Firefighting Hotline Series (20): Automated Analysis of Nginx Service Failures

🔍 Ansible Firefighting Hotline | Struggling with Nginx Service Failures? One-Click Automated Diagnosis Turns You into a Web Expert!

Are you still struggling with chaotic troubleshooting of Nginx service failures? Today, we bring you a comprehensive automated analysis solution for Nginx service failures on RHEL8/9 & CentOS8/9, allowing you to say goodbye to the nightmare of manually typing commands!

🎯 Pain Points Addressed

The daily routine of an operations engineer: Nginx service anomaly → manually checking service status → viewing configuration files → checking port listening → analyzing error logs → troubleshooting SELinux → checking the firewall… After a series of actions, several hours may pass, and the problem could still be elusive.

Even more frightening is: manual troubleshooting can easily overlook key information, lack systematic analysis, and fail to quickly locate the root cause. Have you ever thought that if there were an automated Nginx diagnostic solution, all these problems would be solved?

✨ Solution Preview

Today, we share an automated analysis solution for Nginx service failures on RHEL8/9 & CentOS8/9 using Ansible, which includes six core diagnostic modules, standardizing, automating, and intelligentizing your Web service troubleshooting!

🎯 Results Preview

🧾 Sample of the Original Diagnostic Report (results only)

================== RHEL8/9 & CentOS8/9 Nginx Diagnostic Report ==================
Generation Time: 2025-09-06T10:58:15Z
Target Host: 10.66.208.231

【Basic Information】
- OS: RedHat 9.6
- Kernel: 5.14.0-427.13.1.el9_4.x86_64
- SELinux: Permissive

【Software and Service Status】
- nginx package: Installed
- systemd active status: active
- systemd enabled status: enabled

【Configuration Syntax Check】
- nginx -t: rc=0, output: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

【Port Listening and Firewall】
- Expected Ports: 80, 443
- ss -lntp (snippet):
State  Recv-Q Send-Q Local Address:Port  Peer Address:PortProcess
LISTEN 0      4096       127.0.0.1:631        0.0.0.0:*    users:(("cupsd",pid=105929,fd=8))
LISTEN 0      128          0.0.0.0:12345      0.0.0.0:*    users:(("python3",pid=145151,fd=3))
LISTEN 0      128          0.0.0.0:12306      0.0.0.0:*    users:(("python",pid=133028,fd=3))
LISTEN 0      128          0.0.0.0:22         0.0.0.0:*    users:(("sshd",pid=68439,fd=3))
LISTEN 0      511          0.0.0.0:80         0.0.0.0:*    users:(("nginx",pid=146297,fd=6),("nginx",pid=146296,fd=6),("nginx",pid=146295,fd=6),("nginx",pid=146294,fd=6),("nginx",pid=139598,fd=6))
LISTEN 0      4096           [::1]:631           [::]:*    users:(("cupsd",pid=105929,fd=7))
LISTEN 0      4096               *:9090             *:*    users:(("node_exporter",pid=835615,fd=3))
LISTEN 0      128             [::]:22            [::]:*    users:(("sshd",pid=68439,fd=4))
LISTEN 0      511             [::]:80            [::]:*    users:(("nginx",pid=146297,fd=7),("nginx",pid=146296,fd=7),("nginx",pid=146295,fd=7),("nginx",pid=146294,fd=7),("nginx",pid=139598,fd=7))
- firewalld status: running
- firewalld services: cockpit dhcpv6-client http https ssh
- firewalld ports: 53/tcp 80/tcp 443/tcp 5000/tcp 5050/tcp 5432/tcp 6379/tcp 9000/tcp 9001/tcp 9002/tcp 9003/tcp 9090/tcp 9091/tcp 9092/tcp 9093/tcp 12306/tcp 12345/tcp 27199/tcp 50051/tcp 53/udp 80/udp 443/udp 5000/udp 5432/udp 6379/udp 9000/udp 9001/udp 9002/udp 9003/udp 9090/udp 9091/udp 9092/udp 9093/udp 27199/udp 50051/udp

【SELinux Boolean Values】
- httpd_can_network_connect: httpd_can_network_connect --> off
- httpd_can_network_connect_db: httpd_can_network_connect_db --> off

【Configuration and Directory Permissions】
- nginx.conf exists: True
- nginx configuration directory permissions: 0755
- nginx working directory permissions: 0770
- nginx website directory permissions: 0755

【nginx Configuration Information】
- nginx user configuration: user nginx;
- nginx worker processes: worker_processes auto;

【Process Status】
- nginx process list:


【Log Analysis】
- Error log path: /var/log/nginx/error.log
- Error log tail:


- Access log path: /var/log/nginx/access.log
- Access log tail (recent accesses):


【journalctl Recent Logs】
-- No entries --

【Optional Attempt Start Results】
Attempt to start not enabled (attempt_service_start=false)

【Common Fault Clue Tips】
- Syntax errors: a non-zero return value from nginx -t usually indicates configuration spelling/path errors
- Port occupation: if :80/:443 in ss is occupied by a non-nginx process, it needs to be released or change the port
- SELinux: access denied, please check the AVC log and set the corresponding boolean value/add context
- Firewall: firewalld not open http/https services or ports
- Permissions: abnormal permissions of web directory/log directory or SELinux context
- Process anomalies: check if the nginx master process and worker processes are started normally
- Configuration directory: abnormal permissions of /etc/nginx directory may cause configuration loading failure
- Working directory: abnormal permissions of /var/lib/nginx directory may affect nginx operation
- User configuration: incorrect nginx user configuration may cause permission issues
- Worker processes: improper configuration of worker_processes may affect performance
- Website directory: abnormal permissions of /usr/share/nginx/html may cause 403 errors

🤔 Design Philosophy: Why Our Playbook is Best Practice?

A professional automation solution is not just a simple pile of commands. Our design philosophy incorporates the core practices advocated by Red Hat, allowing your automation solution to leap from “just works” to “professional and reliable”!

1

Non-destructive diagnosis, safety first ✨ We adopt a read-only analysis mode that does not modify any system configurations, ensuring safety in production environments. All checks are non-intrusive, allowing you to execute with confidence.

2

Variable-driven, flexible adaptation 💻 We centralize all configurable parameters (such as ports, log paths, report output locations) in <span><span>group_vars/all.yml</span></span>. This means that when you need to adjust the diagnostic scope, you only need to modify these variables without touching any core automation task logic.

3

Idempotency guarantee, worry-free ✅ All our Playbooks strictly adhere to Ansible’s core principle—idempotency. You can confidently execute this Playbook repeatedly, and Ansible will automatically detect the current state and only perform necessary checks.

4

Closed-loop verification, visible results 🎯 The last step of the Playbook generates a complete diagnostic report. This forms a check-analyze-report closed loop. You not only execute automation but can immediately see the diagnostic results, ensuring that all issues are under control!

⭐ Automation Scenario Rating

Rating Dimension Rating Description
Ease of Use ⭐⭐⭐ One-click execution, detailed comments, beginner-friendly
Reusability ⭐⭐⭐⭐ Variable configuration, supports multi-host parallel execution
Stability ⭐⭐⭐⭐⭐ Idempotent design, comprehensive error handling
Scalability ⭐⭐⭐⭐⭐ Modular design, easy to extend functionality
Best Practice Compliance ⭐⭐⭐⭐⭐ Follows Ansible best practices, code standards

🗂️ Project Directory Structure

05_Nginx Service Failure Automated Analysis/
├── inventory                    # Host inventory configuration
├── group_vars/
│   └── all.yml                 # Global variable configuration
├── playbook.yml                # Main playbook file
├── templates/
│   └── nginx_diagnosis_report.j2# Diagnostic report template
├── README.md                   # Project documentation
└── test_playbook.yml           # Test entry file

📄 Core File Content Overview

🎯 Main Playbook File (playbook.yml)

---
- name: RHEL8/9 & CentOS8/9 Nginx Service Failure Automated Analysis
  hosts: web
  become: true
  gather_facts: true

  vars:
    # Mode only analyzes, does not make changes; if you need to attempt to start the service, you can override attempt_service_start=true in group_vars/all.yml or -e
    attempt_service_start: false

  pre_tasks:
    - name: Assert running only on RHEL/CentOS 8/9
      ansible.builtin.assert:
        that:
          - ansible_distribution in ['RedHat', 'CentOS']
          - ansible_distribution_major_version in ['8', '9']
        fail_msg: "This Playbook only supports RHEL 8/9 or CentOS 8/9 series operating systems."
        quiet: true

  tasks:
    - name: Collect service facts
      ansible.builtin.service_facts:

    - name: Check if nginx package is installed
      ansible.builtin.command: "rpm -q {{ nginx_package }}"
      register: rpm_query
      changed_when: false
      failed_when: false

    - name: Check current active status of nginx
      ansible.builtin.command: "systemctl is-active {{ nginx_service }}"
      register: svc_active
      changed_when: false
      failed_when: false

    - name: Check if nginx is enabled to start at boot
      ansible.builtin.command: "systemctl is-enabled {{ nginx_service }}"
      register: svc_enabled
      changed_when: false
      failed_when: false

    - name: Syntax check - nginx -t
      ansible.builtin.command: "/usr/sbin/nginx -t"
      register: nginx_syntax
      changed_when: false
      failed_when: false

    - name: Listening port check - ss
      ansible.builtin.command: "ss -lntp"
      register: ss_listen
      changed_when: false
      failed_when: false

    - name: Generate diagnostic report
      ansible.builtin.template:
        src: templates/nginx_diagnosis_report.j2
        dest: "{{ report_output_path }}"
        owner: root
        group: root
        mode: '0644'

    - name: Display report path
      ansible.builtin.debug:
        msg: "Nginx diagnostic report generated: {{ report_output_path }}"

🔧 Host Inventory Configuration (inventory)

[web]
# Fill in the target RHEL 8/9 or CentOS 8/9 hosts here
# For example:
# web1.example.com

10.66.208.232

⚙️ Global Variable Configuration (group_vars/all.yml)

---
# Global Variables - RHEL8/9 & CentOS8/9 Nginx Failure Analysis

# Target host group name needs to align with inventory
target_group: web

# Services and packages
nginx_service: nginx
nginx_package: nginx

# Ports (adjust as needed)
http_ports:
  - 80
https_ports:
  - 443

# Logs and reports
nginx_error_log: /var/log/nginx/error.log
nginx_access_log: /var/log/nginx/access.log
nginx_work_dir: /var/lib/nginx
collect_journal_lines: 200
report_output_path: /root/nginx_diagnosis_report.txt

# Whether to attempt to start the service (default false, analysis first)
attempt_service_start: false

📊 Diagnostic Report Template (templates/nginx_diagnosis_report.j2)

================== RHEL8/9 & CentOS8/9 Nginx Diagnostic Report ================
Generation Time: {{ ansible_date_time.iso8601 }}
Target Host: {{ inventory_hostname }}

【Basic Information】
- OS: {{ ansible_distribution }} {{ ansible_distribution_version }}
- Kernel: {{ ansible_kernel }}
- SELinux: {{ selinux_mode.stdout | default('unknown') }}

【Software and Service Status】
- nginx package: {{ 'Installed' if rpm_query.rc  0 else 'Not Installed' }}
- systemd active status: {{ svc_active.stdout | default('unknown') }}
- systemd enabled status: {{ svc_enabled.stdout | default('unknown') }}

【Configuration Syntax Check】
- nginx -t: rc={{ nginx_syntax.rc }}, output: {{ (nginx_syntax.stdout + ' ' + nginx_syntax.stderr) | trim }}

【Port Listening and Firewall】
- Expected Ports: {{ (nginx_expected_ports | map('string')) | join(', ') }}
- ss -lntp (snippet):
{{ ss_listen.stdout | default('') }}
- firewalld status: {{ firewalld_state.stdout | default('unknown') }}

【Common Fault Clue Tips】
- Syntax errors: a non-zero return value from nginx -t usually indicates configuration spelling/path errors
- Port occupation: if :80/:443 in ss is occupied by a non-nginx process, it needs to be released or change the port
- SELinux: access denied, please check the AVC log and set the corresponding boolean value/add context
- Firewall: firewalld not open http/https services or ports
- Permissions: abnormal permissions of web directory/log directory or SELinux context

=============================================================

🛠️ Foolproof Deployment Guide

Seeing it a thousand times in theory is not as good as doing it once!

Prerequisites

1One Ansible control node.2The target server is configured with SSH trust, and the user executing Ansible has<span><span>sudo</span></span> permissions.3The control node has Ansible installed.

Project Directory Structure

This is a very simple project, you only need a few files!

05_Nginx Service Failure Automated Analysis/
├── inventory
├── group_vars/
│   └── all.yml
├── playbook.yml
├── templates/
│   └── nginx_diagnosis_report.j2
└── README.md

How to Use?

1

Modify Variables ✏️: Open <span><span>group_vars/all.yml</span></span> file and modify the variable section according to your needs, such as port configuration, log paths, etc.

2

Update Host Inventory 📝: Edit <span><span>inventory</span></span> file and fill in your web server hostnames or IP addresses.

3

Execute Automation ▶️: Run the following command, then you can go make a cup of coffee ☕️!

# Basic diagnosis (only analysis, no modification)
ansible-playbook -i inventory playbook.yml

# Optional: Attempt to start the service to capture errors
ansible-playbook -i inventory playbook.yml -e attempt_service_start=true

🔍 Diagnostic Coverage

✅ Package Check

Nginx package installation statusVersion information query

✅ Service Status Analysis

systemd service active statusBoot auto-start configurationService dependencies

✅ Configuration Syntax Validation

nginx -t syntax checkConfiguration file path validationConfiguration error localization

✅ Network Port Diagnosis

Port listening status checkProcess binding analysisPort conflict detection

✅ Security Policy Check

SELinux working modeKey boolean value statusFirewall rules check

✅ Log Analysis

Error log extractionAccess log analysisSystem log analysisFault clue summary

✅ Process and Permission Check

Process status checkConfiguration directory permissionsWorking directory permissionsWebsite directory permissions

✅ Comprehensive Error Handling

Idempotent designFault tolerance for failed tasksTimeout control mechanismRetry mechanism support

💡 Tips for Use

🎯 Batch Diagnosis

# Add multiple servers in inventory
[web]
server1 ansible_host=192.168.1.100
server2 ansible_host=192.168.1.101
server3 ansible_host=192.168.1.102

# Execute in parallel, doubling efficiency
ansible-playbook playbook.yml -i inventory --forks 10

🔧 Custom Configuration

Edit <span><span>group_vars/all.yml</span></span> file to adjust according to your environment:

Modify port configurationAdjust log collection linesConfigure report output pathSet service start attempt

🐛 Troubleshooting

If you encounter issues, check the generated diagnostic report:

Report location:<span><span>/root/nginx_diagnosis_report.txt</span></span>Contains complete fault clue tipsProvides common problem solutions

🎯 Advanced Usage

Custom Diagnostic Scope

# Only check specific modules
ansible-playbook playbook.yml -i inventory --tags "service_check"

# Skip certain checks
ansible-playbook playbook.yml -i inventory --skip-tags "firewall_check"

Output Format Customization

# Detailed output mode
ansible-playbook playbook.yml -i inventory -v

# Super detailed output mode
ansible-playbook playbook.yml -i inventory -vvv

🎁 Surprise Time! Get the Complete Annotated Version!

Do you find the above Playbook not detailed enough? Want to delve into the logic behind every line of code and the best practices recommended by the official?

Let you not only use it but also be able to extrapolate, becoming the most outstanding Ansible automation expert in your team!

👉 Click the 【Read Original】 below to get the packaged download of the Playbook project with complete annotations and syntax highlighting! 👈

🎁 Summary

This automated analysis solution for Nginx service failures on RHEL8/9 & CentOS8/9 truly achieves:

🔍 Comprehensive Diagnosis: Analyzing everything from software packages to service status, configuration to network🚀 One-Click Execution: Automating all diagnostic steps without manual intervention📊 Professional Report: Generating structured diagnostic reports, making issues clear at a glance🔧 Highly Customizable: Variable configuration to adapt to different web environments📈 Batch Processing: Supporting multi-host parallel diagnosis, doubling efficiency🛡️ Safe and Reliable: Default read-only analysis, not modifying system configurations

What are you waiting for? Download this automated diagnostic solution now and increase your Web service troubleshooting efficiency by 10 times!

🚀 Advanced Application Scenarios

Enterprise-Level Deployment

Multi-Environment Support: Unified diagnosis for development, testing, and production environmentsCompliance Checks: Meeting enterprise security audit requirementsMonitoring Integration: Seamlessly integrating with existing monitoring systems

Fault Prevention

Regular Checks: Setting scheduled tasks to proactively discover potential issuesTrend Analysis: Analyzing service health trends through historical reportsAlert Mechanism: Setting alert thresholds based on diagnostic results

Team Collaboration

Standardized Processes: Unifying team fault troubleshooting standardsKnowledge Accumulation: Solidifying expert experience into automated scriptsNewcomer Training: Rapidly improving the overall technical level of the team

Tags: #Ansible #Automation Operations #Nginx Diagnosis #RHEL8 #CentOS8 #Web Services #Operational Efficiency

Leave a Comment