File Operations in Python: XML Format

XML (eXtensible Markup Language) is a text file format used to describe structured data.

It organizes data in the form of tags, has self-descriptive properties, and a hierarchical structure, making it suitable for configuration files, data exchange, document storage, and other scenarios.

Almost all programming languages support parsing and generating XML, and XML is also compatible with various standards (such as XHTML, SVG, RSS, etc.).

1. Characteristics of XML Format

The file extension for XML files is typically .xml.

The main characteristics of XML files include:

(1) Tag Structure

Data is surrounded by <tag> and </tag> to form a tree structure, which can be nested.

(2) Self-descriptive

Each tag can describe the meaning of its content.

(3) Attributes

Tags can contain attributes, for example, <student id=”1″>Alice</student>.

(4) Hierarchical Relationships

Supports parent-child nodes, sibling nodes, and nested nodes to represent complex structures.

(5) Encoding

UTF-8 is recommended; at the beginning of the file, you can declare <?xml version=”1.0″ encoding=”UTF-8″?>.

(6) Comments

Use <!– comment content –>.

Example XML File:

<?xml version="1.0" encoding="UTF-8"?><students>    <student id="1">        <name>Alice</name>        <score>95</score>    </student>    <student id="2">        <name>Bob</name>        <score>88</score>    </student>    <student id="3">        <name>Carol</name>        <score>90</score>    </student></students>

The above example demonstrates the basic structure of XML: the root node <students> contains multiple <student> child nodes, each <student> node contains <name> and <score>.

2. Representing XML Data in Python

(1) Using Strings and Memory File Objects

When we want to manipulate XML in memory without immediately writing to disk, we can use io.StringIO:

from io import StringIO
xml_text = """<?xml version="1.0" encoding="UTF-8"?><students>    <student id="1"><name>Alice</name><score>95</score></student>    <student id="2"><name>Bob</name><score>88</score></student></students>"""
# Create a memory text file object
buf = StringIO(xml_text)
print(buf.read())

StringIO is suitable for testing, temporary storage, or parsing small XML content from network requests.

(2) Using Dictionaries/Lists to Represent XML Trees in Memory

Before processing XML data, it is common to prepare structured data in memory.

Example:

students = [    {"id": 1, "name": "Alice", "score": 95},    {"id": 2, "name": "Bob", "score": 88},    {"id": 3, "name": "Carol", "score": 90},]
print(students)

This structure facilitates subsequent XML generation, with each dictionary corresponding to a <student> node.

3. Using xml.etree.ElementTree to Parse and Generate XML

The Python standard library xml.etree.ElementTree (abbreviated as ET) provides support for reading and writing XML, capable of handling tree structures, node searching, and attribute management.

(1) Generating XML Files

import xml.etree.ElementTree as ET
# Root node
root = ET.Element("students")
# Add child nodes
students = [    {"id": "1", "name": "Alice", "score": "95"},    {"id": "2", "name": "Bob", "score": "88"},    {"id": "3", "name": "Carol", "score": "90"},]
for s in students:    student = ET.SubElement(root, "student", id=s["id"])
    name = ET.SubElement(student, "name")    name.text = s["name"]    score = ET.SubElement(student, "score")    score.text = s["score"]
# Write to file
tree = ET.ElementTree(root)
tree.write("students.xml", encoding="utf-8", xml_declaration=True)
print("students.xml file has been saved.")

Notes:

• ET.Element(“tag”) creates a node.

• ET.SubElement(parent, “tag”) creates a child node and automatically attaches it to the parent node.

• tree.write() can specify encoding and whether to write the XML declaration.

(2) Reading XML Files

import xml.etree.ElementTree as ET
tree = ET.parse("students.xml")
root = tree.getroot()
for student in root.findall("student"):    sid = student.get("id")              # Get attribute    name = student.find("name").text     # Get child node text    score = int(student.find("score").text)    print(sid, name, score)

Output:

1 Alice 95
2 Bob 88
3 Carol 90

Notes:

• .find() finds a single child node, .findall() finds all matching child nodes.

• .get(“attr”) gets the node attribute value.

• Node text is accessed via .text.

(3) Modifying and Adding Nodes

import xml.etree.ElementTree as ET
tree = ET.parse("students.xml")
root = tree.getroot()
# Modify student scores
for student in root.findall("student"):    if student.get("id") == "2":        student.find("score").text = "90"
# Add new student
new_student = ET.SubElement(root, "student", id="4")
ET.SubElement(new_student, "name").text = "David"
ET.SubElement(new_student, "score").text = "70"
tree.write("students_updated.xml", encoding="utf-8", xml_declaration=True)
print("students_updated.xml has been updated.")

(4) Preventing XML Injection (Safe Writing)

When handling data from external sources, avoid directly writing XML tag content. It is recommended to use xml.sax.saxutils.escape to escape special characters:

from xml.sax.saxutils import escape
name = '<Alice & Bob>'
safe_name = escape(name)  # Escapes to <Alice & Bob>

This prevents characters like <, >, and & from breaking the XML structure.

4. Using Pandas to Handle XML Files

Starting from Pandas 1.3, the read_xml and to_xml methods provide a convenient way to convert between XML and DataFrame.

(1) Reading XML Files

Necessary dependencies need to be installed:

pip install pandas lxml

Example:

import pandas as pd
df = pd.read_xml("students.xml")
print(df)

Output:

   id   name  score
0   1  Alice     95
1   2    Bob     88
2   3  Carol     90

Pandas automatically parses node tags and converts attributes or child nodes into columns.

(2) Filtering and Statistical Data

# Filter students with scores >= 90
top_students = df[df["score"] >= 90]
print(top_students)
# Average score
avg_score = df["score"].mean()
print("Average score:", avg_score)

(3) Writing Results Back to XML Files

top_students.to_xml("top_students.xml", index=False)
print("High-scoring students XML has been saved.")

5. Comprehensive Example

The following example demonstrates the complete lifecycle of an XML file in data analysis: data creation → storage → reading → processing → output.

import xml.etree.ElementTree as ET
import pandas as pd
# 1. Construct original data
students = [    {"id": "1", "name": "Alice", "score": "95"},    {"id": "2", "name": "Bob", "score": "88"},    {"id": "3", "name": "Carol", "score": "90"},    {"id": "4", "name": "David", "score": "70"},]
# 2. Write to XML file
root = ET.Element("students")
for s in students:    student = ET.SubElement(root, "student", id=s["id"])
    ET.SubElement(student, "name").text = s["name"]
    ET.SubElement(student, "score").text = s["score"]
tree = ET.ElementTree(root)
tree.write("students.xml", encoding="utf-8", xml_declaration=True)
# 3. Use pandas to read the file
df = pd.read_xml("students.xml")
# 4. Filter students with scores >= 85
top = df[df["score"] >= 85]
# 5. Calculate average score
avg_score = df["score"].mean()
print("High-scoring students:\n", top)
print(f"Average score: {avg_score:.1f}")
# 6. Output new XML file
top.to_xml("top_students.xml", index=False)

Running Result:

High-scoring students:   id   name  score
0   1  Alice     95
1   2    Bob     88
2   3  Carol     90
Average score: 91.0

📘 Summary

XML, as a structured data format, has self-descriptive and hierarchical characteristics, making it suitable for various data scenarios. Using Python to process XML data has two main methods: one is to use the standard library xml.etree.ElementTree for parsing, generating, and modifying XML, including node operations, attribute management, and safe writing; the other is to utilize the Pandas library’s read_xml and to_xml methods for efficient conversion between XML and DataFrame, facilitating data analysis and processing.

“Likes are beautiful, appreciation is encouragement”