Python Programming – File Operations

Basic Introduction to File Operations

The open() function in Python is used to open a file and create a file object, which can then be used to read and write data.

• Control the mode of reading and writing file content: t and b (t and b cannot be used alone, they must be combined with r/w/a)
• Reading and writing are done in bytes
• Applicable to all files
• Cannot specify the character encoding encoding parameter
• Reading and writing are done in str (unicode)
• Text files
• Must specify encoding=’utf-8′ as the encoding format
• t for text (default mode)
• b for binary mode
• Control the mode of file operations
• r for read-only mode
• w for write-only mode
• Append mode
• +: r+, w+, a+

Mode	r	r+	w	w+	a	a+
Read	+	+	+	+
Write	+	+	+	+	+
Create	+	+	+	+
Clear Data	+	+
Pointer at Start	+	+	+	+
Pointer at End	+	+

Summary:

1. In handling pure text files, the t mode saves us from the encoding and decoding steps, while the b mode requires manual encoding and decoding.
2. For non-text files (images, videos, audio, etc.), only b mode can be used.

File Operation Process

1. Open the file
2. Operate on the file (read/write)
3. Close the file

f = open('test.txt')
res = f.read()
print(res)
f.close()

Only the file is specified, and all other parameters are set to their default values, i.e., mode=’rt’.

read() refers to reading all content of the text.

Close the file to reclaim resources.

Context Management with ‘with’

The with open() statement can handle a single file or multiple files simultaneously, and it automatically closes the opened files after the program execution is complete.

with open('test.txt',mode='rt') as f:
    res = f.read()
    print(res)

with open('test.txt',mode='rt') as f1, open('test.ini',mode='rt') as f2:
    res1 = f1.read()
    res2 = f2.read()
    print(res1)
    print(res2)

Specifying Character Encoding

with open('test.txt',mode='rt') as f:
    res = f.read() # t mode will decode the content read by f.read() into unicode
    print(res)

a. test.txt exists on the disk in utf-8 binary format.

b. When read() is executed, it reads the content of test.txt into memory in utf-8 binary format. As mentioned above, the t mode reads and writes in str (unicode). Therefore, read() also decodes the utf-8 binary into unicode.

c. Here, open() does not specify the encoding parameter, so the operating system will use the default encoding.

For Linux and Mac systems, the default encoding is utf-8.

For Windows systems, the default encoding is gbk.

with open('test.txt',mode='rt',encoding='utf-8') as f1:
    res1 = f1.read()
    print(res1)

Execution result:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.

r Mode

r is the default operation mode, read-only mode. An error will occur if the file does not exist. If the file exists, the file pointer jumps to the start position.

with open('test.txt',mode='rt',encoding='utf-8') as f1:
    print('First Read'.center(50,'*'))
    res1 = f1.read()
    print(res1)

    print('Second Read'.center(50, '*'))
    res2 = f1.read()
    print(res2)

Execution result:

***********************First Read***********************”Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”
***********************Second Read***********************

The content read during the second read is empty because after the first read, the file pointer has moved to the end of the file, and the second read starts from the end, which is empty.

Example:

In user.txt, save the account password: laobai|123

Use the program to read the account information from user.txt and compare it with the user input to verify if the account information is correct.

input_username = input('Please enter the account name:').strip()
input_password = input('Please enter the password:').strip()

with open('user.txt',mode='rt',encoding='utf-8') as f:
    res = f.read()
    username,password = res.split('|')

if input_username == username and input_password == password:
    print('Login successful')
else:
    print('Account or password is incorrect!')

Execution result:

Please enter the account name: laobai
Please enter the password: 123
Login successful

Handling multiple accounts stored in user.txt

laobai|123
lynn|321
nicheng|123456

input_username = input('Please enter the account name:').strip()
input_password = input('Please enter the password:').strip()

with open('user.txt',mode='rt',encoding='utf-8') as f:
    for line in f:
        username,password = line.strip().split('|')
        if input_username == username and input_password == password:
            print('Login successful')
            break
    else:
        print('Account or password is incorrect!')

Execution result:

Please enter the account name: nicheng
Please enter the password: 123456
Login successful

w Mode

w is write-only mode. When the file does not exist, an empty file will be created; when the file exists, the file will be cleared. The file pointer is at the start position.

The content of test.txt is as follows:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”

with open('test.txt',mode='wt',encoding='utf-8') as f:
    f.write("laobai")

After executing the above code, check the content of test.txt again.

laobai

Note:

1. When writing continuously without closing the file in w mode, the new content is always appended after the old content.
2. If the file is reopened in w mode, the file content will be cleared.

Example:

The content of test1.txt is as follows:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”

with open('test1.txt',mode='rt',encoding='utf-8') as f:
    res = f.read()
    print(res)

Execution result:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.

To copy the content of test1.txt to test2.txt:

with open('test1.txt',mode='rt',encoding='utf-8') as f1,
    open('test2.txt',mode='wt',encoding='utf-8') as f2:
    res = f1.read()
    f2.write(res)

After execution, the content of test1.txt is copied to test2.txt. Executing again will result in only one copy, consistent with the content of test1.txt.

a Mode

a mode is append-only mode. When the file does not exist, an empty document will be created; when the file exists, the file pointer will jump directly to the end.

The content of test.txt is as follows:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”

with open('test.txt',mode='at',encoding='utf-8') as f:
    f.write("laobai")

After executing the above code, check the content of test.txt again.

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”laobai

Note: Similarities and differences between w mode and a mode

1. Similarities: When writing continuously without closing the opened file, the new content will always follow the previously written content.
2. Differences: Reopening the file in a mode will not clear the original file content; it will move the file pointer directly to the end, and new content will always be at the end of the file.

Example:

In user.txt, save the account password, the file content is as follows:

laobai|123
lynn|321
nicheng|123456

Use the program to input new account passwords and save them in user.txt:

input_name = input("Please enter the registration name:").strip()
input_password = input("Please enter the password:").strip()

with open('user.txt',mode='at',encoding='utf-8') as f:
    f.write('{}|{}
'.format(input_name,input_password))

After executing the program, check the content of user.txt again:

laobai|123
lynn|321
nicheng|123456
laobai1|123

+t Mode

+ cannot be used alone; it must be combined with r, w, or a. Besides having read and write functions, other features remain consistent with r, w, and a.

Note: read() and write() both start from the cursor position.

a. r+

Example 1:

The content of test.txt is as follows:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”

Using r+ mode to read and write content to test.txt:

with open('test.txt',mode='r+',encoding='utf-8') as f:
    print(f.read())
    f.write('nicheng')
    print(f.read())

Execution result:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”nicheng

Analysis of the program execution result:

1. f.read() reads all content from test.txt and moves the file pointer to the end of the file.
2. write(‘nicheng’) writes ‘nicheng’ at the end of the file and moves the file pointer to the end of the file.
3. Executing f.read() again starts reading from the end of the file, resulting in an empty read.

Example 2:

The content of test.txt is as follows:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”nicheng

Using r+ mode to write content to test.txt:

with open('test.txt',mode='r+',encoding='utf-8') as f:
    f.write('nicheng')

Opening the file in r mode places the file pointer at the beginning of the file.

write() starts writing content from the beginning of the file.

After executing the program, the content of test.txt is as follows:

nicheng“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”nicheng

The first few characters of the content in test.txt have been overwritten by the newly written content.

b. w+

Example 1:

The content of test.txt is as follows:

nicheng“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
“Shouldering the mundane world, holding onto the original intention.”nicheng

Using w+ mode to read the content of test.txt:

with open('test.txt',mode='w+',encoding='utf-8') as f:
    print(f.read())

Opening the file in w mode first clears all content, and the file pointer is at the beginning of the file.

After executing the program, no content is read because w mode has cleared all content.

Example 2:

The content of test.txt is blank.

Using w+ mode to write and read content:

with open('test.txt',mode='w+',encoding='utf-8') as f:
    f.write('nicheng')
    f.write('laobai')
    f.write('lynn')
    print("------->",f.read())

Execution result:

------->

After executing the program, the content of test.txt is:

nichenglaobailynn

Analysis of the program execution result:

1. w mode opens the file, clears all content, and moves the file pointer to the beginning of the file.
2. write() writes content from the pointer position, writing three times, and the pointer moves to the end of the file.
3. read() starts reading from the pointer position, resulting in an empty read.

c. a+

Example:

The content of test.txt is as follows:

nichenglaobailynn

Using a+ mode to read and write content:

with open('test.txt',mode='a+',encoding='utf-8') as f:
    print("=======》", f.read())
    f.write('\nnicheng\n')
    f.write('laobai\n')
    f.write('lynn\n')
    print("------->",f.read())

Execution result:

=======》 
-------> 

After executing the program, the content of test.txt is:

nichenglaobailynn
nicheng
laobai
lynn

Analysis of the program execution result:

1. a mode opens the file, and the pointer is at the end of the file.
2. read() starts reading from the pointer position, resulting in an empty read, and the pointer remains at the end.
3. write() starts writing from the pointer position, and after multiple writes, the pointer remains at the end.
4. Executing read() again starts reading from the pointer position, resulting in an empty read, and the pointer remains at the end.

b Mode

Handling images:

with open('test.jpg',mode='rb') as f:
    res = f.read() # Read the content in binary from the disk —> in b mode, no conversion is done
    print(res)  # bytes type —> treated as binary
    print(type(res))

Execution result:

b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x00\x00\x90\x00\x90\x00\x00\xff\xe1\x00tExif\x00\x00MM\x00*\x00\x00\x00\x08\x00\x04\x01\x1a\x00\x05\x00\x00\x00\x01\x00\x00\x00>\x01\x1b\x00\x05\x00\x00\x00\x01\x00\x00\x00F\x01(\x00\x03\x00\x00\x00\x01\x00\x02\x00\x00\x87i\x00\x04\x00\x00\x00\x01\x00\x00\x00N\x00\x00\x00\x00\x00\x00\x00\x90\x00\x00\x00\x01\x00\x00\x00\x90\x00\x00\x00\x01\x00\x02\xa0\x02\x00\x04\x00\x00\x00\x01\x00\x00\x02\x92\xa0\x03\x00\x04\x00\x00\x00\x01\x00\x00\x01

Handling text:

with open('user.txt',mode='wb') as f:
    f.write('你好hello'.encode('gbk'))

Since my machine uses macOS, which defaults to utf-8, the ‘你好’ written here will display as garbled text due to the use of gbk.

Example:

# File copy tool
file1 = input('Source file path:').strip()
file2 = input('Destination file path:').strip()

with open(r'{}'.format(file1),mode='rb') as f1,
    open(r'{}'.format(file2),mode='wb') as f2:
    #res = f1.read()  # This method uses too much memory
    for line in f1:
        f2.write(line)

readline() and readlines()

readline()

Reads line by line, one line at a time.

Example 1:

with open('test1.txt',mode='rt',encoding='utf-8') as f:
    res1 = f.readline()
    res2 = f.readline()
    print(res1)
    print(res2)

Execution result:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.

Example 2:

with open('test1.txt',mode='rt',encoding='utf-8') as f:
    while True:
        line = f.readline()
        if len(line) == 0:
            break
        print(line)

Execution result:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.

readlines()

Reads all lines and returns them as a list, with each line as an element.

Example 1:

with open('test1.txt',mode='rt',encoding='utf-8') as f:
    res = f.readlines()
    print(res)

Execution result:

['“Dharma was born to subdue evil. It was once predicted.”
', 'He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
']

f.read() and f.readlines() both read the content into memory at once, which can lead to memory overflow if the content is too large.

Example 2:

You can write the contents of the list back to a file using writelines():

l = ['“Dharma was born to subdue evil. It was once predicted.”
', 'He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.
']

with open('test.txt',mode='wt',encoding='utf-8') as f:
    f.writelines(l)

After execution, the content of test.txt is:

“Dharma was born to subdue evil. It was once predicted.”
He is the son of a king, born under a beautiful Bodhi tree, taught scriptures and martial arts by a highly respected master from a young age. He practiced diligently every day, helping the unfortunate around him, and firmly believed in his mission.

Moving the File Pointer

The units for moving the pointer are in bytes.

There is one special case: in t mode, read(n) represents the number of characters.

with open('test.txt',mode='rt',encoding='utf-8') as f:
    res = f.read(4)
    print(res)

seek()

f.seek(n, mode):

• n refers to the number of bytes to move.
• mode refers to the reference position of the file pointer.
• 0: reference is the start of the file.
• 1: reference is the current pointer position.
• 2: reference is the end of the file (this mode moves backward).

Only mode 0 can be used in t mode; modes 1 and 2 must be used in b mode.

f.tell() gets the current position of the file pointer.

Example: mode 0

with open('test.txt',mode='rb') as f:
    f.seek(9,0)
    f.seek(3,0)
    print(f.tell())

Execution result:

3

Example: mode 1

with open('test.txt',mode='rb') as f:
    f.seek(9,1)
    f.seek(3,1)
    print(f.tell())

Execution result:

12

Example: mode 2

with open('test.txt',mode='rb') as f:
    f.seek(-9,2)
    print(f.tell())
    f.seek(-3,2)
    print(f.tell())

Execution result:

882
888