Reading and Writing Text Files in Python: A Complete Guide

Introduction

Think about the last time you used a computer program—maybe it saved your notes, kept a log of your activity, or opened a document you had written earlier. Behind the scenes, all of this happens through file handling. In fact, without reading and writing files, most programs would feel incomplete.

Text files are the simplest and most common type of files you’ll encounter. They’re human-readable, easy to share, and perfect for storing anything from configuration details to large datasets. For example, a Python script could save the results of a survey to a .txt file, and later another script could read that same file to generate a report.

The best part? Python makes working with text files incredibly straightforward. With just a few lines of code, you can open a file, read its contents, write new data, or even append information without worrying too much about the underlying complexity. Plus, Python takes care of tricky details like different operating system line endings and character encodings, which means your code is portable and reliable.

In this post, we’ll dive into everything you need to know about reading and writing text files in Python. By the end, you’ll not only understand the basics but also pick up some best practices that will save you from common pitfalls.

So, let’s get started and unlock one of the most powerful ways Python lets you interact with the world outside your code!

The open() Function Basics

Before you can do anything with a text file—whether that’s reading its contents or writing new data—you first need to open it. In Python, this is done using the built-in open() function. Think of it as the gateway between your program and the outside world of files.

Here’s the basic syntax:

file_object = open("filename", "mode", encoding="utf-8")

• "filename" → The name (or path) of the file you want to work with.

• "mode" → How you want to interact with the file (read, write, append, etc.).

• encoding → (Optional, but recommended) Defines how characters are represented. UTF-8 is the safest and most common choice.

File Modes in Python

The mode argument tells Python what you intend to do with the file. Here are the most commonly used ones:

• "r" (Read) – Opens the file for reading. This is the default mode. If the file doesn’t exist, you’ll get a FileNotFoundError.

• "w" (Write) – Opens the file for writing. If the file already exists, its contents will be erased. If it doesn’t exist, Python will create it.

• "a" (Append) – Opens the file and lets you add new content at the end. The existing data stays safe.

• "x" (Exclusive creation) – Creates a new file. If the file already exists, Python raises an error instead of overwriting it.

You can also combine modes:

• "r+" – Open for both reading and writing (file must exist).

• "w+" – Open for reading and writing, but first truncate the file (erase old content).

• "a+" – Open for reading and appending.

Text vs Binary Mode

By default, files are opened in text mode (t), which means you’re working with strings (str) in Python. But you can also open a file in binary mode (b) to work with raw bytes—useful for images, videos, or any non-text data.

Examples:

# Text mode (default)
open("notes.txt", "r")

# Binary mode
open("image.png", "rb")

Why Encoding Matters

Text isn’t just text—behind the scenes, it’s stored as bytes. Encoding defines how those bytes are translated into human-readable characters.

• UTF-8 is the most common and widely supported encoding.

• If you don’t specify encoding, Python uses your system’s default, which can cause issues when sharing files across platforms.

Example with encoding:

f = open("data.txt", "r", encoding="utf-8")
content = f.read()
f.close()

This ensures your program can handle special characters (like emojis 😊 or accented letters é) without throwing weird errors.

✅ Quick Tip: Always specify encoding="utf-8" when working with text files, especially if you plan to share your code with others or run it on different systems.

Using Context Managers (with Statement)

When working with files in Python, one of the most common mistakes beginners make is forgetting to close the file after finishing the read or write operations. This can lead to memory leaks, file corruption, or even locked files that other programs cannot access. To avoid these issues, Python provides a powerful feature known as context managers, which are used with the with statement.

Why Use Context Managers?

A context manager ensures that resources like files are automatically managed. When you open a file using with open(...) as file:, Python takes care of:

• Opening the file properly.

• Ensuring the file is closed once the block of code is executed, even if an error occurs inside the block.

This means you don’t have to explicitly call file.close(), making your code safer and cleaner.

Basic Syntax

Here’s the standard way of using a context manager with files:

# Reading a file using context manager
with open("example.txt", "r") as file:
    content = file.read()
    print(content)
# No need to call file.close(), it's done automatically

How It Works

• The open("example.txt", "r") part opens the file in read mode.

• The as file part assigns the file object to the variable file.

• Once the block inside with is completed, Python automatically closes the file, even if exceptions are raised.

Writing with Context Managers

You can also use the same approach for writing:

# Writing to a file using context manager
with open("output.txt", "w") as file:
    file.write("This is a new line of text.")
# File automatically closed here

Multiple Context Managers

Python also allows you to work with multiple files in a single with statement:

with open("input.txt", "r") as infile, open("output.txt", "w") as outfile:
    for line in infile:
        outfile.write(line.upper())

In this example:

• input.txt is opened for reading.

• output.txt is opened for writing.

• Each line from the input file is converted to uppercase and written to the output file.

• Both files are automatically closed after the block finishes.

Benefits of Using with

• Cleaner Code: No need for file.close().

• Error Safety: File is closed even if an exception occurs.

• Readability: Makes file-handling code more structured and easy to understand.

Using context managers is considered the best practice for file handling in Python. Whether you are reading, writing, or working with multiple files, the with statement is the most reliable approach.

Reading from Text Files

Once you’ve opened a file in read mode, the next step is to actually pull out its contents. Python gives you several ways to read text files, and which one you choose often depends on the size of the file and what you plan to do with the data.

1. Reading the Entire File at Once

If you know the file is small enough to fit in memory, you can read the whole thing in one go using the .read() method:

with open("notes.txt", "r", encoding="utf-8") as file:
    content = file.read()
    print(content)

• .read() returns the file’s contents as a single string.

• You can also pass a number (e.g., .read(50)) to read only that many characters at a time.

• When the end of the file (EOF) is reached, .read() simply returns an empty string.

⚠️ Pitfall: Avoid this method with very large files—it could eat up your system’s memory.

2. Reading Line by Line with .readline()

If you want more control, .readline() lets you read one line at a time:

with open("notes.txt", "r", encoding="utf-8") as file:
    line1 = file.readline()
    line2 = file.readline()
    print("First line:", line1)
    print("Second line:", line2)

Each call fetches the next line in the file. When there are no more lines, it returns an empty string.

This approach is handy if you’re parsing files line by line, like logs or configuration files.

3. Reading All Lines into a List with .readlines()

If you prefer working with a list, .readlines() does just that:

with open("notes.txt", "r", encoding="utf-8") as file:
    lines = file.readlines()
    print(lines)

• Each line of the file becomes one element in the list.

• The newline characters (\n) are preserved, so you may want to strip them before processing.

Example:

cleaned = [line.strip() for line in lines]
print(cleaned)

4. Iterating Directly Over the File Object

The most Pythonic way to read a file is to loop over it directly:

with open("notes.txt", "r", encoding="utf-8") as file:
    for line in file:
        print(line.strip())

Why this is awesome:

• It reads the file line by line, which is memory-efficient even for large files.

• The file object acts like an iterator, so you don’t need .readline() or .readlines().

This method is often the best choice when dealing with huge files like server logs or CSV datasets.

5. Handling End of File (EOF)

No matter which method you use, Python gracefully handles the end of the file:

• .read() → returns an empty string.

• .readline() → returns an empty string.

• Iterating → simply stops the loop.

This makes it easy to detect when there’s nothing left to read.

6. Stripping Newlines and Extra Spaces

When reading lines, you’ll often see newline characters (\n) sticking around. Use .strip() or .rstrip("\n") to clean them up:

with open("notes.txt", "r", encoding="utf-8") as file:
    for line in file:
        print(line.strip())

✅ Quick Tip: If you only need specific parts of a file (like the first 10 lines), don’t load everything. Iterate and break early—it saves memory and speeds things up.

Writing to Text Files

Reading is only half the story—most programs also need to save data somewhere. Whether you’re logging errors, storing user input, or exporting results, Python makes writing to text files straightforward.

Just like reading, you use the open() function, but this time with a mode that allows writing.

1. Writing with .write()

The simplest way to write is with the .write() method.

with open("output.txt", "w", encoding="utf-8") as file:
    file.write("Hello, Python!\n")
    file.write("This is the second line.\n")

A few things to note:

• .write() only accepts strings, not numbers or other types. You’ll need to convert (str()) if necessary.

• It does not add a newline automatically, so if you want each line on a new line, you must include \n yourself.

2. Writing Multiple Lines with .writelines()

If you already have a list of strings, .writelines() is more convenient:

lines = ["First line\n", "Second line\n", "Third line\n"]

with open("output.txt", "w", encoding="utf-8") as file:
    file.writelines(lines)

⚠️ Watch out: .writelines() does not add line breaks for you—your strings must already include \n.

3. Write Modes: Overwrite vs Append

The mode you choose determines what happens to the file:

• "w" (Write) → Creates a new file if it doesn’t exist. If it does, the existing contents are erased.

• "a" (Append) → Keeps the existing content and adds new data to the end.

• "x" (Exclusive create) → Creates a new file. If the file already exists, Python raises an error.

Example of appending:

with open("log.txt", "a", encoding="utf-8") as file:
    file.write("New log entry\n")

This ensures your old logs remain untouched while new entries are added.

4. Writing and Reading Together

Sometimes you may want to both read and write in the same file. That’s where combination modes come in:

• "r+" – Read and write (file must exist).

• "w+" – Write and read (erases file first).

• "a+" – Append and read.

Example:

with open("data.txt", "w+", encoding="utf-8") as file:
    file.write("Hello World!\n")
    file.seek(0)  # move cursor to beginning
    print(file.read())

5. Handling Non-String Data

If you try to write numbers directly, you’ll hit an error:

with open("numbers.txt", "w") as file:
    file.write(100)  # ❌ TypeError

Solution: convert to string first.

with open("numbers.txt", "w") as file:
    file.write(str(100))

Or, write multiple values in one go:

values = [1, 2, 3, 4, 5]
with open("numbers.txt", "w") as file:
    file.write(" ".join(str(v) for v in values))

6. Best Practices When Writing Files

• Always use the with statement—it ensures the file is closed properly even if errors occur.

• Be careful with "w" mode—it erases everything before writing. Use "a" if you want to preserve data.

• Always specify encoding="utf-8" when writing text, especially if the content may include special characters like emojis (😊) or accented letters.

• If you’re writing huge amounts of data, consider writing in chunks instead of building giant strings in memory.

✅ Quick Tip: If you’re writing logs or reports, append mode ("a") is often safer—you won’t accidentally wipe out your history.

File Cursor, Seeking, and Telling Position

When you work with files in Python, you may not realize that every read or write operation happens from a specific position in the file, called the file cursor. By default, the cursor starts at the beginning of the file when you open it for reading, or at the end when you open it for appending.

Understanding how to control this cursor using seek and tell can give you more flexibility when handling files — for example, if you need to re-read a portion of the file or skip ahead without reading everything line by line.

The File Cursor

• A cursor (also known as a file pointer) keeps track of the current position in the file.

• Each time you read or write, the cursor moves forward automatically.

• If you want to jump to a different location, you need to move the cursor manually.

The tell() Method

The tell() method lets you check the current cursor position (in bytes) from the beginning of the file.

with open("example.txt", "r") as file:
    content = file.read(5)   # Read first 5 characters
    print("After reading 5 chars, cursor is at:", file.tell())

Output example:

After reading 5 chars, cursor is at: 5

Here, the cursor moved forward by 5 bytes (one for each character).

The seek() Method

The seek() method allows you to move the cursor to a new position in the file. Its syntax is:

file.seek(offset, whence)

• offset → number of bytes to move.

• whence → reference point (default is 0).

  • 0: from the beginning of the file

  • 1: from the current cursor position

  • 2: from the end of the file

Example 1: Moving to the Beginning

with open("example.txt", "r") as file:
    file.read()              # Read entire file
    file.seek(0)             # Move back to start
    print("Cursor reset to:", file.tell())

Example 2: Skipping Ahead

with open("example.txt", "r") as file:
    file.seek(10)            # Move cursor to byte 10
    print(file.read(5))      # Read 5 characters from position 10

Example 3: Moving from the End

with open("example.txt", "rb") as file:   # must be in binary mode
    file.seek(-5, 2)                      # Go 5 bytes before the end
    print(file.read())                    # Read last 5 bytes

Why Are seek() and tell() Useful?

• Re-reading Data: Go back to the start or a specific section without reopening the file.

• Skipping Headers: Useful in structured files like CSVs or logs where you want to skip metadata.

• Working with Large Files: Jump to specific positions instead of loading the entire file into memory.

• Partial Reads: Extract just the needed portion without parsing everything.

Key Points to Remember

• Always think in bytes, not characters. With text files, Python handles encoding, so the position may not match exactly with characters if you’re using multi-byte encodings like UTF-8.

• For precise cursor control, especially with binary files, use "rb" or "wb" modes.

• Combining seek() and tell() gives you full control over file navigation.

Mastering the file cursor, along with seek() and tell(), gives you powerful control over how you interact with files in Python. Instead of being restricted to sequential reads and writes, you can now navigate freely within your files.

Handling Encodings and Newlines Properly

When working with text files in Python, it’s easy to assume that everything is plain ASCII or UTF-8. But in reality, files often come from different systems and applications, which may use different encodings and newline characters. If you don’t handle these properly, you could end up with garbled text, unreadable characters, or unexpected line breaks in your program.

Let’s break down how Python helps you manage encodings and newlines effectively.

What Is Encoding?

An encoding determines how characters (letters, numbers, symbols) are represented in bytes. Common encodings include:

• UTF-8: The most widely used today; supports almost every language.

• ASCII: Limited to basic English characters.

• ISO-8859-1 / Latin-1: Used in older European systems.

• UTF-16 / UTF-32: Uses more bytes per character; sometimes found in Windows-generated files.

If you open a file without specifying the encoding, Python uses the system default, which can vary depending on your operating system. This often leads to problems when sharing files across different platforms.

Specifying Encoding in Python

You can avoid issues by explicitly setting the encoding when opening a file:

# Safe way: specify encoding explicitly
with open("data.txt", "r", encoding="utf-8") as file:
    content = file.read()
    print(content)

If you’re unsure about the encoding of a file, utf-8 is usually the safest choice.

Common Encoding Errors

Sometimes you’ll see errors like:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 10

This happens when the file is not actually UTF-8 encoded. You can fix it by trying another encoding:

with open("data.txt", "r", encoding="latin-1") as file:
    content = file.read()

For robust programs, you can even catch encoding errors:

with open("data.txt", "r", encoding="utf-8", errors="replace") as file:
    content = file.read()
    print(content)  # Unrecognized chars replaced with �

Handling Newlines Across Platforms

Different operating systems use different newline characters:

• Linux/Unix: \n

• Windows: \r\n

• Mac (old versions): \r

Python automatically handles these differences in text mode ("r" or "w") by translating them into \n internally.

Example:

with open("example.txt", "r") as file:
    lines = file.readlines()
    print(lines)   # Each line ends with '\n' regardless of OS

The newline Parameter

If you want full control over how newlines are handled, use the newline parameter in open():

# Preserve exact newlines as they are in the file
with open("example.txt", "r", newline="") as file:
    for line in file:
        print(repr(line))

• newline=None (default): Universal newline mode. Converts all \r, \n, and \r\n into \n.

• newline="": No conversion; returns raw newlines exactly as in the file.

• newline="\n", "\r", or "\r\n": Force a specific newline when writing.

Example for writing with Windows-style newlines:

with open("output.txt", "w", newline="\r\n") as file:
    file.write("First line\nSecond line")

The resulting file will use \r\n line endings.

Best Practices for Encodings and Newlines

• Always specify encoding (prefer utf-8) instead of relying on defaults.

• Use errors="replace" or errors="ignore" for resilience when processing unknown files.

• Trust Python’s default newline handling unless you need exact raw line endings.

• Test files on multiple systems if they are meant to be shared across platforms.

By handling encodings and newlines properly, you ensure that your Python programs can read and write text files reliably — whether they come from your own system, a colleague’s computer, or a completely different environment.

Error Handling in File Operations

Working with files in Python may seem straightforward, but it’s not always smooth sailing. Files might be missing, locked by another program, have restricted permissions, or contain unexpected data. If these scenarios aren’t handled properly, your program will crash with unhandled exceptions.

To make your file-handling code robust and reliable, Python provides structured ways to deal with errors through exceptions. Let’s explore the common problems you may face and how to handle them effectively.

Common File-Related Errors

When working with files, you may encounter several types of exceptions:

• FileNotFoundError – Trying to open a file that doesn’t exist.

• PermissionError – Attempting to read/write a file without sufficient access rights.

• IsADirectoryError – Opening a directory as if it were a file.

• UnicodeDecodeError – Reading a file with the wrong encoding.

• OSError – Covers a variety of system-related errors.

Example of an unhandled error:

file = open("nonexistent.txt", "r")
# Raises: FileNotFoundError: [Errno 2] No such file or directory

Using try and except for File Handling

To prevent crashes, wrap file operations inside a try-except block:

try:
    with open("data.txt", "r") as file:
        content = file.read()
        print(content)
except FileNotFoundError:
    print("Error: The file was not found.")
except PermissionError:
    print("Error: You don’t have permission to access this file.")
except Exception as e:
    print("Unexpected error:", e)

Here:

• Specific exceptions (FileNotFoundError, PermissionError) are handled first.

• A general Exception block catches any unexpected issues.

The finally Block

Sometimes, you may want to guarantee cleanup actions (like closing a file) even if an error occurs. That’s where finally comes in:

try:
    file = open("example.txt", "r")
    print(file.read())
except FileNotFoundError:
    print("File does not exist.")
finally:
    if 'file' in locals() and not file.closed:
        file.close()
        print("File closed safely.")

While using with open(...) is cleaner (since it auto-closes files), finally is useful in more complex workflows.

Handling Encoding Errors

When dealing with text files from unknown sources, encoding errors are common. Handle them gracefully using the errors parameter:

try:
    with open("foreign_data.txt", "r", encoding="utf-8") as file:
        data = file.read()
except UnicodeDecodeError:
    print("Encoding issue: trying fallback encoding...")
    with open("foreign_data.txt", "r", encoding="latin-1") as file:
        data = file.read()

This ensures your program doesn’t crash due to unreadable characters.

Best Practices for Error Handling in Files

• Always use with open(...) to avoid forgetting close().

• Catch specific exceptions first for clarity.

• Provide helpful error messages so users know what went wrong.

• Use errors="replace" or fallback encodings for robust text file processing.

• Avoid silent failures — don’t just ignore errors without handling them.

Example: Safe File Reader Function

Here’s a function that demonstrates good practices for file error handling:

def safe_read(filename, encoding="utf-8"):
    try:
        with open(filename, "r", encoding=encoding, errors="replace") as file:
            return file.read()
    except FileNotFoundError:
        print(f"Error: {filename} was not found.")
    except PermissionError:
        print(f"Error: You don’t have permission to open {filename}.")
    except Exception as e:
        print("Unexpected error:", e)
    return None

# Usage
content = safe_read("notes.txt")
if content:
    print(content)

This approach ensures your program remains stable while still providing useful feedback.

By handling errors properly in file operations, you not only prevent program crashes but also make your applications more user-friendly, predictable, and professional.

Exercises

Here are some practice exercises to reinforce what you’ve learned about reading and writing text files in Python:

1. Read a File Line by Line

   • Create a text file with a few lines of content.

   • Write a Python program that opens the file and prints each line with its line number.

2. Write User Input to a File

   • Ask the user to enter three lines of text.

   • Save these lines into a file called notes.txt.

3. Append to an Existing File

   • Open notes.txt again, but this time append a few more lines instead of overwriting.

   • Verify by reading the file after appending.

4. Count Words in a File

   • Write a program that reads a text file and counts the total number of words in it.

5. Challenge Exercise: Simple Log File Manager

   • Write a program that simulates a log system:

     • Every time the program runs, it appends a new line to log.txt with the current date and time.

     • The program should also allow the user to choose between viewing the last 5 log entries or clearing the file.

   • Use file reading, writing, appending, and error handling to complete this task.

Summary and Key Takeaways

File handling is one of the most important aspects of Python programming because almost every real-world application needs to read from or write to files. In this post, we’ve explored the complete process of reading and writing text files and learned best practices to make our code reliable and efficient.

Here are the key lessons to remember:

• The open() Function: Use open(filename, mode, encoding) to start working with files. The mode ("r", "w", "a", "x", etc.) defines how you interact with the file.

• Reading Files: Use .read(), .readline(), or .readlines() depending on whether you want the whole file, a single line, or all lines at once.

• Writing Files: Methods like .write() and .writelines() allow you to add or overwrite text. The mode ("w" vs "a") controls whether old content is erased or preserved.

• Context Managers (with Statement): Always prefer with open(...) as file: because it automatically closes the file and prevents resource leaks.

• File Cursor Control: With .seek() and .tell(), you can move the file pointer and check positions — essential for advanced file operations.

• Encodings and Newlines: Always specify an encoding (preferably utf-8) to avoid cross-platform issues. Use the newline parameter if you need precise control over line endings.

• Error Handling: Wrap file operations in try-except blocks to gracefully handle missing files, permission errors, or encoding problems. Provide clear feedback instead of letting the program crash.

Best Practice Snapshot

1. Always use with open(...) for safety.

2. Explicitly set encoding="utf-8" for portability.

3. Use .seek() and .tell() when precise cursor movement is required.

4. Handle errors explicitly with helpful messages.

5. Test your file-handling code on different operating systems if the files will be shared.

By mastering these techniques, you’ll be able to handle text files confidently and build Python programs that can process data, store results, and interact with users through files in a professional way.

This foundation will also prepare you for more advanced file handling tasks, such as working with CSV, JSON, or binary files.