Python Context Managers – How to Handle Resources Like a Pro

When I first started coding in Python, one of the most common mistakes I made was forgetting to close files after I opened them. This can lead to resource leaks or even data corruption, which are headaches you don’t want to deal with. The solution, I quickly learned, lies in one of Python’s most elegant features: context managers, typically used with the with statement.

Context managers are a game-changer for handling resources like files, database connections, or threading locks. They automate the setup and teardown processes, guaranteeing that cleanup code—like closing a file—runs every single time, even if errors pop up midway through. In this guide, I’ll walk you through how they work, how to build your own, and how to use Python’s powerful

contextlib module to make your code safer and cleaner. For those just getting started, you might want to check out a beginner’s introduction to Python first.

---

Common Context Managers in the Standard Library

Before building your own, it’s helpful to see how context managers are already used throughout Python. I’ve found these to be some of the most common and useful examples.

File Handling

This is the classic example. The built-in open() function acts as a context manager to ensure files are automatically closed.

Python

with open('my_file.txt', 'w') as f:
    f.write('Hello, world!')
# The file is automatically closed here 

Thread Safety with Locks

In multithreaded applications, you need to prevent race conditions where multiple threads modify the same resource simultaneously. The

threading.Lock class provides a context manager to make sure locks are always released.

Python

import threading

lock = threading.Lock()
with lock:
    # Safely access shared resources inside this block 
# The lock is automatically released 

Database Connection Management

Properly managing database connections is critical. The

sqlite3 module’s connect() function returns a context manager that handles closing the connection for you.

Python

import sqlite3

with sqlite3.connect('my_database.db') as conn:
    cursor = conn.cursor()
    cursor.execute("SELECT * FROM tablename")
    results = cursor.fetchall()
# The database connection is closed automatically 

---

How to Create Your Own Context Managers

While the built-in options are great, the real power comes from creating your own. I’ve used custom context managers for everything from timing code blocks to temporarily changing application settings.

The Classic Method: Using Dunder Methods

You can create a context manager by defining a class with two special “dunder” methods:

__enter__ and __exit__.

• __enter__(self): This method handles the setup. It’s called when the with block is entered and typically returns the resource that will be used.
• __exit__(self, exc_type, exc_value, traceback): This method handles the cleanup. It’s called when the with block is exited, and it receives any exception information if an error occurred. This is crucial for building robust code that can handle errors gracefully.

Here’s an example from the youtube-dl library that creates a context manager to lock a file during access.

Python

# File: youtube_dl/utils.py
class locked_file(object):
    def __init__(self, filename, mode, encoding=None):
        self.f = io.open(filename, mode, encoding=encoding)
        self.mode = mode

    def __enter__(self):
        exclusive = self.mode != 'r'
        try:
            _lock_file(self.f, exclusive)
        except IOError:
            self.f.close()
            raise
        return self

    def __exit__(self, etype, value, traceback):
        try:
            _unlock_file(self.f)
        finally:
            self.f.close()

In this class,

__enter__ locks the file, and __exit__ ensures it’s unlocked and closed, no matter what happens inside the with block.

The Simple Way: The @contextmanager Decorator

Writing a full class feels like overkill for simple cases. That’s where the

@contextmanager decorator from the contextlib module comes in. It lets you define a context manager using a simple generator function with a yield statement.

Everything before the

yield is the setup (__enter__), and everything after the yield is the cleanup (__exit__). In my experience, this is the most common and readable way to create a custom context manager.

This example from the transformers library temporarily checks out a specific git commit.

Python

# File: transformers/benchmark/benchmark.py
from contextlib import contextmanager

@contextmanager
def checkout_commit(repo: Repo, commit_id: str):
    """
    Context manager that checks out a given commit when entered,
    but gets back to the reference it was at on exit.
    """
    current_head = repo.head.commit if repo.head.is_detached else repo.head.ref
    try:
        repo.git.checkout(commit_id)
        yield
    finally:
        repo.git.checkout(current_head)

The

try...finally block here guarantees that the repository is always returned to its original state.

---

Advanced Tools in contextlib

The contextlib module offers more than just decorators. These advanced tools have helped me solve complex resource management problems in a clean way.

ExitStack: Managing Multiple Dynamic Contexts

What if you need to open a variable number of files? Nesting with statements can get messy.

ExitStack allows you to manage multiple context managers within a single with block, even if you don’t know how many you’ll need until runtime.

In this example from Posthog,

ExitStack is used to conditionally apply a freeze_time context manager only when a created_at timestamp is present in the data.

Python

# File: posthog/models/person/util.py
def bulk_create_persons(persons_list: list[dict]):
    persons = []
    for _person in persons_list:
        with ExitStack() as stack:
            if _person.get("created_at"):
                # Dynamically enter the context only when needed
                stack.enter_context(freeze_time(_person["created_at"]))
            persons.append(Person(**{...}))

suppress: Ignoring Specific Exceptions

Sometimes, you expect an exception to occur and want to safely ignore it without a try...except pass block.

contextlib.suppress() lets you specify which exceptions to ignore within its context.

This is perfect for situations like trying to load a cache file that might not exist yet. If an

OSError occurs, it’s suppressed, and the code continues, returning a default value.

Python

# File: yt_dlp/cache.py
def load(self, section, key, ...):
    ...
    cache_fn = self._get_cache_fn(section, key, dtype)
    with contextlib.suppress(OSError):
        try:
            with open(cache_fn, encoding="utf-8") as cachef:
                return self._validate(json.load(cachef), min_ver)
        except (ValueError, KeyError):
            ...
    return default

redirect_stdout: Capturing Output

I’ve often used this for testing or logging.

redirect_stdout temporarily sends standard output to a file or another stream-like object. This allows you to capture the output of a function that would normally print to the console.

Python

# File: perfetto/tools/extract_linux_syscall_tables
def Main():
    ...
    with open(tmp_file, "w") as f:
        with contextlib.redirect_stdout(f):
            print_tables() # Output of this function goes to the file f

Conclusion

Context managers are a fundamental part of writing clean, reliable, and professional Python code. They provide a simple syntax for a powerful concept: guaranteeing that resources are properly managed. Whether you’re using the built-in ones for files and locks or creating your own to time code or patch environments, they help prevent bugs and make your code’s intent much clearer.

By mastering tools like @contextmanager and ExitStack, you can handle even complex setup and cleanup logic with ease, making your code more robust for advanced applications in fields like data science or cybersecurity.

More Topics

• Python’s Itertools Module – How to Loop More Efficiently
• Python Multithreading – How to Handle Concurrent Tasks
• Python Multiprocessing – How to Use Multiple CPU Cores
• Python Asyncio – How to Write Concurrent Code
• Python Data Serialization – How to Store and Transmit Your Data
• Python Project Guide: NumPy & SciPy: For Science!
• Python Project Guide: Times, Dates & Numbers