After one error, debug mode stayed on for everyone

See the problem

Your app keeps its settings in one dict. To build a detailed report, you turn debug on, build it, and turn debug off.

SETTINGS["debug"] = True
text = build_report()
SETTINGS["debug"] = False

One day build_report() raises. The last line never runs. Debug stays on, and every user now sees internal details on every page, until someone restarts the server.

Define the task

• Input: a settings dict, a key and a temporary value.
• Output: inside a block, the key has the new value. After the block, it has its old value again.
• Rules: the old value comes back even if the block raises. If the key wasn’t there before, it is removed after.
• The hard part: undo work that no error can skip, written once.

Example: with override(SETTINGS, "debug", True): build_report(). After the block, SETTINGS["debug"] is False again, even if build_report raised.

Pause and think

⏸ Pause and think

Which Python statement runs its code whether the block before it returns, raises or finishes?

Give it two minutes with pen and paper before you open a hint.

Hints open one at a time:

Hint 1 of 3

try: … finally: … runs the finally part in every case.

Hint 2 of 3

With @contextmanager, the code before yield is set-up, the with block runs at yield, and the code after is clean-up.

Hint 3 of 3

Put the yield inside try, and the undo in finally.

Understand the idea

A context manager is an object with two methods. with cm as x: runs:

Step	What runs
1	x = cm.__enter__(): the set-up. Its return value goes to as x.
2	The body of the with block.
3	cm.__exit__(type, error, traceback): the clean-up, always. All three are None when nothing failed.
4	If __exit__ returns True, the error is swallowed. Otherwise it continues up.

You know some already: with open(path) as f: closes the file, and with lock: releases the lock.

@contextmanager from contextlib turns a generator into a context manager. Code before yield is step 1, the with block runs at yield, and code after is step 3. If the block raises, the error appears at the yield, so wrap it in try and finally.

★Key insight

A context manager pairs set-up with clean-up. with runs the clean-up when the block ends, however it ends. Write one with @contextmanager and try: yield … finally:.

Climb the ladder

All three do O(1) extra work. What changes is whether the undo can be skipped, and how often you write it.

Okay

Set, run, put back

The idea: Save the old value, set the new one, build the report, then restore the old value.

When nothing fails, debug is back off after the report.

SETTINGS: dict[str, object] = {"debug": False, "currency": "GBP"}


def report_in_debug() -> str:
    old = SETTINGS["debug"]
    SETTINGS["debug"] = True
    text = build_report()  # if this raises, the next line never runs
    SETTINGS["debug"] = old
    return text


def build_report() -> str:
    return f"debug={SETTINGS['debug']}"

Time

O(1) extra

Space

O(1)

Why

One saved value.

Where it breaks: if build_report() raises, the restore line never runs, and debug stays on.

Better

try and finally

The idea: Build the report inside try, and restore the old value in finally.

The waste in Okay: an error skips the undo.

finally runs whether the try part returns or raises.

Highlighted lines are new in this rung.

SETTINGS: dict[str, object] = {"debug": False, "currency": "GBP"}


def report_in_debug() -> str:
    old = SETTINGS["debug"]
    SETTINGS["debug"] = True
    try:
        return build_report()
    finally:  # runs whether build_report returns or raises
        SETTINGS["debug"] = old


def build_report() -> str:
    return f"debug={SETTINGS['debug']}"

Time

O(1) extra

Space

O(1)

Why

One saved value.

What changed from the previous rung

SETTINGS: dict[str, object] = {"debug": False, "currency": "GBP"}


def report_in_debug() -> str:
    old = SETTINGS["debug"]
    SETTINGS["debug"] = True
    text = build_report()  # if this raises, the next line never runs
    SETTINGS["debug"] = old
    return text
    try:
        return build_report()
    finally:  # runs whether build_report returns or raises
        SETTINGS["debug"] = old


def build_report() -> str:
    return f"debug={SETTINGS['debug']}"

Where it breaks: every place that changes a setting needs the same five lines. Forget the finally once, and the old bug is back.

Best

A context manager

The idea: override(settings, key, value) sets the value, yields inside try, and puts the old value back in finally.

The waste in Better: the same try and finally written at every call.

Now any code writes with override(SETTINGS, "debug", True):. The MISSING marker tells “no value” apart from a real None, so a key that wasn’t there is removed.

Highlighted lines are new in this rung.

from collections.abc import Iterator
from contextlib import contextmanager

SETTINGS: dict[str, object] = {"debug": False, "currency": "GBP"}
MISSING = object()  # a marker: "there was no value"


@contextmanager
def override(settings: dict, key: str, value: object) -> Iterator[dict]:
    old = settings.get(key, MISSING)
    settings[key] = value
    try:
        yield settings  # the body of the with block runs here
    finally:
        if old is MISSING:
            del settings[key]  # it wasn't there before, so remove it
        else:
            settings[key] = old


def report_in_debug() -> str:
    with override(SETTINGS, "debug", True):
        return build_report()


def build_report() -> str:
    return f"debug={SETTINGS['debug']}"

Time

O(1) extra

Space

O(1)

Why

One saved value.

What changed from the previous rung

from collections.abc import Iterator
from contextlib import contextmanager

SETTINGS: dict[str, object] = {"debug": False, "currency": "GBP"}
MISSING = object()  # a marker: "there was no value"


def report_in_debug() -> str:
    old = SETTINGS["debug"]
    SETTINGS["debug"] = True
@contextmanager
def override(settings: dict, key: str, value: object) -> Iterator[dict]:
    old = settings.get(key, MISSING)
    settings[key] = value
    try:
        yield settings  # the body of the with block runs here
    finally:
        if old is MISSING:
            del settings[key]  # it wasn't there before, so remove it
        else:
            settings[key] = old


def report_in_debug() -> str:
    with override(SETTINGS, "debug", True):
        return build_report()
    finally:  # runs whether build_report returns or raises
        SETTINGS["debug"] = old


def build_report() -> str:
    return f"debug={SETTINGS['debug']}"

When Best isn’t best: for one-off work, plain try and finally is fine. Test code often uses unittest.mock.patch.dict, which does this already.

Write it yourself

Write override with @contextmanager. The starter sets the value but never puts it back.

Change a setting for one block

Ctrl + Enter runs

from contextlib import contextmanager @contextmanager def override(settings, key, value): settings[key] = value yield settings

Run testsReset codeLoads about 6 MB the first time. Best on Wi-Fi.

1. ○Not run yet: debug inside the block, then after it → [True, False]
2. ○Not run yet: the block raises: is debug back off? → {'debug': False}
3. ○Not run yet: a key that wasn’t there is removed after → {'a': 1}
4. ○Not run yet: Extra test

Extra tests check tricky cases. You see their input only if they fail.

Try your own input

Arguments, separated by commas as in Python

override()Run onceVisualize

Compare

Way	Undo after an error	Lines at each use	Use it when
Okay: set and put back	No	3	Never, if anything can fail
Better: try and finally	Yes	5	Once, in one place
Best: a context manager	Yes	1	The same set-up in many places

Common mistakes and edge cases

!yield without try

In a @contextmanager, an error in the with block appears at the yield. Without try and finally around the yield, the clean-up after it never runs. That is the Okay bug again, hidden inside a context manager.

• __exit__ returning True swallows the error. Return None or False unless swallowing is the whole point.
• as x gets what __enter__ returns. open() returns the file. A class that forgets return self gives None.
• Several at once: with open(a) as f, open(b) as g: closes both, in reverse order.
• Ready-made ones: contextlib.suppress, contextlib.chdir (3.11+), tempfile.TemporaryDirectory(), threading.Lock() and decimal.localcontext().
• A context manager doesn’t end the variable: f still exists after with open(...) as f:; it is just closed.

Say it in the interview

Questions to ask first

• What must always be undone: a file, a lock, a connection, a setting?
• Should errors inside the block continue up, or be handled here?

A 60-second walkthrough

“Set-up that must be undone is a context manager. I’d write override with contextlib.contextmanager: save the old value with a marker for ‘missing’, set the new one, then yield inside try with the restore in finally. The with block runs at the yield, and if it raises, the error comes out there, so finally still restores. As a class, __enter__ does the set-up, and its return value goes to as. __exit__ does the clean-up, and returns falsy so errors continue up.”

Likely follow-ups

• What are __exit__’s three arguments? The error’s type, the error and its traceback, or three Nones if nothing failed.
• How do you open a varying number of files? contextlib.ExitStack: stack.enter_context(open(p)) for each, all closed at the end.
• Can a context manager be async? Yes: async with, with __aenter__ and __aexit__, or @asynccontextmanager.

Practice

Three problems, from easy to hard. Each has hints, tests you can run here, and its own Okay, Better and Best answers. Try before you open them.

1. A timer for a block (easy)

Write a class Timer. with Timer() as t: times the block, and afterwards t.seconds holds how long it took. It must still record the time when the block raises, and let the error continue.

Hint 1 of 3

Read time.perf_counter() in __enter__, and again in __exit__.

Hint 2 of 3

"as t" gets whatever __enter__ returns. Return self.

Hint 3 of 3

__exit__ returns None, so errors continue up.

A timer for a block

Ctrl + Enter runs

import time class Timer: def __enter__(self): self.began = time.perf_counter() def __exit__(self, *exc): pass

Run testsReset codeLoads about 6 MB the first time. Best on Wi-Fi.

1. ○Not run yet: the block runs: sum(range(10)) → 45
2. ○Not run yet: with Timer() as t; t.seconds >= 0 → True
3. ○Not run yet: Extra test

Extra tests check tricky cases. You see their input only if they fail.

Try your own input

Arguments, separated by commas as in Python

Timer()Run onceVisualize

Show the Okay, Better and Best answers

Okay

start() and stop()

The idea: Two methods the caller must remember to call.

It times whatever sits between the calls.

import time


class Timer:
    def start(self) -> None:
        self.began = time.perf_counter()

    def stop(self) -> None:
        self.seconds = time.perf_counter() - self.began

Time

O(1)

Space

O(1)

Why

Two clock reads.

Where it breaks: if the code between raises, stop() never runs. And with Timer(): fails, because there is no __enter__.

Better

__enter__ and __exit__

The idea: Read the clock in __enter__ and work out seconds in __exit__.

The waste in Okay: the caller has to remember stop().

Highlighted lines are new in this rung.

import time


class Timer:
    def __enter__(self) -> None:  # returns None, so "as t" gives None
        self.began = time.perf_counter()

    def __exit__(self, *exc: object) -> None:
        self.seconds = time.perf_counter() - self.began

Time

O(1)

Space

O(1)

Why

Two clock reads.

What changed from the previous rung

import time


class Timer:
    def start(self) -> None:
    def __enter__(self) -> None:  # returns None, so "as t" gives None
        self.began = time.perf_counter()

    def stop(self) -> None:
    def __exit__(self, *exc: object) -> None:
        self.seconds = time.perf_counter() - self.began

Where it breaks: __enter__ returns None, so as t sets t to None and t.seconds raises AttributeError.

Best

Return self

The idea: __enter__ returns self, so as t gives the timer.

The waste in Better: the timer did its work, but nobody can read it.

Highlighted lines are new in this rung.

import time
from typing import Self


class Timer:
    def __enter__(self) -> Self:
        self.began = time.perf_counter()
        return self  # this is what "as t" gets

    def __exit__(self, *exc: object) -> None:  # None: errors carry on up
        self.seconds = time.perf_counter() - self.began

Time

O(1)

Space

O(1)

Why

Two clock reads.

What changed from the previous rung

import time
from typing import Self


class Timer:
    def __enter__(self) -> None:  # returns None, so "as t" gives None
    def __enter__(self) -> Self:
        self.began = time.perf_counter()
        return self  # this is what "as t" gets

    def __exit__(self, *exc: object) -> None:
    def __exit__(self, *exc: object) -> None:  # None: errors carry on up
        self.seconds = time.perf_counter() - self.began

When Best isn’t best: to time one statement many times, use the timeit module. It repeats the code and ignores noise.

2. Ignore some errors (medium)

Write a class ignore(*errors), a small contextlib.suppress. with ignore(ValueError): swallows a ValueError from the block, or any error that is a kind of one. Every other error continues up.

Hint 1 of 3

__exit__ gets the error type first. It is None when nothing failed.

Hint 2 of 3

issubclass(exc_type, self.errors) works with a tuple of classes.

Hint 3 of 3

Return True to swallow the error, False to let it go.

Ignore some errors

Ctrl + Enter runs

class ignore: def __init__(self, *errors): self.errors = errors def __enter__(self): return None def __exit__(self, exc_type, exc, tb): return False

Run testsReset codeLoads about 6 MB the first time. Best on Wi-Fi.

1. ○Not run yet: ignore(ValueError): [1].remove(9) → 'carried on'
2. ○Not run yet: ignore(ValueError): a KeyError → raises KeyError
3. ○Not run yet: ignore(LookupError): an IndexError → 'carried on'
4. ○Not run yet: Extra test

Extra tests check tricky cases. You see their input only if they fail.

Try your own input

Arguments, separated by commas as in Python

ignore()Run onceVisualize

Show the Okay, Better and Best answers

Okay

try and except, each time

The idea: Write try: … except ValueError: pass at every place.

Clear, and only ValueError is caught.

def remove_all(items: list, values: list) -> list:
    for v in values:
        try:  # noqa: SIM105  (the rung shows the try that suppress replaces)
            items.remove(v)
        except ValueError:
            pass
    return items

Time

O(1)

Space

O(1)

Why

One check per error.

Where it breaks: four lines every time, for one idea: “this error is fine here”.

Better

__exit__ returns True

The idea: A context manager whose __exit__ always returns True.

The waste in Okay: the same four lines everywhere.

Highlighted lines are new in this rung.

class ignore:  # lower case, like contextlib.suppress
    def __init__(self, *errors: type[BaseException]) -> None:
        self.errors = errors

    def __enter__(self) -> None:
        return None

    def __exit__(self, *exc: object) -> bool:
        return True  # True means "the error is handled": every error vanishes

Time

O(1)

Space

O(1)

Why

No checks at all.

What changed from the previous rung

def remove_all(items: list, values: list) -> list:
    for v in values:
        try:  # noqa: SIM105  (the rung shows the try that suppress replaces)
            items.remove(v)
        except ValueError:
            pass
    return items
class ignore:  # lower case, like contextlib.suppress
    def __init__(self, *errors: type[BaseException]) -> None:
        self.errors = errors

    def __enter__(self) -> None:
        return None

    def __exit__(self, *exc: object) -> bool:
        return True  # True means "the error is handled": every error vanishes

Where it breaks: it never looks at the error, so a KeyError from a real bug disappears too.

Best

Swallow only the listed errors

The idea: Return True only if exc_type is a subclass of one of self.errors.

The waste in Better: it hides bugs along with the error you expected.

Highlighted lines are new in this rung.

from types import TracebackType


class ignore:  # lower case, like contextlib.suppress
    def __init__(self, *errors: type[BaseException]) -> None:
        self.errors = errors

    def __enter__(self) -> None:
        return None

    def __exit__(
        self,
        exc_type: type[BaseException] | None,
        exc: BaseException | None,
        tb: TracebackType | None,
    ) -> bool:
        # Swallow only the errors we were told about. No error: exc_type is None.
        return exc_type is not None and issubclass(exc_type, self.errors)

Time

O(e)

Space

O(1)

Why

issubclass checks each of the e listed classes.

What changed from the previous rung

from types import TracebackType


class ignore:  # lower case, like contextlib.suppress
    def __init__(self, *errors: type[BaseException]) -> None:
        self.errors = errors

    def __enter__(self) -> None:
        return None

    def __exit__(self, *exc: object) -> bool:
        return True  # True means "the error is handled": every error vanishes
    def __exit__(
        self,
        exc_type: type[BaseException] | None,
        exc: BaseException | None,
        tb: TracebackType | None,
    ) -> bool:
        # Swallow only the errors we were told about. No error: exc_type is None.
        return exc_type is not None and issubclass(exc_type, self.errors)

When Best isn’t best: use contextlib.suppress, which is this, already written and tested.

3. All or nothing (hard)

Write transaction(balances) with @contextmanager. Changes made in the block stay if it finishes. If it raises, every balance goes back to what it was before, new keys are removed, and the error continues up.

Hint 1 of 3

Save a copy of the dict before the yield.

Hint 2 of 3

In except, clear the dict and update it from the copy. That keeps the same dict object.

Hint 3 of 3

End the except with a bare raise, so the caller sees the error.

All or nothing

Ctrl + Enter runs

from contextlib import contextmanager @contextmanager def transaction(balances): yield balances

Run testsReset codeLoads about 6 MB the first time. Best on Wi-Fi.

1. ○Not run yet: a transfer that fails half way → {'ana': 10, 'bo': 0}
2. ○Not run yet: does the caller still see the error? → raises ValueError
3. ○Not run yet: a transfer that works → {'ana': 6, 'bo': 4}
4. ○Not run yet: Extra test

Extra tests check tricky cases. You see their input only if they fail.

Try your own input

Arguments, separated by commas as in Python

transaction()Run onceVisualize

Show the Okay, Better and Best answers

Okay

Just yield

The idea: A context manager with no undo.

Errors continue up, and good transfers stay.

from collections.abc import Iterator
from contextlib import contextmanager


@contextmanager
def transaction(balances: dict[str, int]) -> Iterator[dict[str, int]]:
    yield balances  # no undo: half a transfer stays done

Time

O(1)

Space

O(1)

Why

Nothing is saved.

Where it breaks: a transfer that fails half way leaves ana 50 down, and bo got nothing.

Better

Undo in except

The idea: Copy the dict first; on an error, put the copy back.

The waste in Okay: half a transfer stays done.

Highlighted lines are new in this rung.

from collections.abc import Iterator
from contextlib import contextmanager


@contextmanager
def transaction(balances: dict[str, int]) -> Iterator[dict[str, int]]:
    saved = dict(balances)
    try:
        yield balances
    except Exception:  # the flaw: the error is swallowed
        balances.clear()
        balances.update(saved)

Time

O(n)

Space

O(n)

Why

The copy holds all n balances.

What changed from the previous rung

from collections.abc import Iterator
from contextlib import contextmanager


@contextmanager
def transaction(balances: dict[str, int]) -> Iterator[dict[str, int]]:
    yield balances  # no undo: half a transfer stays done
    saved = dict(balances)
    try:
        yield balances
    except Exception:  # the flaw: the error is swallowed
        balances.clear()
        balances.update(saved)

Where it breaks: the except never raises again, so the error is swallowed. The caller thinks the transfer worked.

Best

Undo, then raise

The idea: Put the copy back, then a bare raise sends the same error on.

The waste in Better: it hides the failure it just cleaned up.

Highlighted lines are new in this rung.

from collections.abc import Iterator
from contextlib import contextmanager


@contextmanager
def transaction(balances: dict[str, int]) -> Iterator[dict[str, int]]:
    saved = dict(balances)
    try:
        yield balances
    except BaseException:
        balances.clear()
        balances.update(saved)  # put every balance back
        raise  # and let the caller know it failed

Time

O(n)

Space

O(n)

Why

The copy holds all n balances.

What changed from the previous rung

from collections.abc import Iterator
from contextlib import contextmanager


@contextmanager
def transaction(balances: dict[str, int]) -> Iterator[dict[str, int]]:
    saved = dict(balances)
    try:
        yield balances
    except Exception:  # the flaw: the error is swallowed
    except BaseException:
        balances.clear()
        balances.update(saved)
        balances.update(saved)  # put every balance back
        raise  # and let the caller know it failed

When Best isn’t best: dict(balances) is a shallow copy. If values are lists, changes inside them aren’t undone; use copy.deepcopy. Real money lives in a database, whose own transactions do this safely.

Check yourself

A few quick questions. Ones you miss are remembered, so you can find your weak spots later.

1. Predict the outputWhat does this print?

   class Box:
       def __enter__(self):
           print("in")
       def __exit__(self, *exc):
           print("out")
   
   with Box() as b:
       print(b)

   in, then outin, None, outin, <Box object>, outAn error

   Check answer
2. Spot the bugWhen the with block raises, the lock is never released. Why?

   @contextmanager
   def held(lock):
       lock.acquire()
       yield
       lock.release()

   yield must return the lockThe error appears at yield, so the line after it never runs; use try and finallycontextmanager only works on classesacquire must be after yield

   Check answer
3. Multiple choiceWhat happens when __exit__ returns True?

   The block runs againThe error from the block is swallowedThe error is raised twiceNothing special

   Check answer
4. Predict the outputWhat does this print?

   from contextlib import suppress
   
   with suppress(KeyError):
       print("a")
       {}["x"]
       print("b")
   print("c")

   a b ca ca, then a KeyErrorc

   Check answer

Flashcards

Add these to your review deck. Each card comes back a little later every time you get it right.

• Card 1What does a with statement run, in order?Show answer
• Card 2What does as x get?Show answer
• Card 3How do you write a context manager without a class?Show answer
• Card 4What does __exit__ returning True do?Show answer

Add 4 cards to my review deck

Cheat sheet

• with cm as x: runs __enter__, the block, then __exit__, always.
• as x gets what __enter__ returns.
• __exit__ returning True swallows the error.
• @contextmanager: set-up, try: yield … finally: clean-up.
• contextlib.suppress(Error), ExitStack for many at once.

Go deeper

• The with statement, in the language reference.
• contextlib: contextmanager, suppress, ExitStack.
• PEP 343, which added with.
• Before this: Exceptions. Next: collections, itertools and functools.

Can you do these now?

If you’re not sure of any of these, scroll back to that part before moving on.

• Say what with does, and why the clean-up runs even after an error
• Write a context manager with @contextmanager, try and finally
• Write one as a class with __enter__ and __exit__, and say what each returns
• Decide when __exit__ should swallow an error, and when it must let it go

✓Mark as completeI already know this

Status: Not started