Python – Understanding the ‘with’ Statement in Python

flaskpython

I am trying to understand the with statement in python. Everywhere I look it talks of opening and closing a file, and is meant to replace the try-finally block. Could someone post some other examples too. I am just trying out flask and there are with statements galore in it. Definitely request someone to provide some clarity on it.

Best Answer

The idea of the with statement is to make "doing the right thing" the path of least resistance. While the file example is the simplest, threading locks actually provide a more classic example of non-obviously buggy code:

try:
    lock.acquire()
    # do stuff
finally:
    lock.release()

This code is broken - if the lock acquisition ever fails, either the wrong exception will be thrown (since the code will attempt to release a lock that it never acquired), or, worse, if this is a recursive lock, it will be released early. The correct code looks like this:

lock.acquire()
try:
    # do stuff
finally:
    # If lock.acquire() fails, this *doesn't* run
    lock.release()

By using a with statement, it becomes impossible to get this wrong, since it is built into the context manager:

with lock: # The lock *knows* how to correctly handle acquisition and release
  # do stuff

The other place where the with statement helps greatly is similar to the major benefit of function and class decorators: it takes "two piece" code, which may be separated by an arbitrary number of lines of code (the function definition for decorators, the try block in the current case) and turns it into "one piece" code where the programmer simply declares up front what they're trying to do.

For short examples, this doesn't look like a big gain, but it actually makes a huge difference when reviewing code. When I see lock.acquire() in a piece of code, I need to scroll down and check for a corresponding lock.release(). When I see with lock:, though, no such check is needed - I can see immediately that the lock will be released correctly.

Related Solutions

Python ‘with’ Statement – What is it Designed For?

I believe this has already been answered by other users before me, so I only add it for the sake of completeness: the with statement simplifies exception handling by encapsulating common preparation and cleanup tasks in so-called context managers. More details can be found in PEP 343. For instance, the open statement is a context manager in itself, which lets you open a file, keep it open as long as the execution is in the context of the with statement where you used it, and close it as soon as you leave the context, no matter whether you have left it because of an exception or during regular control flow. The with statement can thus be used in ways similar to the RAII pattern in C++: some resource is acquired by the with statement and released when you leave the with context.

Some examples are: opening files using with open(filename) as fp:, acquiring locks using with lock: (where lock is an instance of threading.Lock). You can also construct your own context managers using the contextmanager decorator from contextlib. For instance, I often use this when I have to change the current directory temporarily and then return to where I was:

from contextlib import contextmanager
import os

@contextmanager
def working_directory(path):
    current_dir = os.getcwd()
    os.chdir(path)
    try:
        yield
    finally:
        os.chdir(current_dir)

with working_directory("data/stuff"):
    # do something within data/stuff
# here I am back again in the original working directory

Here's another example that temporarily redirects sys.stdin, sys.stdout and sys.stderr to some other file handle and restores them later:

from contextlib import contextmanager
import sys

@contextmanager
def redirected(**kwds):
    stream_names = ["stdin", "stdout", "stderr"]
    old_streams = {}
    try:
        for sname in stream_names:
            stream = kwds.get(sname, None)
            if stream is not None and stream != getattr(sys, sname):
                old_streams[sname] = getattr(sys, sname)
                setattr(sys, sname, stream)
        yield
    finally:
        for sname, stream in old_streams.iteritems():
            setattr(sys, sname, stream)

with redirected(stdout=open("/tmp/log.txt", "w")):
     # these print statements will go to /tmp/log.txt
     print "Test entry 1"
     print "Test entry 2"
# back to the normal stdout
print "Back to normal stdout again"

And finally, another example that creates a temporary folder and cleans it up when leaving the context:

from tempfile import mkdtemp
from shutil import rmtree

@contextmanager
def temporary_dir(*args, **kwds):
    name = mkdtemp(*args, **kwds)
    try:
        yield name
    finally:
        shutil.rmtree(name)

with temporary_dir() as dirname:
    # do whatever you want

Flask Jinja2 – How Does the ‘With’ Statement Work?

{% with %} statement in Jinja lets you define variable, but limits the scope of a variable with the {% endwith %}

statement. For example :

{% with myvar=1 %}
...
{% endwith %}

Any elements declared in the body will have access to the myvar variable.

Please, refer - https://www.webforefront.com/django/usebuiltinjinjastatements.html

Best Answer

Related Solutions

Python ‘with’ Statement – What is it Designed For?

Flask Jinja2 – How Does the ‘With’ Statement Work?

Related Question