I have been studying fastai course part2.

In lesson 15 and lesson 16 Jeremy introduced Learner, a class that include model, dataloaders, loss function, optimizer.

Jeremy used some advanced python features that I can’t understand well, here is the experiment that I used to help me understand those python features.

1 attrgetter

from operator import attrgetter

attrgetter?

Init signature: attrgetter(self, /, *args, **kwargs)
Docstring:     
attrgetter(attr, ...) --> attrgetter object
Return a callable object that fetches the given attribute(s) from its operand.
After f = attrgetter('name'), the call f(r) returns r.name.
After g = attrgetter('name', 'date'), the call g(r) returns (r.name, r.date).
After h = attrgetter('name.first', 'name.last'), the call h(r) returns
(r.name.first, r.name.last).
File:           ~/conda/envs/ml/lib/python3.10/operator.py
Type:           type
Subclasses:

f = attrgetter('name')

class person:
    name = 'Joe'
    date = '05-01-2023'

p = person()

f(p)

'Joe'

f2 = attrgetter('name', 'date')
f2(p)

('Joe', '05-01-2023')

The source of the course use attrgetter is:

def run_cbs(cbs, method_nm, learn=None):
    for cb in sorted(cbs, key=attrgetter('order')):
        method = getattr(cb, method_nm, None)
        if method is not None: method(learn)

sorted?

Signature: sorted(iterable, /, *, key=None, reverse=False)
Docstring:
Return a new list containing all items from the iterable in ascending order.
A custom key function can be supplied to customize the sort order, and the
reverse flag can be set to request the result in descending order.
Type:      builtin_function_or_method

the key argument of sorted function can be a custom function, in the source, the function is attrgetter('order'), that is get the order attribute of cb object.

Let’s create another example of sorted:

l = {'Joe': 'P1', 'Min': 'P3', 'Reba': 'P2'}

sorted(l, key = lambda x: int(l[x][1]))

['Joe', 'Reba', 'Min']

2 getattr

The source of the course use attrgetter is:

def run_cbs(cbs, method_nm, learn=None):
    for cb in sorted(cbs, key=attrgetter('order')):
        method = getattr(cb, method_nm, None)
        if method is not None: method(learn)

getattr?

Docstring:
getattr(object, name[, default]) -> value
Get a named attribute from an object; getattr(x, 'y') is equivalent to x.y.
When a default argument is given, it is returned when the attribute doesn't
exist; without it, an exception is raised in that case.
Type:      builtin_function_or_method

Ha, This one method is easy.

getattr(p, 'name')

'Joe'

getattr(p, 'age')

AttributeError: 'person' object has no attribute 'age'

getattr(p, 'age', None)

3 Exception

The source code use Exception are:

class CancelFitException(Exception): pass
class CancelBatchException(Exception): pass
class CancelEpochException(Exception): pass

By create a new class inherit Exception class, can create a custom Exception class

4 @contextmanager

from contextlib import contextmanager

contextmanager?

Signature: contextmanager(func)
Docstring:
@contextmanager decorator.
Typical usage:
    @contextmanager
    def some_generator(<arguments>):
        <setup>
        try:
            yield <value>
        finally:
            <cleanup>
This makes this:
    with some_generator(<arguments>) as <variable>:
        <body>
equivalent to this:
    <setup>
    try:
        <variable> = <value>
        <body>
    finally:
        <cleanup>
File:      ~/conda/envs/ml/lib/python3.10/contextlib.py
Type:      function

@contextmanager
def test_func(*args, **kargs):
    print("Before try")
    try:
        print("Exec try, before yield")
        yield
        print("Exec try, after yield")
    except ZeroDivisionError:
        print("except: Divided by zero")
    finally:
        print("Exec finally")

with test_func():
    print("In with 1")
    print("In with 2")

Before try
Exec try, before yield
In with 1
In with 2
Exec try, after yield
Exec finally

4.1 Raise error in with statement?

with test_func():
    print("In with")
    a = 1 / 0
    print("In with 2")

Before try
Exec try, before yield
In with
except: Divided by zero
Exec finally

4.2 Raise error before yield?

@contextmanager
def test_func(*args, **kargs):
    print("Before try")
    try:
        print("Exec try, before yield")
        a = 1 / 0
        yield
        print("Exec try, after yield")
    except ZeroDivisionError:
        print("except: Divided by zero")
    finally:
        print("Exec finally")

with test_func():
    print("In with 1")
    print("In with 2")

Before try
Exec try, before yield
except: Divided by zero
Exec finally

RuntimeError: generator didn't yield

Tip

So contextmanager can’t handle error before yield, as Jeremy have tested in the class.

4.3 Raise error after yield

@contextmanager
def test_func(*args, **kargs):
    print("Before try")
    try:
        print("Exec try, before yield")
        yield
        print("Exec try, after yield")
        a = 1 / 0
    except ZeroDivisionError:
        print("except: Divided by zero")
    finally:
        print("Exec finally")

with test_func():
    print("In with 1")
    print("In with 2")

Before try
Exec try, before yield
In with 1
In with 2
Exec try, after yield
except: Divided by zero
Exec finally

4.4 yield value

@contextmanager
def test_func(*args, **kargs):
    print("Before try")
    try:
        print("Exec try, before yield")
        yield range(3)
        print("Exec try, after yield")
    except ZeroDivisionError:
        print("except: Divided by zero")
    finally:
        print("Exec finally")

with test_func() as f:
    print("In with 1")
    for i in f:
        print(i)
    print("In with 2")

Before try
Exec try, before yield
In with 1
0
1
2
In with 2
Exec try, after yield
Exec finally