diff --git a/.gitignore b/.gitignore index a7c9c10..f42bd22 100644 --- a/.gitignore +++ b/.gitignore @@ -72,13 +72,8 @@ ENV/ env.bak/ venv.bak/ -# mypy -.mypy_cache/ -.dmypy.json -dmypy.json - # Ruff stuff: .ruff_cache/ # PyPI configuration file -.pypirc \ No newline at end of file +.pypirc diff --git a/structural/decorator/Decorator_py.ipynb b/structural/decorator/Decorator_py.ipynb new file mode 100644 index 0000000..6d2d4d5 --- /dev/null +++ b/structural/decorator/Decorator_py.ipynb @@ -0,0 +1,1431 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "07216077", + "metadata": {}, + "source": [ + "# Decorator\n", + "\n", + "## TOC\n", + "\n", + "- [Intro](#intro)\n", + "- [Decorators with parametres](#decorators-with-parametres)\n", + "- [Data in decorators](#data-in-decorators)\n", + "- [Decorating methods](#decorating-methods)\n", + "- [Classes as decorators](#classes-as-decorators)\n", + "- [Decorating classes](#decorating-classes)\n", + "- [Wrapping coroutines](#wrapping-coroutines)\n", + "- [Docstrings of decorated functions](#docstrings-of-decorated-functions)\n", + "- [Wrapping up](#wrapping-up)\n", + "- [Summary](#summary)\n", + "\n", + "### Intro\n", + "\n", + "Suppose there is a *foo* [function](https://docs.python.org/3/glossary.html#term-function) that accepts a string and returns its modified version. Imagine that this function is crucial and neither its signature nor body can be changed. In certain cases, the default formatted message must be different, so the task is \"to enclose the original message in brackets\". Hm, easy-peasy, yet another function that reuses the target (*foo*) function." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "fe7103ce", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "original: string is 'hello'\n", + "modified: [string is 'hello']\n" + ] + } + ], + "source": [ + "def foo(s: str) -> str:\n", + " \"\"\"Returns a fancy string.\n", + "\n", + " Args:\n", + " s (str): original string\n", + "\n", + " Returns:\n", + " str\n", + " \"\"\"\n", + " return f\"string is {s!r}\"\n", + "\n", + "\n", + "def add_brackets_v1(s: str) -> str:\n", + " return f\"[{foo(s)}]\"\n", + "\n", + "\n", + "s = 'hello'\n", + "print(f\"original: {foo(s)}\")\n", + "print(f\"modified: {add_brackets_v1(s)}\")" + ] + }, + { + "cell_type": "markdown", + "id": "55a0d6fa", + "metadata": {}, + "source": [ + "An ugly solution for `add_brackets` depends on *foo*, so it is just a function that can reuse only the *foo* function and that is all. Ok, my bad, `add_brackets` can be less coupled with *foo* and be used with any string." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a71e936c", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "original: string is 'hello'\n", + "modified: [string is 'hello']\n" + ] + } + ], + "source": [ + "def add_brackets_v2(s: str) -> str:\n", + " return f\"[{s}]\"\n", + "\n", + "\n", + "s = 'hello'\n", + "print(f\"original: {foo(s)}\")\n", + "print(f\"modified: {add_brackets_v2(foo(s))}\")" + ] + }, + { + "cell_type": "markdown", + "id": "2a01bf9a", + "metadata": {}, + "source": [ + "Later, the new wish combinations are asked, so now to maintain four cases:\n", + "1. a string in brackets (original -> \"[s]\")\n", + "2. a string in parentheses (\"(s)\")\n", + "3. case 1 and then case 2 (\"([s])\")\n", + "4. case 2 and then case 1 (\"[(s)]\")\n", + "\n", + "Damn, revise the `add_brackets` and define `add_parentheses` and defi...no, their composition is enough (nested calls)!" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "19db9231", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "foo(s): string is 'hello'\n", + "[foo(s)]: [string is 'hello']\n", + "(foo(s)): (string is 'hello')\n", + "[(foo(s))]: [(string is 'hello')]\n", + "([foo(s)]): ([string is 'hello'])\n" + ] + } + ], + "source": [ + "def add_brackets(s: str) -> str:\n", + " return f\"[{s}]\"\n", + "\n", + "\n", + "def add_parentheses(s: str) -> str:\n", + " return f\"({s})\"\n", + "\n", + "\n", + "s = 'hello'\n", + "print(f\"foo(s): {foo(s)}\")\n", + "print(f\"[foo(s)]: {add_brackets(foo(s))}\")\n", + "print(f\"(foo(s)): {add_parentheses(foo(s))}\")\n", + "print(f\"[(foo(s))]: {add_brackets(add_parentheses(foo(s)))}\")\n", + "print(f\"([foo(s)]): {add_parentheses(add_brackets(foo(s)))}\")" + ] + }, + { + "cell_type": "markdown", + "id": "61a091d8", + "metadata": {}, + "source": [ + "It may seem clever, yet the chain of nested functions can get lengthy. If you need this logic more than once:\n", + "1. you forge this chain again or ...\n", + "2. you write a helper function for each case, e.g. and you may get a combinatorial boom, e.g., `parentheses_brackets` for \"([])\" and this paves a road to combinatorial explosion and ...\n", + "3. even if you write one function, you may expect if-else ladder which may smell.\n", + "\n", + "However, the remedy is on the way. This string of invocations can be reduced and, moreover, can be (dis/en)abled dynamically and the [Decorator](https://en.wikipedia.org/wiki/Decorator_pattern) pattern aims for it. With a decorator you can add \"behaviour\" to an individual object dynamically without affecting the behavior of other instances of the same class!\n", + "\n", + "A warm-up example before jumping to decorators." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "83c8480c-2b64-4f9c-afe4-2661487f1285", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "8\n" + ] + } + ], + "source": [ + "def warm(up):\n", + " return up and up\n", + "\n", + "\n", + "print(warm) # jus a callable object\n", + "\n", + "f = warm # that can be referenced by a variable\n", + "\n", + "print(f(8)) # ACKSHUALLY doesn't warm" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "5a8891f5-2e14-4a0c-b350-dbb87a6c6202", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ".inner at 0x72d24ae5e840>\n" + ] + }, + { + "data": { + "text/plain": [ + "0" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def outer(outparam):\n", + " def inner(innparam):\n", + " # outparam is visible here because it is enclosed\n", + " return innparam + outparam\n", + " # outparam is associated with inner even after return\n", + " # see LEGB and closures\n", + " return inner\n", + "\n", + "\n", + "f = outer(5) # f = inner\n", + "print(f)\n", + "\n", + "f(-5)" + ] + }, + { + "cell_type": "markdown", + "id": "f120e684-1f51-4b79-85e2-9413c650129e", + "metadata": {}, + "source": [ + "Now you are ready to get familiar with decorators. A simple decorator in Python is a callable object (we start from functions) that:\n", + "\n", + "1. receives a callable object -> the function of interest;\n", + "2. returns an inner function that reuses the target function and may add new behaviour.\n", + "\n", + "Basic example:\n", + "\n", + "```python\n", + "def decorating_function(callable_to_decorate: Callable) -> Callable:\n", + " def wrapper(*args: Any, **kwargs: Any) -> Any: # so args and kwargs are variadic and can be any -> this is most generic and flexible form\n", + " # some (pre) logic here\n", + " result = callable_to_decorate(*args, **kwargs)\n", + " # some (post) logic here\n", + " # you can return the result if you like\n", + "\n", + " # you return a callable object itself! \n", + " return wrapper\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "053969b4", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "tada\n", + "[string is 'tada']\n", + "(string is 'tada')\n", + "[(string is 'tada')]\n", + "([string is 'tada'])\n" + ] + } + ], + "source": [ + "from collections.abc import Callable\n", + "\n", + "# `bracketize` decorator function\n", + "def bracketize(func: Callable) -> Callable:\n", + " # `func` is local within `bracketize` decorator (!)\n", + " # `func` is enclosed (!) for `wrapper`\n", + " def wrapper(*args, **kwargs):\n", + " return f\"[{func(*args, **kwargs)}]\"\n", + " # return a function that can accept any parametres and produce anything\n", + " # every time `wrapper` is called, `func` is also called (closure)\n", + " return wrapper\n", + "\n", + "\n", + "# `parenthesize` decorator function\n", + "def parenthesize(func: Callable) -> Callable:\n", + " # another decorating function `parenthesize` that suits\n", + " # `_` is a valid name for an actual wrapping function\n", + " # because, again, the decorator pattern is also known as the wrapper pattern\n", + " def _(*args, **kwargs):\n", + " return f\"({func(*args, **kwargs)})\"\n", + " return _\n", + "\n", + "\n", + "bra = bracketize(foo)\n", + "# bra references to the result of `bracketize`\n", + "# which is its inner `wrapper` function\n", + "# that aceepts *params, **kwparams\n", + "# that are given to the wrapped `foo` function\n", + "par = parenthesize(foo)\n", + "\n", + "s1 = \"tada\"\n", + "print(s1)\n", + "\n", + "print(bra(s1)) # case 1 - done\n", + "print(par(s1)) # case 2 - done\n", + "\n", + "bra_par = bracketize(par)\n", + "# par is callable, so `bracketize` can works with it\n", + "# it is the same as `bracketize(parenthesize(foo))`\n", + "par_bra = parenthesize(bra)\n", + "\n", + "print(bra_par(s1))\n", + "print(par_bra(s1))" + ] + }, + { + "cell_type": "markdown", + "id": "ec2ef3b2", + "metadata": {}, + "source": [ + "In Python, [callable](https://docs.python.org/3/glossary.html#term-callable) objects (these are not only functions) can be decorated with `@decorator` [syntactic sugar](https://en.wikipedia.org/wiki/Syntactic_sugar). It follows Python's idiomacy when callable objects are meant to be decorated statically, so the decoration becomes permanent once the function is \"sugared\"." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "8e21df16", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "([2.6457513110645907])\n", + "[(3)]\n" + ] + } + ], + "source": [ + "@parenthesize\n", + "# since the inner wrapper is `wrapper(*args, **kwargs)`\n", + "# the following function can be called without restrictions\n", + "@bracketize\n", + "# the order matters\n", + "def goo(a: int, b: int) -> float:\n", + " return (a + b) ** .5\n", + "# equivalent to `goo = parenthesize(bracketize(goo))`\n", + "\n", + "@bracketize\n", + "@parenthesize\n", + "def hoo(*args) -> int:\n", + " return len(args)\n", + "# equivalent to `hoo = bracketize(parenthesize(hoo))`\n", + "\n", + "print(goo(5, 2))\n", + "print(hoo(1, 4, 8))" + ] + }, + { + "cell_type": "markdown", + "id": "91fd0f34", + "metadata": {}, + "source": [ + "Well, I would like to have the result of `goo` rounded to the 2nd digit after the decimal point. Easy, another decorator." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "18d12283", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "([2.6457513110645907])\n", + "([2.65])\n" + ] + } + ], + "source": [ + "def round_two(cb: Callable) -> Callable:\n", + " def _deco(*args, **kwargs):\n", + " return float(cb(*args, **kwargs))\n", + " return _deco\n", + "\n", + "\n", + "@parenthesize\n", + "@bracketize\n", + "@round_two\n", + "def goo_v2(a: int, b: int) -> float:\n", + " # not reusing goo on purpose -> consider this standalone\n", + " return round((a + b) ** .5, 2)\n", + "\n", + "# goo_v2 = parenthesize(bracketize(round_two(goo_v2)))\n", + "\n", + "\n", + "print(goo(5, 2))\n", + "print(goo_v2(5, 2))" + ] + }, + { + "cell_type": "markdown", + "id": "d5ae34d8", + "metadata": {}, + "source": [ + "The major benefit about decorators is that we don't need to mess with target functions at all! You just add a decorator or as many as you need.\n", + "\n", + "### Decorators with parametres\n", + "\n", + "The main parameter (and argument) for a decorating function is a callable object that needs to be wrapped. What if passing any other parameters is unavoidable? In the following example, specifying extra parametres does not help." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "2bc2af8e", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Greetings, None! -> laddie or lassie\n", + "Greetings, string! -> laddie or lassie\n", + "Hello, 21 -> \n" + ] + } + ], + "source": [ + "from typing import Any\n", + "\n", + "\n", + "# try to remove the default value for `suffix` parametre...aha\n", + "def suffux(func: Callable, suffix: str = \"\") -> Callable:\n", + " def _wrapper(*args, **kwargs):\n", + " return f\"{func(*args, **kwargs)} -> {suffix}\"\n", + " return _wrapper\n", + "\n", + "\n", + "def greeter(obj: Any = None) -> str:\n", + " return f\"Greetings, {obj}!\"\n", + "\n", + "\n", + "greeter1 = suffux(greeter, \"laddie or lassie\")\n", + "print(greeter1()) # it works, `suffix` parametre is enclosed\n", + "print(greeter1(\"string\")) # prints the expected result\n", + "\n", + "\n", + "@suffux\n", + "def greeter2(obj: Any) -> str:\n", + " return f\"Hello, {obj}\"\n", + "\n", + "\n", + "print(greeter2(21))\n", + "# and how to change the suffix for the greeter2? :)\n", + "# with `greeter1 = suffux(greeter, \"laddie or lassie\")` it worked\n", + "# with greeter2 it is misused!" + ] + }, + { + "cell_type": "markdown", + "id": "ee4b5651", + "metadata": {}, + "source": [ + "The situation goes bad if the default argument is not envisaged." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "ba465389", + "metadata": {}, + "outputs": [], + "source": [ + "# dammit!\n", + "def prefix(func: Callable, prefix: str) -> Callable:\n", + " def wrapper(*args, **kwargs):\n", + " result = func(*args, **kwargs)\n", + " return f\"{prefix} -> {result}\"\n", + " return wrapper\n", + "\n", + "\n", + "# Try uncommenting the following cases\n", + "\n", + "\n", + "# No way anymore\n", + "# @prefix\n", + "# def answer() -> int:\n", + "# return 42\n", + "\n", + "\n", + "# Nice try...not at all :)\n", + "# @prefix(prefix=\"no way\")\n", + "# def answer() -> int:\n", + "# return 42\n", + "\n", + "# The same as `answer = prefix(prefix=\"no way\")`...shit!\n", + "# `func` parametre is required!\n", + "# `@prefix(prefix=\"no way\")` is calling the prefix (decorating) function\n", + "# and this function requires a Callable at his time which is not the `answer` function" + ] + }, + { + "cell_type": "markdown", + "id": "a76518c3", + "metadata": {}, + "source": [ + "What to do?! Hints before the following solution:\n", + "1. you can call a decorator function with an argument -> `@deco(param)`;\n", + "2. this `param` can be an enclosed variable in the scope of the `deco` function;\n", + "3. calling `@deco(param)` can return a function that expects a function that wraps ...\n", + "4. ...\n", + "5. maybe PROFIT!" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "d875d88b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "H > 21 < T\n" + ] + } + ], + "source": [ + "def deco(prefix: str = \"prefix\", suffix: str = \"suffix\") -> Callable:\n", + " def decorator(func: Callable) -> Callable:\n", + " def _wrapper(*args, **kwargs):\n", + " result = func(*args, **kwargs)\n", + " return f\"{prefix} > {result} < {suffix}\"\n", + " return _wrapper\n", + " return decorator\n", + " # `deco(\"p\", \"s\")` returns `decorator`\n", + " # that can consume a function,\n", + " # i.e., `decorator(func)` returns a `_wrapper`\n", + " # that is called as `_wrapper(*args, **kwargs)`\n", + " # that invokes the wrapped `func`\n", + "\n", + "\n", + "@deco(\"H\", \"T\")\n", + "def half_answer() -> int:\n", + " return 21\n", + "# equivalent form:\n", + "# half_answer = deco(\"H\", \"T\")(half_answer)\n", + "\n", + "print(half_answer())" + ] + }, + { + "cell_type": "markdown", + "id": "cd0d16d6", + "metadata": {}, + "source": [ + "I guess it looks like PROFIT.\n", + "\n", + "Let's see some more action with multiplicator decorator." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "7312ff65", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "12\n", + "-24\n", + "21.0\n", + "10.0\n" + ] + } + ], + "source": [ + "def multiplier(coef: float = 1) -> Callable:\n", + " def _deco(func: Callable) -> Callable:\n", + " def _(*args, **kwargs):\n", + " return coef * func(*args, **kwargs)\n", + " return _\n", + " return _deco\n", + "\n", + "\n", + "@multiplier()\n", + "def twelve() -> int:\n", + " return 12\n", + "\n", + "\n", + "@multiplier(-2)\n", + "def twenty_one() -> int:\n", + " return 12\n", + "\n", + "\n", + "@multiplier(.5)\n", + "def fourty_two() -> int:\n", + " return 42\n", + "\n", + "\n", + "print(twelve()) # yes\n", + "print(twenty_one()) # not exactly -> -42\n", + "print(fourty_two()) # twisted -> 21.0\n", + "\n", + "\n", + "# even like this\n", + "invariant = multiplier(-.5)(multiplier(-2)(lambda: 10))\n", + "print(invariant())" + ] + }, + { + "cell_type": "markdown", + "id": "1dacaf52", + "metadata": {}, + "source": [ + "### Data in decorators\n", + "\n", + "There are situation when decorators should have data stored under the hood. If these data are mutable, risks of changing them unexpectedly are viable. Consider the example on a decorator that tracks function call count and also has a small cache for return results in order to add some performance via [memoisation](https://en.wikipedia.org/wiki/Memoization) technique." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "a4a66aa9", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(3, 1)" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def memoize(func: Callable) -> Callable:\n", + " def _wrapper(*args, **kwargs):\n", + " # this implementation is not saviour\n", + " cache: dict[str, Any] = {}\n", + " kws = {k:kwargs[k] for k in sorted(kwargs)}\n", + " if (params := f\"{sorted(args)}:{kws}\") in cache:\n", + " print(f\"{func.__name__}: result from cache\")\n", + " return cache[params]\n", + " res = func(*args, **kwargs)\n", + " cache[params] = res\n", + " return res\n", + " return _wrapper\n", + "\n", + "\n", + "@memoize\n", + "def divide(a: int, b: int) -> tuple[int, int]:\n", + " return divmod(a, b)\n", + "\n", + "\n", + "divide(13, 4)\n", + "divide(13, 4)" + ] + }, + { + "cell_type": "markdown", + "id": "4ee1ebd8", + "metadata": {}, + "source": [ + "This works, but not as expected. Every time the wrapped function is called, the _cache_ local varaible is initialised. When the *_wrapper* scope is left, the cache is gone. Let's move the cache instantiation out of the *_wrapper* function." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "e8882066", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "divide: result from cache\n", + "answer: result from cache\n" + ] + }, + { + "data": { + "text/plain": [ + "42" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "def memoize(func: Callable) -> Callable:\n", + " cache: dict[str, Any] = {}\n", + " def _wrapper(*args, **kwargs):\n", + " kws = {k:kwargs[k] for k in sorted(kwargs)}\n", + " if (params := f\"{sorted(args)}:{kws}\") in cache:\n", + " print(f\"{func.__name__}: result from cache\")\n", + " return cache[params]\n", + " res = func(*args, **kwargs)\n", + " cache[params] = res\n", + " return res\n", + " return _wrapper\n", + "\n", + "\n", + "@memoize\n", + "def divide(a: int, b: int) -> tuple[int, int]:\n", + " return divmod(a, b)\n", + "\n", + "\n", + "divide(13, 4)\n", + "divide(13, 4) # from cache\n", + "\n", + "### looks good\n", + "\n", + "@memoize\n", + "def answer() -> int:\n", + " return 42\n", + "\n", + "answer()\n", + "divide(0, 6)\n", + "answer() # from cache" + ] + }, + { + "cell_type": "markdown", + "id": "df545eec", + "metadata": {}, + "source": [ + "Now it is better. Every time a function (callable object), shall we say *foo*, is wrapped with the *memoize* decorator, the *cache* is initialised at the time of `foo = memoize(foo)` (or `@memoize` sugar) andv it stays well defined and managed for a particular wrapped function. You can guess what may go wrong if a cache would have been defined globally outside the decorator. The example above is easy to change to see the illustrative effect.\n", + "\n", + "### Decorating methods\n", + "\n", + "[Method](https://docs.python.org/3/glossary.html#term-method)s are callable objects associated with a class. The first parametre/argument for a method is either a reference to an instance of this class (*self*) or a class (*cls*). The next example is OK regardless TypeError." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "9f97b2a0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "result = 10\n" + ] + }, + { + "ename": "TypeError", + "evalue": "decoco.._() takes 2 positional arguments but 3 were given", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mTypeError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[15]\u001b[39m\u001b[32m, line 21\u001b[39m\n\u001b[32m 16\u001b[39m \u001b[38;5;129m@decoco\u001b[39m \u001b[38;5;66;03m# signature mismatch\u001b[39;00m\n\u001b[32m 17\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[34madd\u001b[39m(\u001b[38;5;28mself\u001b[39m, x: \u001b[38;5;28mint\u001b[39m, y: \u001b[38;5;28mint\u001b[39m) -> \u001b[38;5;28mint\u001b[39m:\n\u001b[32m 18\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m x + y\n\u001b[32m---> \u001b[39m\u001b[32m21\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[43mAdder\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m.\u001b[49m\u001b[43madd\u001b[49m\u001b[43m(\u001b[49m\u001b[32;43m3\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[32;43m7\u001b[39;49m\u001b[43m)\u001b[49m)\n", + "\u001b[31mTypeError\u001b[39m: decoco.._() takes 2 positional arguments but 3 were given" + ] + } + ], + "source": [ + "def decoco(func: Callable[[int, int], int]) -> Callable[[int, int], str]:\n", + " def _(a: int, b: int) -> str:\n", + " return f\"result = {func(a, b)}\"\n", + " return _\n", + "\n", + "\n", + "@decoco\n", + "def add(x: int, y: int) -> int:\n", + " return x + y\n", + "\n", + "\n", + "print(add(3, 7)) # OK\n", + "\n", + "\n", + "class Adder:\n", + " @decoco # signature mismatch\n", + " def add(self, x: int, y: int) -> int:\n", + " return x + y\n", + "\n", + "\n", + "print(Adder().add(3, 7))" + ] + }, + { + "cell_type": "markdown", + "id": "6095ccef", + "metadata": {}, + "source": [ + "The first argument is for self or cls object, so the first argument is associated with the self|cls parametre and the second argument is bound to the *x* parametre, so the *y* parametre is not given, but required. Let's try with variadic params and keyword params." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "8564e28b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "result = 10\n", + "result = 10\n" + ] + } + ], + "source": [ + "def decoqo(func: Callable[..., int]) -> Callable[..., str]:\n", + " def _(*params, **kwparams) -> str:\n", + " return f\"result = {func(*params, **kwparams)}\"\n", + " return _\n", + "\n", + "\n", + "@decoqo\n", + "def add(x: int, y: int) -> int:\n", + " return x + y\n", + "\n", + "\n", + "print(add(3, 7)) # OK\n", + "\n", + "\n", + "class Adder:\n", + " @decoqo # whatever signature\n", + " def add(self, x: int, y: int) -> int:\n", + " return x + y\n", + "\n", + "# TypeError: decoco.._() takes 2 positional arguments but 3 were given\n", + "print(Adder().add(3, 7))" + ] + }, + { + "cell_type": "markdown", + "id": "a4ceb35b", + "metadata": {}, + "source": [ + "### Classes as decorators\n", + "\n", + "Not only functions can decorate functions/methods. Again, a decorator is a callable object that accepts another callable object. An object is callable if it can be used with parentheses where zero or more (kw)params can be placed, i.e., `obj(*args, kwargs)`. An instance of a class is callable if it defines the [\\_\\_call\\_\\_()](https://docs.python.org/3/reference/datamodel.html#object.__call__) dunder. A callable object is tested via the [callable](https://docs.python.org/3/library/functions.html#callable) builtin." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "4830a8e4", + "metadata": {}, + "outputs": [], + "source": [ + "def sumargs(*args, **kwargs):\n", + " return sum([len(args), len(kwargs)])\n", + "\n", + "assert callable(sumargs)\n", + "# foo(...) is foo.__call__(...)\n", + "assert sumargs(1, 2, a='b') == sumargs.__call__(1, 2, a='b')\n", + "\n", + "\n", + "class Dummy:\n", + " def __init__(self) -> None:\n", + " self.c = []\n", + "\n", + "\n", + "assert not callable(Dummy()) # no __call__ dunder\n", + "\n", + "\n", + "class DummyCallable:\n", + " def __init__(self) -> None:\n", + " self._c: list = []\n", + "\n", + " def items(self) -> list:\n", + " return self._c[:]\n", + "\n", + " def __call__(self, *args: Any) -> None:\n", + " self._c.extend(args)\n", + "\n", + "\n", + "dc = DummyCallable()\n", + "assert callable(dc)\n", + "\n", + "dc(1, 2)\n", + "dc.__call__(-4, 3)\n", + "\n", + "assert dc.items() == [1, 2, -4, 3]" + ] + }, + { + "cell_type": "markdown", + "id": "6d36e7bb", + "metadata": {}, + "source": [ + "Time for a class that is callable and designed to decorate other callables." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "512b233a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "16\n", + "9\n" + ] + }, + { + "ename": "ValueError", + "evalue": "call limit exceeded", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mValueError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[18]\u001b[39m\u001b[32m, line 29\u001b[39m\n\u001b[32m 27\u001b[39m \u001b[38;5;28mprint\u001b[39m(power(\u001b[32m2\u001b[39m, \u001b[32m4\u001b[39m))\n\u001b[32m 28\u001b[39m \u001b[38;5;28mprint\u001b[39m(power(\u001b[32m3\u001b[39m, \u001b[32m2\u001b[39m))\n\u001b[32m---> \u001b[39m\u001b[32m29\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[43mpower\u001b[49m\u001b[43m(\u001b[49m\u001b[32;43m1\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[32;43m1\u001b[39;49m\u001b[43m)\u001b[49m)\n", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[18]\u001b[39m\u001b[32m, line 10\u001b[39m, in \u001b[36mDecorator._wrapper\u001b[39m\u001b[34m(self, *args, **kwargs)\u001b[39m\n\u001b[32m 8\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[34m_wrapper\u001b[39m(\u001b[38;5;28mself\u001b[39m, *args, **kwargs):\n\u001b[32m 9\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m._limit:\n\u001b[32m---> \u001b[39m\u001b[32m10\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[33m\"\u001b[39m\u001b[33mcall limit exceeded\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 11\u001b[39m res = \u001b[38;5;28mself\u001b[39m._func(*args, **kwargs)\n\u001b[32m 12\u001b[39m \u001b[38;5;28mself\u001b[39m._limit -= \u001b[32m1\u001b[39m\n", + "\u001b[31mValueError\u001b[39m: call limit exceeded" + ] + } + ], + "source": [ + "class Decorator:\n", + " def __init__(self, limit: int = 5) -> None:\n", + " if limit < 0:\n", + " raise ValueError(\"limit < 0\")\n", + " self._limit = limit\n", + " self._func: Callable # must be defined, e.g. in the __call__\n", + "\n", + " def _wrapper(self, *args, **kwargs):\n", + " if not self._limit:\n", + " raise ValueError(\"call limit exceeded\")\n", + " res = self._func(*args, **kwargs)\n", + " self._limit -= 1\n", + " return res\n", + "\n", + " def __call__(self, func: Callable) -> Any:\n", + " # or you can define a wrapper here as an ordinary function\n", + " self._func = func\n", + " return self._wrapper\n", + "\n", + "\n", + "@Decorator(limit=2)\n", + "def power(base: float, exp: float) -> float:\n", + " return base ** exp\n", + "\n", + "# power = Decorator(limit=2)(power)\n", + "\n", + "print(power(2, 4))\n", + "print(power(3, 2))\n", + "print(power(1, 1))" + ] + }, + { + "cell_type": "markdown", + "id": "aabf806a", + "metadata": {}, + "source": [ + "### Decorating classes\n", + "\n", + "We know that functions are definitely callable objects, but what about wrapping classes? Can we decorate them and if so, is there more than one way to do so? Let's just try." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "3a799f14", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Dummy -> Dummy(5)\n", + "\n", + "scaled by -2: -10\n", + "number = 5\n" + ] + } + ], + "source": [ + "def plog(func: Callable) -> Callable:\n", + " def _(*args: Any, **kwargs: Any) -> Any:\n", + " res = func(*args, **kwargs)\n", + " print(f\"{func.__name__} -> {res}\")\n", + " return res\n", + " return _\n", + "\n", + "\n", + "@plog\n", + "class Dummy:\n", + " def __init__(self, number: float) -> None:\n", + " self._nbr = number\n", + "\n", + " def __repr__(self) -> str:\n", + " return f\"{type(self).__name__}({self._nbr})\"\n", + "\n", + " @property\n", + " def number(self) -> float:\n", + " return self._nbr\n", + "\n", + " def scale(self, coef: float) -> float:\n", + " return self.number * coef\n", + "\n", + "dummy = Dummy(5) # only here the decorator works\n", + "print()\n", + "print(f\"scaled by -2: {dummy.scale(-2)}\") # no effect\n", + "print(f\"number = {dummy.number}\") # no plogging is done" + ] + }, + { + "cell_type": "markdown", + "id": "416236f9", + "metadata": {}, + "source": [ + "So, the decorating logic works only when the class is instantiated and is not in effect for other methods even dunders.\n", + "\n", + "In fact, the classes are callable. From the [docs](https://docs.python.org/3/reference/datamodel.html#classes):\n", + "> Classes are callable. These objects normally act as factories for new instances of themselves, but variations are possible for class types that override [\\_\\_new\\_\\_()](https://docs.python.org/3/reference/datamodel.html#object.__new__). The arguments of the call are passed to \\_\\_new\\_\\_() and, in the typical case, to [\\_\\_init\\_\\_()](https://docs.python.org/3/reference/datamodel.html#object.__init__) to initialize the new instance.\n", + "\n", + "A [quote](https://docs.python.org/3/reference/datamodel.html#class-instances) about class instances:\n", + "> Instances of arbitrary classes can be made callable by defining a \\_\\_call\\_\\_() method in their class.\n", + "\n", + "So, wrappping a class is like wrapping its \\_\\_init\\_\\_() dunder. It makes sense because calling a class means to create and return its instance.\n", + "\n", + "### Wrapping coroutines\n", + "\n", + "Coroutines are easy to wrap yet some points to remember:\n", + "- `await` inside a sync `def` function is SyntaxError\n", + "- so, a wrapper should be an `async def` function" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "d08b8509", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "before \n", + "I am just coroutine with ()\n", + "after -> None\n" + ] + } + ], + "source": [ + "def wrap_coro(coro: Callable) -> Callable:\n", + " async def _wrapper(*args, **kwargs):\n", + " print(f\"before {coro}\")\n", + " result = await coro(*args, **kwargs)\n", + " print(f\"after {coro} -> {result}\")\n", + " return result\n", + " return _wrapper\n", + "\n", + "\n", + "@wrap_coro\n", + "async def coro(*args):\n", + " print(f\"I am just coroutine with {args}\")\n", + "\n", + "\n", + "await coro()" + ] + }, + { + "cell_type": "markdown", + "id": "20e0f750", + "metadata": {}, + "source": [ + "One decorator for wrapping either a routine or a coroutine? It is doable and [this solution](https://stackoverflow.com/questions/44169998/how-to-create-a-python-decorator-that-can-wrap-either-coroutine-or-function) looks good to me yet it may be a step to overengineering." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "3185b288", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Hello\n", + "Time to sync\n", + "Bye\n", + "Wrapping a coroutine\n", + "Hello\n", + "Our sync is run async\n", + "Bye\n" + ] + } + ], + "source": [ + "import inspect\n", + "from contextlib import contextmanager\n", + "\n", + "\n", + "def powerful_decorator(func):\n", + " @contextmanager\n", + " def wrapping_logic():\n", + " print(\"Hello\")\n", + " yield\n", + " print(\"Bye\")\n", + "\n", + " def wrapper(*args, **kwargs):\n", + " if not inspect.iscoroutinefunction(func):\n", + " with wrapping_logic():\n", + " return func(*args, **kwargs)\n", + " print(\"Wrapping a coroutine\")\n", + " async def tmp():\n", + " with wrapping_logic():\n", + " return (await func(*args, **kwargs))\n", + " return tmp()\n", + "\n", + " return wrapper\n", + "\n", + "\n", + "@powerful_decorator\n", + "def synco():\n", + " print(\"Time to sync\")\n", + "\n", + "\n", + "@powerful_decorator\n", + "async def asynco():\n", + " print(\"Our sync is run async\")\n", + "\n", + "\n", + "synco()\n", + "await asynco()" + ] + }, + { + "cell_type": "markdown", + "id": "42d26fc7", + "metadata": {}, + "source": [ + "### Docstrings of decorated functions\n", + "\n", + "So far the things go so good. Writing decorators is fun, we can decorate functions, methods, classes yet they were undocumented and documenting public entities is what should be done for maintainable code's sake. So, what can go wrong if a docstring is just a docstring and not a big deal to add it." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "2dd9d926", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Perform `a / b` logic.\n", + "\n", + " Args:\n", + " a (float): dividend\n", + " b (float): divisor\n", + "\n", + " Raises:\n", + " ZeroDivisionError\n", + "\n", + " Returns:\n", + " int\n", + " \n", + "None\n" + ] + } + ], + "source": [ + "def docstring_perdu(cb: Callable) -> Callable:\n", + " def _wrapper(*args, **kwargs):\n", + " return cb(*args, **kwargs)\n", + " return _wrapper\n", + "\n", + "\n", + "def divide_without_conquering(a: float, b: float) -> float:\n", + " \"\"\"Perform `a / b` logic.\n", + "\n", + " Args:\n", + " a (float): dividend\n", + " b (float): divisor\n", + "\n", + " Raises:\n", + " ZeroDivisionError\n", + "\n", + " Returns:\n", + " int\n", + " \"\"\"\n", + " return a / b\n", + "\n", + "\n", + "print(divide_without_conquering.__doc__) # OK\n", + "\n", + "decorated = docstring_perdu(divide_without_conquering)\n", + "print(decorated.__doc__) # What ?!" + ] + }, + { + "cell_type": "markdown", + "id": "8636123e", + "metadata": {}, + "source": [ + "Docstring is lost because *_wrapper* does not have it. The idea of specifying a docstring in the *_wrapper* is of no use:\n", + "\n", + "- the wrapped object must not be stripped of its docstring, obviously;\n", + "- docstring in the *_wrapper* is duplication;\n", + "- what will happen if the same decorator wraps different functions with different docstring?\n", + "\n", + "Gladly, thre is much more fancy solution -> [functools.wraps](https://docs.python.org/3/library/functools.html).\n" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "e2cd29da", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Perform `a / b` logic.\n", + "\n", + " Args:\n", + " a (float): dividend\n", + " b (float): divisor\n", + "\n", + " Raises:\n", + " ZeroDivisionError\n", + "\n", + " Returns:\n", + " int\n", + " \n" + ] + } + ], + "source": [ + "import functools\n", + "\n", + "def docstring_saved(cb: Callable) -> Callable:\n", + " @functools.wraps(cb) # accepts the target cb and wraps the wrapper\n", + " def _wrapper(*args, **kwargs):\n", + " return cb(*args, **kwargs)\n", + " return _wrapper\n", + "\n", + "\n", + "@docstring_saved\n", + "def divide_without_conquering(a: float, b: float) -> float:\n", + " \"\"\"Perform `a / b` logic.\n", + "\n", + " Args:\n", + " a (float): dividend\n", + " b (float): divisor\n", + "\n", + " Raises:\n", + " ZeroDivisionError\n", + "\n", + " Returns:\n", + " int\n", + " \"\"\"\n", + " return a / b\n", + "\n", + "\n", + "print(divide_without_conquering.__doc__) # OK" + ] + }, + { + "cell_type": "markdown", + "id": "ea32adfa-9fc7-4a95-a78f-1c70408fbc99", + "metadata": {}, + "source": [ + "Nicely done.\n", + "\n", + "### Wrapping up\n", + "\n", + "Let's implement a simple [Retry](https://www.geeksforgeeks.org/system-design/retry-pattern-in-microservices/) strategy to handle temporary/occasional failures related to I/O operations. Imagine there is an async `get_data` (coroutine) function that requests data from a remote source. If a request fails because of some connnection issues, the function is reinvoked until it reaches certain limits.\n", + "\n", + "We understand that:\n", + "- the number of retries is supposed to be limited;\n", + "- the certain exceptions should be intercepted, not all and blindly;\n", + "- the delay between attempts is desirable for preventing overwhelming the system with retries.\n", + "\n", + "The `retry` decorator will handle the case yet it will be kept as simple as possible and avoid the [Retry Storm situation](https://dev.to/willvelida/the-retry-pattern-and-retry-storm-anti-pattern-4k6k)." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "id": "16cad344-5f8b-4475-a7c7-6fae49957cbd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "get_data(args=('URL-1',), kwargs={})\n", + "get_data(args=('URL-2',), kwargs={})\n", + "get_data(args=('URL-3',), kwargs={})\n", + "get_data(args=('URL-4',), kwargs={})\n", + "get_data(args=('URL-5',), kwargs={})\n", + "get_data(args=('URL-6',), kwargs={})\n", + "get_data(args=('URL-7',), kwargs={})\n", + "get_data(args=('URL-8',), kwargs={})\n", + "get_data(args=('URL-9',), kwargs={})\n", + "get_data(args=('URL-10',), kwargs={})\n", + "request failed for get_data(args=('URL-10',), kwargs={}): I'm tired\n", + "400\n", + "200\n", + "request failed for get_data(args=('URL-2',), kwargs={}): I'm tired\n", + "get_data(args=('URL-10',), kwargs={})\n", + "get_data(args=('URL-2',), kwargs={})\n", + "400\n", + "[Exception('unexpected'), {'URL-2': 400}, Exception('unexpected'), {'URL-4': 200}, Exception('unexpected'), {'URL-6': 400}, Exception('unexpected'), Exception('unexpected'), Exception('unexpected'), Exception('unexpected')]\n" + ] + } + ], + "source": [ + "import asyncio\n", + "import functools\n", + "import random\n", + "\n", + "from collections.abc import Iterable\n", + "\n", + "\n", + "_CODES = [200, 201, 400, 500, 502]\n", + "\n", + "\n", + "def _respond() -> int:\n", + " if random.randint(0, 2) == 2:\n", + " raise Exception(\"unexpected\")\n", + " if (code := random.choice(_CODES)) == 500:\n", + " raise ConnectionError(\"I'm tired\")\n", + " print(code)\n", + " return code\n", + "\n", + "\n", + "async def get_data(url: str) -> dict[str, Any]:\n", + " # simulate I/O\n", + " await asyncio.sleep(random.random())\n", + " return {f\"{url}\": _respond()}\n", + "\n", + "\n", + "def retry(times: int = 3, *, backoff: float = 1, exceptions: Iterable[Exception] = (Exception,)) -> Callable:\n", + " def _deco(func: Callable) -> Callable:\n", + " nonlocal exc\n", + " @functools.wraps(func)\n", + " async def _wrapper(*args: Any, **kwargs: Any) -> Any:\n", + " fdm = f\"{func.__name__}(args={args}, kwargs={kwargs})\"\n", + " for attempt in range(1, times + 1):\n", + " try:\n", + " print(fdm)\n", + " return await func(*args, **kwargs)\n", + " except exc as e:\n", + " print(f\"request failed for {fdm}: {e}\")\n", + " if attempt == times:\n", + " print(\"No chance\")\n", + " raise\n", + " jitter = random.random()\n", + " # here the backoff strategy is simple (no exponential rate)\n", + " # yet with jitter -> random addendum between attempts\n", + " await asyncio.sleep(max(1, backoff + jitter))\n", + " return _wrapper\n", + " exc = tuple(exceptions)\n", + " return _deco\n", + "\n", + "\n", + "async def main() -> None:\n", + " f = retry(exceptions=[ConnectionError])(get_data)\n", + " tasks: list[asyncio.Task] = []\n", + " for i in range(1, 11):\n", + " tasks.append(asyncio.create_task(f(f\"URL-{i}\")))\n", + " results = await asyncio.gather(*tasks, return_exceptions=True)\n", + " print(results)\n", + " # not interested in results here\n", + "\n", + "\n", + "await main()" + ] + }, + { + "cell_type": "markdown", + "id": "67f91702", + "metadata": {}, + "source": [ + "Easy peasy lemon squeezy.\n", + "\n", + "### Summary\n", + "\n", + "Honestly, I thought that [Decorator](https://en.wikipedia.org/wiki/Decorator_pattern), also referred to as Wrapper, is a *behavioural* design pattern for we can change behaviour of a decorated function or class and stuff. Yet this pattern is **structural** and structural patterns focus on the composition and (re)structuring that is why such name. The fact that Decorator/Wrapper is structural has grounded reasons because:\n", + "\n", + "1. The decorated callable does not change neither in structure not behaviour.\n", + "2. A wrapper over it uses the wrappee as a component which means composition (preferred to inheritance).\n", + "3. The resultant object expresses the relationship between the decorator and wrappee which is modified structure.\n", + "\n", + "That is all for now." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}