/usr/share/doc/python3-docs/html/_sources/library
Edit: /usr/share/doc/python3-docs/html/_sources/library/functools.rst.txt (18503B)
:mod:`functools` --- Higher-order functions and operations on callable objects
==============================================================================
.. module:: functools
:synopsis: Higher-order functions and operations on callable objects.
.. moduleauthor:: Peter Harris
.. moduleauthor:: Raymond Hettinger
.. moduleauthor:: Nick Coghlan
.. moduleauthor:: Łukasz Langa
.. sectionauthor:: Peter Harris
**Source code:** :source:`Lib/functools.py`
--------------
The :mod:`functools` module is for higher-order functions: functions that act on
or return other functions. In general, any callable object can be treated as a
function for the purposes of this module.
The :mod:`functools` module defines the following functions:
.. function:: cmp_to_key(func)
Transform an old-style comparison function to a :term:`key function`. Used
with tools that accept key functions (such as :func:`sorted`, :func:`min`,
:func:`max`, :func:`heapq.nlargest`, :func:`heapq.nsmallest`,
:func:`itertools.groupby`). This function is primarily used as a transition
tool for programs being converted from Python 2 which supported the use of
comparison functions.
A comparison function is any callable that accept two arguments, compares them,
and returns a negative number for less-than, zero for equality, or a positive
number for greater-than. A key function is a callable that accepts one
argument and returns another value to be used as the sort key.
Example::
sorted(iterable, key=cmp_to_key(locale.strcoll)) # locale-aware sort order
For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`.
.. versionadded:: 3.2
.. decorator:: lru_cache(maxsize=128, typed=False)
Decorator to wrap a function with a memoizing callable that saves up to the
*maxsize* most recent calls. It can save time when an expensive or I/O bound
function is periodically called with the same arguments.
Since a dictionary is used to cache results, the positional and keyword
arguments to the function must be hashable.
If *maxsize* is set to ``None``, the LRU feature is disabled and the cache can
grow without bound. The LRU feature performs best when *maxsize* is a
power-of-two.
If *typed* is set to true, function arguments of different types will be
cached separately. For example, ``f(3)`` and ``f(3.0)`` will be treated
as distinct calls with distinct results.
To help measure the effectiveness of the cache and tune the *maxsize*
parameter, the wrapped function is instrumented with a :func:`cache_info`
function that returns a :term:`named tuple` showing *hits*, *misses*,
*maxsize* and *currsize*. In a multi-threaded environment, the hits
and misses are approximate.
The decorator also provides a :func:`cache_clear` function for clearing or
invalidating the cache.
The original underlying function is accessible through the
:attr:`__wrapped__` attribute. This is useful for introspection, for
bypassing the cache, or for rewrapping the function with a different cache.
An `LRU (least recently used) cache
`_ works
best when the most recent calls are the best predictors of upcoming calls (for
example, the most popular articles on a news server tend to change each day).
The cache's size limit assures that the cache does not grow without bound on
long-running processes such as web servers.
Example of an LRU cache for static web content::
@lru_cache(maxsize=32)
def get_pep(num):
'Retrieve text of a Python Enhancement Proposal'
resource = 'http://www.python.org/dev/peps/pep-%04d/' % num
try:
with urllib.request.urlopen(resource) as s:
return s.read()
except urllib.error.HTTPError:
return 'Not Found'
>>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991:
... pep = get_pep(n)
... print(n, len(pep))
>>> get_pep.cache_info()
CacheInfo(hits=3, misses=8, maxsize=32, currsize=8)
Example of efficiently computing
`Fibonacci numbers `_
using a cache to implement a
`dynamic programming `_
technique::
@lru_cache(maxsize=None)
def fib(n):
if n < 2:
return n
return fib(n-1) + fib(n-2)
>>> [fib(n) for n in range(16)]
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610]
>>> fib.cache_info()
CacheInfo(hits=28, misses=16, maxsize=None, currsize=16)
.. versionadded:: 3.2
.. versionchanged:: 3.3
Added the *typed* option.
.. decorator:: total_ordering
Given a class defining one or more rich comparison ordering methods, this
class decorator supplies the rest. This simplifies the effort involved
in specifying all of the possible rich comparison operations:
The class must define one of :meth:`__lt__`, :meth:`__le__`,
:meth:`__gt__`, or :meth:`__ge__`.
In addition, the class should supply an :meth:`__eq__` method.
For example::
@total_ordering
class Student:
def _is_valid_operand(self, other):
return (hasattr(other, "lastname") and
hasattr(other, "firstname"))
def __eq__(self, other):
if not self._is_valid_operand(other):
return NotImplemented
return ((self.lastname.lower(), self.firstname.lower()) ==
(other.lastname.lower(), other.firstname.lower()))
def __lt__(self, other):
if not self._is_valid_operand(other):
return NotImplemented
return ((self.lastname.lower(), self.firstname.lower()) <
(other.lastname.lower(), other.firstname.lower()))
.. note::
While this decorator makes it easy to create well behaved totally
ordered types, it *does* come at the cost of slower execution and
more complex stack traces for the derived comparison methods. If
performance benchmarking indicates this is a bottleneck for a given
application, implementing all six rich comparison methods instead is
likely to provide an easy speed boost.
.. versionadded:: 3.2
.. versionchanged:: 3.4
Returning NotImplemented from the underlying comparison function for
unrecognised types is now supported.
.. function:: partial(func, *args, **keywords)
Return a new :class:`partial` object which when called will behave like *func*
called with the positional arguments *args* and keyword arguments *keywords*. If
more arguments are supplied to the call, they are appended to *args*. If
additional keyword arguments are supplied, they extend and override *keywords*.
Roughly equivalent to::
def partial(func, *args, **keywords):
def newfunc(*fargs, **fkeywords):
newkeywords = keywords.copy()
newkeywords.update(fkeywords)
return func(*args, *fargs, **newkeywords)
newfunc.func = func
newfunc.args = args
newfunc.keywords = keywords
return newfunc
The :func:`partial` is used for partial function application which "freezes"
some portion of a function's arguments and/or keywords resulting in a new object
with a simplified signature. For example, :func:`partial` can be used to create
a callable that behaves like the :func:`int` function where the *base* argument
defaults to two:
>>> from functools import partial
>>> basetwo = partial(int, base=2)
>>> basetwo.__doc__ = 'Convert base 2 string to an int.'
>>> basetwo('10010')
18
.. class:: partialmethod(func, *args, **keywords)
Return a new :class:`partialmethod` descriptor which behaves
like :class:`partial` except that it is designed to be used as a method
definition rather than being directly callable.
*func* must be a :term:`descriptor` or a callable (objects which are both,
like normal functions, are handled as descriptors).
When *func* is a descriptor (such as a normal Python function,
:func:`classmethod`, :func:`staticmethod`, :func:`abstractmethod` or
another instance of :class:`partialmethod`), calls to ``__get__`` are
delegated to the underlying descriptor, and an appropriate
:class:`partial` object returned as the result.
When *func* is a non-descriptor callable, an appropriate bound method is
created dynamically. This behaves like a normal Python function when
used as a method: the *self* argument will be inserted as the first
positional argument, even before the *args* and *keywords* supplied to
the :class:`partialmethod` constructor.
Example::
>>> class Cell(object):
... def __init__(self):
... self._alive = False
... @property
... def alive(self):
... return self._alive
... def set_state(self, state):
... self._alive = bool(state)
... set_alive = partialmethod(set_state, True)
... set_dead = partialmethod(set_state, False)
...
>>> c = Cell()
>>> c.alive
False
>>> c.set_alive()
>>> c.alive
True
.. versionadded:: 3.4
.. function:: reduce(function, iterable[, initializer])
Apply *function* of two arguments cumulatively to the items of *sequence*, from
left to right, so as to reduce the sequence to a single value. For example,
``reduce(lambda x, y: x+y, [1, 2, 3, 4, 5])`` calculates ``((((1+2)+3)+4)+5)``.
The left argument, *x*, is the accumulated value and the right argument, *y*, is
the update value from the *sequence*. If the optional *initializer* is present,
it is placed before the items of the sequence in the calculation, and serves as
a default when the sequence is empty. If *initializer* is not given and
*sequence* contains only one item, the first item is returned.
Roughly equivalent to::
def reduce(function, iterable, initializer=None):
it = iter(iterable)
if initializer is None:
value = next(it)
else:
value = initializer
for element in it:
value = function(value, element)
return value
.. decorator:: singledispatch
Transform a function into a :term:`single-dispatch ` :term:`generic function`.
To define a generic function, decorate it with the ``@singledispatch``
decorator. Note that the dispatch happens on the type of the first argument,
create your function accordingly::
>>> from functools import singledispatch
>>> @singledispatch
... def fun(arg, verbose=False):
... if verbose:
... print("Let me just say,", end=" ")
... print(arg)
To add overloaded implementations to the function, use the :func:`register`
attribute of the generic function. It is a decorator, taking a type
parameter and decorating a function implementing the operation for that
type::
>>> @fun.register(int)
... def _(arg, verbose=False):
... if verbose:
... print("Strength in numbers, eh?", end=" ")
... print(arg)
...
>>> @fun.register(list)
... def _(arg, verbose=False):
... if verbose:
... print("Enumerate this:")
... for i, elem in enumerate(arg):
... print(i, elem)
To enable registering lambdas and pre-existing functions, the
:func:`register` attribute can be used in a functional form::
>>> def nothing(arg, verbose=False):
... print("Nothing.")
...
>>> fun.register(type(None), nothing)
The :func:`register` attribute returns the undecorated function which
enables decorator stacking, pickling, as well as creating unit tests for
each variant independently::
>>> @fun.register(float)
... @fun.register(Decimal)
... def fun_num(arg, verbose=False):
... if verbose:
... print("Half of your number:", end=" ")
... print(arg / 2)
...
>>> fun_num is fun
False
When called, the generic function dispatches on the type of the first
argument::
>>> fun("Hello, world.")
Hello, world.
>>> fun("test.", verbose=True)
Let me just say, test.
>>> fun(42, verbose=True)
Strength in numbers, eh? 42
>>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
Enumerate this:
0 spam
1 spam
2 eggs
3 spam
>>> fun(None)
Nothing.
>>> fun(1.23)
0.615
Where there is no registered implementation for a specific type, its
method resolution order is used to find a more generic implementation.
The original function decorated with ``@singledispatch`` is registered
for the base ``object`` type, which means it is used if no better
implementation is found.
To check which implementation will the generic function choose for
a given type, use the ``dispatch()`` attribute::
>>> fun.dispatch(float)
>>> fun.dispatch(dict) # note: default implementation
To access all registered implementations, use the read-only ``registry``
attribute::
>>> fun.registry.keys()
dict_keys([, , ,
, ,
])
>>> fun.registry[float]
>>> fun.registry[object]
.. versionadded:: 3.4
.. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
Update a *wrapper* function to look like the *wrapped* function. The optional
arguments are tuples to specify which attributes of the original function are
assigned directly to the matching attributes on the wrapper function and which
attributes of the wrapper function are updated with the corresponding attributes
from the original function. The default values for these arguments are the
module level constants ``WRAPPER_ASSIGNMENTS`` (which assigns to the wrapper
function's ``__module__``, ``__name__``, ``__qualname__``, ``__annotations__``
and ``__doc__``, the documentation string) and ``WRAPPER_UPDATES`` (which
updates the wrapper function's ``__dict__``, i.e. the instance dictionary).
To allow access to the original function for introspection and other purposes
(e.g. bypassing a caching decorator such as :func:`lru_cache`), this function
automatically adds a ``__wrapped__`` attribute to the wrapper that refers to
the function being wrapped.
The main intended use for this function is in :term:`decorator` functions which
wrap the decorated function and return the wrapper. If the wrapper function is
not updated, the metadata of the returned function will reflect the wrapper
definition rather than the original function definition, which is typically less
than helpful.
:func:`update_wrapper` may be used with callables other than functions. Any
attributes named in *assigned* or *updated* that are missing from the object
being wrapped are ignored (i.e. this function will not attempt to set them
on the wrapper function). :exc:`AttributeError` is still raised if the
wrapper function itself is missing any attributes named in *updated*.
.. versionadded:: 3.2
Automatic addition of the ``__wrapped__`` attribute.
.. versionadded:: 3.2
Copying of the ``__annotations__`` attribute by default.
.. versionchanged:: 3.2
Missing attributes no longer trigger an :exc:`AttributeError`.
.. versionchanged:: 3.4
The ``__wrapped__`` attribute now always refers to the wrapped
function, even if that function defined a ``__wrapped__`` attribute.
(see :issue:`17482`)
.. decorator:: wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
This is a convenience function for invoking :func:`update_wrapper` as a
function decorator when defining a wrapper function. It is equivalent to
``partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated)``.
For example::
>>> from functools import wraps
>>> def my_decorator(f):
... @wraps(f)
... def wrapper(*args, **kwds):
... print('Calling decorated function')
... return f(*args, **kwds)
... return wrapper
...
>>> @my_decorator
... def example():
... """Docstring"""
... print('Called example function')
...
>>> example()
Calling decorated function
Called example function
>>> example.__name__
'example'
>>> example.__doc__
'Docstring'
Without the use of this decorator factory, the name of the example function
would have been ``'wrapper'``, and the docstring of the original :func:`example`
would have been lost.
.. _partial-objects:
:class:`partial` Objects
------------------------
:class:`partial` objects are callable objects created by :func:`partial`. They
have three read-only attributes:
.. attribute:: partial.func
A callable object or function. Calls to the :class:`partial` object will be
forwarded to :attr:`func` with new arguments and keywords.
.. attribute:: partial.args
The leftmost positional arguments that will be prepended to the positional
arguments provided to a :class:`partial` object call.
.. attribute:: partial.keywords
The keyword arguments that will be supplied when the :class:`partial` object is
called.
:class:`partial` objects are like :class:`function` objects in that they are
callable, weak referencable, and can have attributes. There are some important
differences. For instance, the :attr:`~definition.__name__` and :attr:`__doc__` attributes
are not created automatically. Also, :class:`partial` objects defined in
classes behave like static methods and do not transform into bound methods
during instance attribute look-up.