2575 lines
91 KiB
ReStructuredText
2575 lines
91 KiB
ReStructuredText
****************************
|
||
What's New In Python 3.5
|
||
****************************
|
||
|
||
:Editors: Elvis Pranskevichus <elvis@magic.io>, Yury Selivanov <yury@magic.io>
|
||
|
||
.. Rules for maintenance:
|
||
|
||
* Anyone can add text to this document. Do not spend very much time
|
||
on the wording of your changes, because your text will probably
|
||
get rewritten to some degree.
|
||
|
||
* The maintainer will go through Misc/NEWS periodically and add
|
||
changes; it's therefore more important to add your changes to
|
||
Misc/NEWS than to this file.
|
||
|
||
* This is not a complete list of every single change; completeness
|
||
is the purpose of Misc/NEWS. Some changes I consider too small
|
||
or esoteric to include. If such a change is added to the text,
|
||
I'll just remove it. (This is another reason you shouldn't spend
|
||
too much time on writing your addition.)
|
||
|
||
* If you want to draw your new text to the attention of the
|
||
maintainer, add 'XXX' to the beginning of the paragraph or
|
||
section.
|
||
|
||
* It's OK to just add a fragmentary note about a change. For
|
||
example: "XXX Describe the transmogrify() function added to the
|
||
socket module." The maintainer will research the change and
|
||
write the necessary text.
|
||
|
||
* You can comment out your additions if you like, but it's not
|
||
necessary (especially when a final release is some months away).
|
||
|
||
* Credit the author of a patch or bugfix. Just the name is
|
||
sufficient; the e-mail address isn't necessary.
|
||
|
||
* It's helpful to add the bug/patch number as a comment:
|
||
|
||
XXX Describe the transmogrify() function added to the socket
|
||
module.
|
||
(Contributed by P.Y. Developer in :issue:`12345`.)
|
||
|
||
This saves the maintainer the effort of going through the Mercurial log
|
||
when researching a change.
|
||
|
||
This article explains the new features in Python 3.5, compared to 3.4.
|
||
Python 3.5 was released on September 13, 2015. See the
|
||
`changelog <https://docs.python.org/3.5/whatsnew/changelog.html>`_ for a full
|
||
list of changes.
|
||
|
||
.. seealso::
|
||
|
||
:pep:`478` - Python 3.5 Release Schedule
|
||
|
||
|
||
Summary -- Release highlights
|
||
=============================
|
||
|
||
New syntax features:
|
||
|
||
* :ref:`PEP 492 <whatsnew-pep-492>`, coroutines with async and await syntax.
|
||
* :ref:`PEP 465 <whatsnew-pep-465>`, a new matrix multiplication operator: ``a @ b``.
|
||
* :ref:`PEP 448 <whatsnew-pep-448>`, additional unpacking generalizations.
|
||
|
||
|
||
New library modules:
|
||
|
||
* :mod:`typing`: :ref:`PEP 484 -- Type Hints <whatsnew-pep-484>`.
|
||
* :mod:`zipapp`: :ref:`PEP 441 Improving Python ZIP Application Support
|
||
<whatsnew-zipapp>`.
|
||
|
||
|
||
New built-in features:
|
||
|
||
* ``bytes % args``, ``bytearray % args``: :ref:`PEP 461 <whatsnew-pep-461>` --
|
||
Adding ``%`` formatting to bytes and bytearray.
|
||
|
||
* New :meth:`bytes.hex`, :meth:`bytearray.hex` and :meth:`memoryview.hex`
|
||
methods. (Contributed by Arnon Yaari in :issue:`9951`.)
|
||
|
||
* :class:`memoryview` now supports tuple indexing (including multi-dimensional).
|
||
(Contributed by Antoine Pitrou in :issue:`23632`.)
|
||
|
||
* Generators have a new ``gi_yieldfrom`` attribute, which returns the
|
||
object being iterated by ``yield from`` expressions. (Contributed
|
||
by Benno Leslie and Yury Selivanov in :issue:`24450`.)
|
||
|
||
* A new :exc:`RecursionError` exception is now raised when maximum
|
||
recursion depth is reached. (Contributed by Georg Brandl
|
||
in :issue:`19235`.)
|
||
|
||
|
||
CPython implementation improvements:
|
||
|
||
* When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale),
|
||
:py:data:`sys.stdin` and :py:data:`sys.stdout` now use the
|
||
``surrogateescape`` error handler, instead of the ``strict`` error handler.
|
||
(Contributed by Victor Stinner in :issue:`19977`.)
|
||
|
||
* ``.pyo`` files are no longer used and have been replaced by a more flexible
|
||
scheme that includes the optimization level explicitly in ``.pyc`` name.
|
||
(See :ref:`PEP 488 overview <whatsnew-pep-488>`.)
|
||
|
||
* Builtin and extension modules are now initialized in a multi-phase process,
|
||
which is similar to how Python modules are loaded.
|
||
(See :ref:`PEP 489 overview <whatsnew-pep-489>`.)
|
||
|
||
|
||
Significant improvements in the standard library:
|
||
|
||
* :class:`collections.OrderedDict` is now
|
||
:ref:`implemented in C <whatsnew-ordereddict>`, which makes it
|
||
4 to 100 times faster.
|
||
|
||
* The :mod:`ssl` module gained
|
||
:ref:`support for Memory BIO <whatsnew-sslmemorybio>`, which decouples SSL
|
||
protocol handling from network IO.
|
||
|
||
* The new :func:`os.scandir` function provides a
|
||
:ref:`better and significantly faster way <whatsnew-pep-471>`
|
||
of directory traversal.
|
||
|
||
* :func:`functools.lru_cache` has been mostly
|
||
:ref:`reimplemented in C <whatsnew-lrucache>`, yielding much better
|
||
performance.
|
||
|
||
* The new :func:`subprocess.run` function provides a
|
||
:ref:`streamlined way to run subprocesses <whatsnew-subprocess>`.
|
||
|
||
* The :mod:`traceback` module has been significantly
|
||
:ref:`enhanced <whatsnew-traceback>` for improved
|
||
performance and developer convenience.
|
||
|
||
|
||
Security improvements:
|
||
|
||
* SSLv3 is now disabled throughout the standard library.
|
||
It can still be enabled by instantiating a :class:`ssl.SSLContext`
|
||
manually. (See :issue:`22638` for more details; this change was
|
||
backported to CPython 3.4 and 2.7.)
|
||
|
||
* HTTP cookie parsing is now stricter, in order to protect
|
||
against potential injection attacks. (Contributed by Antoine Pitrou
|
||
in :issue:`22796`.)
|
||
|
||
|
||
Windows improvements:
|
||
|
||
* A new installer for Windows has replaced the old MSI.
|
||
See :ref:`using-on-windows` for more information.
|
||
|
||
* Windows builds now use Microsoft Visual C++ 14.0, and extension modules
|
||
should use the same.
|
||
|
||
|
||
Please read on for a comprehensive list of user-facing changes, including many
|
||
other smaller improvements, CPython optimizations, deprecations, and potential
|
||
porting issues.
|
||
|
||
|
||
New Features
|
||
============
|
||
|
||
.. _whatsnew-pep-492:
|
||
|
||
PEP 492 - Coroutines with async and await syntax
|
||
------------------------------------------------
|
||
|
||
:pep:`492` greatly improves support for asynchronous programming in Python
|
||
by adding :term:`awaitable objects <awaitable>`,
|
||
:term:`coroutine functions <coroutine function>`,
|
||
:term:`asynchronous iteration <asynchronous iterable>`,
|
||
and :term:`asynchronous context managers <asynchronous context manager>`.
|
||
|
||
Coroutine functions are declared using the new :keyword:`async def` syntax::
|
||
|
||
>>> async def coro():
|
||
... return 'spam'
|
||
|
||
Inside a coroutine function, the new :keyword:`await` expression can be used
|
||
to suspend coroutine execution until the result is available. Any object
|
||
can be *awaited*, as long as it implements the :term:`awaitable` protocol by
|
||
defining the :meth:`__await__` method.
|
||
|
||
PEP 492 also adds :keyword:`async for` statement for convenient iteration
|
||
over asynchronous iterables.
|
||
|
||
An example of a rudimentary HTTP client written using the new syntax::
|
||
|
||
import asyncio
|
||
|
||
async def http_get(domain):
|
||
reader, writer = await asyncio.open_connection(domain, 80)
|
||
|
||
writer.write(b'\r\n'.join([
|
||
b'GET / HTTP/1.1',
|
||
b'Host: %b' % domain.encode('latin-1'),
|
||
b'Connection: close',
|
||
b'', b''
|
||
]))
|
||
|
||
async for line in reader:
|
||
print('>>>', line)
|
||
|
||
writer.close()
|
||
|
||
loop = asyncio.get_event_loop()
|
||
try:
|
||
loop.run_until_complete(http_get('example.com'))
|
||
finally:
|
||
loop.close()
|
||
|
||
|
||
Similarly to asynchronous iteration, there is a new syntax for asynchronous
|
||
context managers. The following script::
|
||
|
||
import asyncio
|
||
|
||
async def coro(name, lock):
|
||
print('coro {}: waiting for lock'.format(name))
|
||
async with lock:
|
||
print('coro {}: holding the lock'.format(name))
|
||
await asyncio.sleep(1)
|
||
print('coro {}: releasing the lock'.format(name))
|
||
|
||
loop = asyncio.get_event_loop()
|
||
lock = asyncio.Lock()
|
||
coros = asyncio.gather(coro(1, lock), coro(2, lock))
|
||
try:
|
||
loop.run_until_complete(coros)
|
||
finally:
|
||
loop.close()
|
||
|
||
will output::
|
||
|
||
coro 2: waiting for lock
|
||
coro 2: holding the lock
|
||
coro 1: waiting for lock
|
||
coro 2: releasing the lock
|
||
coro 1: holding the lock
|
||
coro 1: releasing the lock
|
||
|
||
Note that both :keyword:`async for` and :keyword:`async with` can only
|
||
be used inside a coroutine function declared with :keyword:`async def`.
|
||
|
||
Coroutine functions are intended to be run inside a compatible event loop,
|
||
such as the :ref:`asyncio loop <asyncio-event-loop>`.
|
||
|
||
|
||
.. note::
|
||
|
||
.. versionchanged:: 3.5.2
|
||
Starting with CPython 3.5.2, ``__aiter__`` can directly return
|
||
:term:`asynchronous iterators <asynchronous iterator>`. Returning
|
||
an :term:`awaitable` object will result in a
|
||
:exc:`PendingDeprecationWarning`.
|
||
|
||
See more details in the :ref:`async-iterators` documentation
|
||
section.
|
||
|
||
|
||
.. seealso::
|
||
|
||
:pep:`492` -- Coroutines with async and await syntax
|
||
PEP written and implemented by Yury Selivanov.
|
||
|
||
|
||
.. _whatsnew-pep-465:
|
||
|
||
PEP 465 - A dedicated infix operator for matrix multiplication
|
||
--------------------------------------------------------------
|
||
|
||
:pep:`465` adds the ``@`` infix operator for matrix multiplication.
|
||
Currently, no builtin Python types implement the new operator, however, it
|
||
can be implemented by defining :meth:`__matmul__`, :meth:`__rmatmul__`,
|
||
and :meth:`__imatmul__` for regular, reflected, and in-place matrix
|
||
multiplication. The semantics of these methods is similar to that of
|
||
methods defining other infix arithmetic operators.
|
||
|
||
Matrix multiplication is a notably common operation in many fields of
|
||
mathematics, science, engineering, and the addition of ``@`` allows writing
|
||
cleaner code::
|
||
|
||
S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r)
|
||
|
||
instead of::
|
||
|
||
S = dot((dot(H, beta) - r).T,
|
||
dot(inv(dot(dot(H, V), H.T)), dot(H, beta) - r))
|
||
|
||
NumPy 1.10 has support for the new operator::
|
||
|
||
>>> import numpy
|
||
|
||
>>> x = numpy.ones(3)
|
||
>>> x
|
||
array([ 1., 1., 1.])
|
||
|
||
>>> m = numpy.eye(3)
|
||
>>> m
|
||
array([[ 1., 0., 0.],
|
||
[ 0., 1., 0.],
|
||
[ 0., 0., 1.]])
|
||
|
||
>>> x @ m
|
||
array([ 1., 1., 1.])
|
||
|
||
|
||
.. seealso::
|
||
|
||
:pep:`465` -- A dedicated infix operator for matrix multiplication
|
||
PEP written by Nathaniel J. Smith; implemented by Benjamin Peterson.
|
||
|
||
|
||
.. _whatsnew-pep-448:
|
||
|
||
PEP 448 - Additional Unpacking Generalizations
|
||
----------------------------------------------
|
||
|
||
:pep:`448` extends the allowed uses of the ``*`` iterable unpacking
|
||
operator and ``**`` dictionary unpacking operator. It is now possible
|
||
to use an arbitrary number of unpackings in :ref:`function calls <calls>`::
|
||
|
||
>>> print(*[1], *[2], 3, *[4, 5])
|
||
1 2 3 4 5
|
||
|
||
>>> def fn(a, b, c, d):
|
||
... print(a, b, c, d)
|
||
...
|
||
|
||
>>> fn(**{'a': 1, 'c': 3}, **{'b': 2, 'd': 4})
|
||
1 2 3 4
|
||
|
||
Similarly, tuple, list, set, and dictionary displays allow multiple
|
||
unpackings (see :ref:`exprlists` and :ref:`dict`)::
|
||
|
||
>>> *range(4), 4
|
||
(0, 1, 2, 3, 4)
|
||
|
||
>>> [*range(4), 4]
|
||
[0, 1, 2, 3, 4]
|
||
|
||
>>> {*range(4), 4, *(5, 6, 7)}
|
||
{0, 1, 2, 3, 4, 5, 6, 7}
|
||
|
||
>>> {'x': 1, **{'y': 2}}
|
||
{'x': 1, 'y': 2}
|
||
|
||
.. seealso::
|
||
|
||
:pep:`448` -- Additional Unpacking Generalizations
|
||
PEP written by Joshua Landau; implemented by Neil Girdhar,
|
||
Thomas Wouters, and Joshua Landau.
|
||
|
||
|
||
.. _whatsnew-pep-461:
|
||
|
||
PEP 461 - percent formatting support for bytes and bytearray
|
||
------------------------------------------------------------
|
||
|
||
:pep:`461` adds support for the ``%``
|
||
:ref:`interpolation operator <bytes-formatting>` to :class:`bytes`
|
||
and :class:`bytearray`.
|
||
|
||
While interpolation is usually thought of as a string operation, there are
|
||
cases where interpolation on ``bytes`` or ``bytearrays`` makes sense, and the
|
||
work needed to make up for this missing functionality detracts from the
|
||
overall readability of the code. This issue is particularly important when
|
||
dealing with wire format protocols, which are often a mixture of binary
|
||
and ASCII compatible text.
|
||
|
||
Examples::
|
||
|
||
>>> b'Hello %b!' % b'World'
|
||
b'Hello World!'
|
||
|
||
>>> b'x=%i y=%f' % (1, 2.5)
|
||
b'x=1 y=2.500000'
|
||
|
||
Unicode is not allowed for ``%b``, but it is accepted by ``%a`` (equivalent of
|
||
``repr(obj).encode('ascii', 'backslashreplace')``)::
|
||
|
||
>>> b'Hello %b!' % 'World'
|
||
Traceback (most recent call last):
|
||
File "<stdin>", line 1, in <module>
|
||
TypeError: %b requires bytes, or an object that implements __bytes__, not 'str'
|
||
|
||
>>> b'price: %a' % '10€'
|
||
b"price: '10\\u20ac'"
|
||
|
||
Note that ``%s`` and ``%r`` conversion types, although supported, should
|
||
only be used in codebases that need compatibility with Python 2.
|
||
|
||
.. seealso::
|
||
|
||
:pep:`461` -- Adding % formatting to bytes and bytearray
|
||
PEP written by Ethan Furman; implemented by Neil Schemenauer and
|
||
Ethan Furman.
|
||
|
||
|
||
.. _whatsnew-pep-484:
|
||
|
||
PEP 484 - Type Hints
|
||
--------------------
|
||
|
||
Function annotation syntax has been a Python feature since version 3.0
|
||
(:pep:`3107`), however the semantics of annotations has been left undefined.
|
||
|
||
Experience has shown that the majority of function annotation
|
||
uses were to provide type hints to function parameters and return values. It
|
||
became evident that it would be beneficial for Python users, if the
|
||
standard library included the base definitions and tools for type annotations.
|
||
|
||
:pep:`484` introduces a :term:`provisional module <provisional API>` to
|
||
provide these standard definitions and tools, along with some conventions
|
||
for situations where annotations are not available.
|
||
|
||
For example, here is a simple function whose argument and return type
|
||
are declared in the annotations::
|
||
|
||
def greeting(name: str) -> str:
|
||
return 'Hello ' + name
|
||
|
||
While these annotations are available at runtime through the usual
|
||
:attr:`__annotations__` attribute, *no automatic type checking happens at
|
||
runtime*. Instead, it is assumed that a separate off-line type checker
|
||
(e.g. `mypy <http://mypy-lang.org>`_) will be used for on-demand
|
||
source code analysis.
|
||
|
||
The type system supports unions, generic types, and a special type
|
||
named :class:`~typing.Any` which is consistent with (i.e. assignable to
|
||
and from) all types.
|
||
|
||
.. seealso::
|
||
|
||
* :mod:`typing` module documentation
|
||
* :pep:`484` -- Type Hints
|
||
PEP written by Guido van Rossum, Jukka Lehtosalo, and Łukasz Langa;
|
||
implemented by Guido van Rossum.
|
||
* :pep:`483` -- The Theory of Type Hints
|
||
PEP written by Guido van Rossum
|
||
|
||
|
||
.. _whatsnew-pep-471:
|
||
|
||
PEP 471 - os.scandir() function -- a better and faster directory iterator
|
||
-------------------------------------------------------------------------
|
||
|
||
:pep:`471` adds a new directory iteration function, :func:`os.scandir`,
|
||
to the standard library. Additionally, :func:`os.walk` is now
|
||
implemented using ``scandir``, which makes it 3 to 5 times faster
|
||
on POSIX systems and 7 to 20 times faster on Windows systems. This is
|
||
largely achieved by greatly reducing the number of calls to :func:`os.stat`
|
||
required to walk a directory tree.
|
||
|
||
Additionally, ``scandir`` returns an iterator, as opposed to returning
|
||
a list of file names, which improves memory efficiency when iterating
|
||
over very large directories.
|
||
|
||
The following example shows a simple use of :func:`os.scandir` to display all
|
||
the files (excluding directories) in the given *path* that don't start with
|
||
``'.'``. The :meth:`entry.is_file() <os.DirEntry.is_file>` call will generally
|
||
not make an additional system call::
|
||
|
||
for entry in os.scandir(path):
|
||
if not entry.name.startswith('.') and entry.is_file():
|
||
print(entry.name)
|
||
|
||
.. seealso::
|
||
|
||
:pep:`471` -- os.scandir() function -- a better and faster directory iterator
|
||
PEP written and implemented by Ben Hoyt with the help of Victor Stinner.
|
||
|
||
|
||
.. _whatsnew-pep-475:
|
||
|
||
PEP 475: Retry system calls failing with EINTR
|
||
----------------------------------------------
|
||
|
||
An :py:data:`errno.EINTR` error code is returned whenever a system call, that
|
||
is waiting for I/O, is interrupted by a signal. Previously, Python would
|
||
raise :exc:`InterruptedError` in such cases. This meant that, when writing a
|
||
Python application, the developer had two choices:
|
||
|
||
#. Ignore the ``InterruptedError``.
|
||
#. Handle the ``InterruptedError`` and attempt to restart the interrupted
|
||
system call at every call site.
|
||
|
||
The first option makes an application fail intermittently.
|
||
The second option adds a large amount of boilerplate that makes the
|
||
code nearly unreadable. Compare::
|
||
|
||
print("Hello World")
|
||
|
||
and::
|
||
|
||
while True:
|
||
try:
|
||
print("Hello World")
|
||
break
|
||
except InterruptedError:
|
||
continue
|
||
|
||
:pep:`475` implements automatic retry of system calls on
|
||
``EINTR``. This removes the burden of dealing with ``EINTR``
|
||
or :exc:`InterruptedError` in user code in most situations and makes
|
||
Python programs, including the standard library, more robust. Note that
|
||
the system call is only retried if the signal handler does not raise an
|
||
exception.
|
||
|
||
Below is a list of functions which are now retried when interrupted
|
||
by a signal:
|
||
|
||
* :func:`open` and :func:`io.open`;
|
||
|
||
* functions of the :mod:`faulthandler` module;
|
||
|
||
* :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`,
|
||
:func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`,
|
||
:func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`,
|
||
:func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.open`,
|
||
:func:`~os.posix_fadvise`, :func:`~os.posix_fallocate`, :func:`~os.pread`,
|
||
:func:`~os.pwrite`, :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`,
|
||
:func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`,
|
||
:func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`,
|
||
:func:`~os.writev`;
|
||
|
||
* special cases: :func:`os.close` and :func:`os.dup2` now ignore
|
||
:py:data:`~errno.EINTR` errors; the syscall is not retried (see the PEP
|
||
for the rationale);
|
||
|
||
* :mod:`select` functions: :func:`devpoll.poll() <select.devpoll.poll>`,
|
||
:func:`epoll.poll() <select.epoll.poll>`,
|
||
:func:`kqueue.control() <select.kqueue.control>`,
|
||
:func:`poll.poll() <select.poll.poll>`, :func:`~select.select`;
|
||
|
||
* methods of the :class:`~socket.socket` class: :meth:`~socket.socket.accept`,
|
||
:meth:`~socket.socket.connect` (except for non-blocking sockets),
|
||
:meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`,
|
||
:meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`,
|
||
:meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`,
|
||
:meth:`~socket.socket.sendto`;
|
||
|
||
* :func:`signal.sigtimedwait` and :func:`signal.sigwaitinfo`;
|
||
|
||
* :func:`time.sleep`.
|
||
|
||
.. seealso::
|
||
|
||
:pep:`475` -- Retry system calls failing with EINTR
|
||
PEP and implementation written by Charles-François Natali and
|
||
Victor Stinner, with the help of Antoine Pitrou (the French connection).
|
||
|
||
|
||
.. _whatsnew-pep-479:
|
||
|
||
PEP 479: Change StopIteration handling inside generators
|
||
--------------------------------------------------------
|
||
|
||
The interaction of generators and :exc:`StopIteration` in Python 3.4 and
|
||
earlier was sometimes surprising, and could conceal obscure bugs. Previously,
|
||
``StopIteration`` raised accidentally inside a generator function was
|
||
interpreted as the end of the iteration by the loop construct driving the
|
||
generator.
|
||
|
||
:pep:`479` changes the behavior of generators: when a ``StopIteration``
|
||
exception is raised inside a generator, it is replaced with a
|
||
:exc:`RuntimeError` before it exits the generator frame. The main goal of
|
||
this change is to ease debugging in the situation where an unguarded
|
||
:func:`next` call raises ``StopIteration`` and causes the iteration controlled
|
||
by the generator to terminate silently. This is particularly pernicious in
|
||
combination with the ``yield from`` construct.
|
||
|
||
This is a backwards incompatible change, so to enable the new behavior,
|
||
a :term:`__future__` import is necessary::
|
||
|
||
>>> from __future__ import generator_stop
|
||
|
||
>>> def gen():
|
||
... next(iter([]))
|
||
... yield
|
||
...
|
||
>>> next(gen())
|
||
Traceback (most recent call last):
|
||
File "<stdin>", line 2, in gen
|
||
StopIteration
|
||
|
||
The above exception was the direct cause of the following exception:
|
||
|
||
Traceback (most recent call last):
|
||
File "<stdin>", line 1, in <module>
|
||
RuntimeError: generator raised StopIteration
|
||
|
||
Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be
|
||
raised whenever a :exc:`StopIteration` exception is raised inside a generator.
|
||
|
||
.. seealso::
|
||
|
||
:pep:`479` -- Change StopIteration handling inside generators
|
||
PEP written by Chris Angelico and Guido van Rossum. Implemented by
|
||
Chris Angelico, Yury Selivanov and Nick Coghlan.
|
||
|
||
|
||
.. _whatsnew-pep-485:
|
||
|
||
PEP 485: A function for testing approximate equality
|
||
----------------------------------------------------
|
||
|
||
:pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose`
|
||
functions which tell whether two values are approximately equal or
|
||
"close" to each other. Whether or not two values are considered
|
||
close is determined according to given absolute and relative tolerances.
|
||
Relative tolerance is the maximum allowed difference between ``isclose``
|
||
arguments, relative to the larger absolute value::
|
||
|
||
>>> import math
|
||
>>> a = 5.0
|
||
>>> b = 4.99998
|
||
>>> math.isclose(a, b, rel_tol=1e-5)
|
||
True
|
||
>>> math.isclose(a, b, rel_tol=1e-6)
|
||
False
|
||
|
||
It is also possible to compare two values using absolute tolerance, which
|
||
must be a non-negative value::
|
||
|
||
>>> import math
|
||
>>> a = 5.0
|
||
>>> b = 4.99998
|
||
>>> math.isclose(a, b, abs_tol=0.00003)
|
||
True
|
||
>>> math.isclose(a, b, abs_tol=0.00001)
|
||
False
|
||
|
||
.. seealso::
|
||
|
||
:pep:`485` -- A function for testing approximate equality
|
||
PEP written by Christopher Barker; implemented by Chris Barker and
|
||
Tal Einat.
|
||
|
||
|
||
.. _whatsnew-pep-486:
|
||
|
||
PEP 486: Make the Python Launcher aware of virtual environments
|
||
---------------------------------------------------------------
|
||
|
||
:pep:`486` makes the Windows launcher (see :pep:`397`) aware of an active
|
||
virtual environment. When the default interpreter would be used and the
|
||
``VIRTUAL_ENV`` environment variable is set, the interpreter in the virtual
|
||
environment will be used.
|
||
|
||
.. seealso::
|
||
|
||
:pep:`486` -- Make the Python Launcher aware of virtual environments
|
||
PEP written and implemented by Paul Moore.
|
||
|
||
|
||
.. _whatsnew-pep-488:
|
||
|
||
PEP 488: Elimination of PYO files
|
||
---------------------------------
|
||
|
||
:pep:`488` does away with the concept of ``.pyo`` files. This means that
|
||
``.pyc`` files represent both unoptimized and optimized bytecode. To prevent the
|
||
need to constantly regenerate bytecode files, ``.pyc`` files now have an
|
||
optional ``opt-`` tag in their name when the bytecode is optimized. This has the
|
||
side-effect of no more bytecode file name clashes when running under either
|
||
:option:`-O` or :option:`-OO`. Consequently, bytecode files generated from
|
||
:option:`-O`, and :option:`-OO` may now exist simultaneously.
|
||
:func:`importlib.util.cache_from_source` has an updated API to help with
|
||
this change.
|
||
|
||
.. seealso::
|
||
|
||
:pep:`488` -- Elimination of PYO files
|
||
PEP written and implemented by Brett Cannon.
|
||
|
||
|
||
.. _whatsnew-pep-489:
|
||
|
||
PEP 489: Multi-phase extension module initialization
|
||
----------------------------------------------------
|
||
|
||
:pep:`489` updates extension module initialization to take advantage of the
|
||
two step module loading mechanism introduced by :pep:`451` in Python 3.4.
|
||
|
||
This change brings the import semantics of extension modules that opt-in to
|
||
using the new mechanism much closer to those of Python source and bytecode
|
||
modules, including the ability to use any valid identifier as a module name,
|
||
rather than being restricted to ASCII.
|
||
|
||
.. seealso::
|
||
|
||
:pep:`489` -- Multi-phase extension module initialization
|
||
PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan;
|
||
implemented by Petr Viktorin.
|
||
|
||
|
||
Other Language Changes
|
||
======================
|
||
|
||
Some smaller changes made to the core Python language are:
|
||
|
||
* Added the ``"namereplace"`` error handlers. The ``"backslashreplace"``
|
||
error handlers now work with decoding and translating.
|
||
(Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.)
|
||
|
||
* The :option:`-b` option now affects comparisons of :class:`bytes` with
|
||
:class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`.)
|
||
|
||
* New Kazakh ``kz1048`` and Tajik ``koi8_t`` :ref:`codecs <standard-encodings>`.
|
||
(Contributed by Serhiy Storchaka in :issue:`22682` and :issue:`22681`.)
|
||
|
||
* Property docstrings are now writable. This is especially useful for
|
||
:func:`collections.namedtuple` docstrings.
|
||
(Contributed by Berker Peksag in :issue:`24064`.)
|
||
|
||
* Circular imports involving relative imports are now supported.
|
||
(Contributed by Brett Cannon and Antoine Pitrou in :issue:`17636`.)
|
||
|
||
|
||
New Modules
|
||
===========
|
||
|
||
typing
|
||
------
|
||
|
||
The new :mod:`typing` :term:`provisional <provisional API>` module
|
||
provides standard definitions and tools for function type annotations.
|
||
See :ref:`Type Hints <whatsnew-pep-484>` for more information.
|
||
|
||
.. _whatsnew-zipapp:
|
||
|
||
zipapp
|
||
------
|
||
|
||
The new :mod:`zipapp` module (specified in :pep:`441`) provides an API and
|
||
command line tool for creating executable Python Zip Applications, which
|
||
were introduced in Python 2.6 in :issue:`1739468`, but which were not well
|
||
publicized, either at the time or since.
|
||
|
||
With the new module, bundling your application is as simple as putting all
|
||
the files, including a ``__main__.py`` file, into a directory ``myapp``
|
||
and running:
|
||
|
||
.. code-block:: shell-session
|
||
|
||
$ python -m zipapp myapp
|
||
$ python myapp.pyz
|
||
|
||
The module implementation has been contributed by Paul Moore in
|
||
:issue:`23491`.
|
||
|
||
.. seealso::
|
||
|
||
:pep:`441` -- Improving Python ZIP Application Support
|
||
|
||
|
||
Improved Modules
|
||
================
|
||
|
||
argparse
|
||
--------
|
||
|
||
The :class:`~argparse.ArgumentParser` class now allows disabling
|
||
:ref:`abbreviated usage <prefix-matching>` of long options by setting
|
||
:ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh,
|
||
Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.)
|
||
|
||
|
||
asyncio
|
||
-------
|
||
|
||
Since the :mod:`asyncio` module is :term:`provisional <provisional API>`,
|
||
all changes introduced in Python 3.5 have also been backported to Python 3.4.x.
|
||
|
||
Notable changes in the :mod:`asyncio` module since Python 3.4.0:
|
||
|
||
* New debugging APIs: :meth:`loop.set_debug() <asyncio.loop.set_debug>`
|
||
and :meth:`loop.get_debug() <asyncio.loop.get_debug>` methods.
|
||
(Contributed by Victor Stinner.)
|
||
|
||
* The proactor event loop now supports SSL.
|
||
(Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.)
|
||
|
||
* A new :meth:`loop.is_closed() <asyncio.loop.is_closed>` method to
|
||
check if the event loop is closed.
|
||
(Contributed by Victor Stinner in :issue:`21326`.)
|
||
|
||
* A new :meth:`loop.create_task() <asyncio.loop.create_task>`
|
||
to conveniently create and schedule a new :class:`~asyncio.Task`
|
||
for a coroutine. The ``create_task`` method is also used by all
|
||
asyncio functions that wrap coroutines into tasks, such as
|
||
:func:`asyncio.wait`, :func:`asyncio.gather`, etc.
|
||
(Contributed by Victor Stinner.)
|
||
|
||
* A new :meth:`transport.get_write_buffer_limits() <asyncio.WriteTransport.get_write_buffer_limits>`
|
||
method to inquire for *high-* and *low-* water limits of the flow
|
||
control.
|
||
(Contributed by Victor Stinner.)
|
||
|
||
* The :func:`~asyncio.async` function is deprecated in favor of
|
||
:func:`~asyncio.ensure_future`.
|
||
(Contributed by Yury Selivanov.)
|
||
|
||
* New :meth:`loop.set_task_factory()
|
||
<asyncio.loop.set_task_factory>` and
|
||
:meth:`loop.get_task_factory() <asyncio.loop.get_task_factory>`
|
||
methods to customize the task factory that :meth:`loop.create_task()
|
||
<asyncio.loop.create_task>` method uses. (Contributed by Yury
|
||
Selivanov.)
|
||
|
||
* New :meth:`Queue.join() <asyncio.Queue.join>` and
|
||
:meth:`Queue.task_done() <asyncio.Queue.task_done>` queue methods.
|
||
(Contributed by Victor Stinner.)
|
||
|
||
* The ``JoinableQueue`` class was removed, in favor of the
|
||
:class:`asyncio.Queue` class.
|
||
(Contributed by Victor Stinner.)
|
||
|
||
Updates in 3.5.1:
|
||
|
||
* The :func:`~asyncio.ensure_future` function and all functions that
|
||
use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`,
|
||
now accept all kinds of :term:`awaitable objects <awaitable>`.
|
||
(Contributed by Yury Selivanov.)
|
||
|
||
* New :func:`~asyncio.run_coroutine_threadsafe` function to submit
|
||
coroutines to event loops from other threads.
|
||
(Contributed by Vincent Michel.)
|
||
|
||
* New :meth:`Transport.is_closing() <asyncio.BaseTransport.is_closing>`
|
||
method to check if the transport is closing or closed.
|
||
(Contributed by Yury Selivanov.)
|
||
|
||
* The :meth:`loop.create_server() <asyncio.loop.create_server>`
|
||
method can now accept a list of hosts.
|
||
(Contributed by Yann Sionneau.)
|
||
|
||
Updates in 3.5.2:
|
||
|
||
* New :meth:`loop.create_future() <asyncio.loop.create_future>`
|
||
method to create Future objects. This allows alternative event
|
||
loop implementations, such as
|
||
`uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster
|
||
:class:`asyncio.Future` implementation.
|
||
(Contributed by Yury Selivanov.)
|
||
|
||
* New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>`
|
||
method to get the current exception handler.
|
||
(Contributed by Yury Selivanov.)
|
||
|
||
* New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>`
|
||
method to read data from the stream until a separator bytes
|
||
sequence appears.
|
||
(Contributed by Mark Korenberg.)
|
||
|
||
* The :meth:`loop.create_connection() <asyncio.loop.create_connection>`
|
||
and :meth:`loop.create_server() <asyncio.loop.create_server>`
|
||
methods are optimized to avoid calling the system ``getaddrinfo``
|
||
function if the address is already resolved.
|
||
(Contributed by A. Jesse Jiryu Davis.)
|
||
|
||
* The :meth:`loop.sock_connect(sock, address) <asyncio.loop.sock_connect>`
|
||
no longer requires the *address* to be resolved prior to the call.
|
||
(Contributed by A. Jesse Jiryu Davis.)
|
||
|
||
|
||
bz2
|
||
---
|
||
|
||
The :meth:`BZ2Decompressor.decompress <bz2.BZ2Decompressor.decompress>`
|
||
method now accepts an optional *max_length* argument to limit the maximum
|
||
size of decompressed data. (Contributed by Nikolaus Rath in :issue:`15955`.)
|
||
|
||
|
||
cgi
|
||
---
|
||
|
||
The :class:`~cgi.FieldStorage` class now supports the :term:`context manager`
|
||
protocol. (Contributed by Berker Peksag in :issue:`20289`.)
|
||
|
||
|
||
cmath
|
||
-----
|
||
|
||
A new function :func:`~cmath.isclose` provides a way to test for approximate
|
||
equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.)
|
||
|
||
|
||
code
|
||
----
|
||
|
||
The :func:`InteractiveInterpreter.showtraceback() <code.InteractiveInterpreter.showtraceback>`
|
||
method now prints the full chained traceback, just like the interactive
|
||
interpreter. (Contributed by Claudiu Popa in :issue:`17442`.)
|
||
|
||
|
||
collections
|
||
-----------
|
||
|
||
.. _whatsnew-ordereddict:
|
||
|
||
The :class:`~collections.OrderedDict` class is now implemented in C, which
|
||
makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.)
|
||
|
||
:meth:`OrderedDict.items() <collections.OrderedDict.items>`,
|
||
:meth:`OrderedDict.keys() <collections.OrderedDict.keys>`,
|
||
:meth:`OrderedDict.values() <collections.OrderedDict.values>` views now support
|
||
:func:`reversed` iteration.
|
||
(Contributed by Serhiy Storchaka in :issue:`19505`.)
|
||
|
||
The :class:`~collections.deque` class now defines
|
||
:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and
|
||
:meth:`~collections.deque.copy`, and supports the ``+`` and ``*`` operators.
|
||
This allows deques to be recognized as a :class:`~collections.abc.MutableSequence`
|
||
and improves their substitutability for lists.
|
||
(Contributed by Raymond Hettinger in :issue:`23704`.)
|
||
|
||
Docstrings produced by :func:`~collections.namedtuple` can now be updated::
|
||
|
||
Point = namedtuple('Point', ['x', 'y'])
|
||
Point.__doc__ += ': Cartesian coodinate'
|
||
Point.x.__doc__ = 'abscissa'
|
||
Point.y.__doc__ = 'ordinate'
|
||
|
||
(Contributed by Berker Peksag in :issue:`24064`.)
|
||
|
||
The :class:`~collections.UserString` class now implements the
|
||
:meth:`__getnewargs__`, :meth:`__rmod__`, :meth:`~str.casefold`,
|
||
:meth:`~str.format_map`, :meth:`~str.isprintable`, and :meth:`~str.maketrans`
|
||
methods to match the corresponding methods of :class:`str`.
|
||
(Contributed by Joe Jevnik in :issue:`22189`.)
|
||
|
||
|
||
collections.abc
|
||
---------------
|
||
|
||
The :meth:`Sequence.index() <collections.abc.Sequence.index>` method now
|
||
accepts *start* and *stop* arguments to match the corresponding methods
|
||
of :class:`tuple`, :class:`list`, etc.
|
||
(Contributed by Devin Jeanpierre in :issue:`23086`.)
|
||
|
||
A new :class:`~collections.abc.Generator` abstract base class. (Contributed
|
||
by Stefan Behnel in :issue:`24018`.)
|
||
|
||
New :class:`~collections.abc.Awaitable`, :class:`~collections.abc.Coroutine`,
|
||
:class:`~collections.abc.AsyncIterator`, and
|
||
:class:`~collections.abc.AsyncIterable` abstract base classes.
|
||
(Contributed by Yury Selivanov in :issue:`24184`.)
|
||
|
||
For earlier Python versions, a backport of the new ABCs is available in an
|
||
external `PyPI package <https://pypi.org/project/backports_abc>`_.
|
||
|
||
|
||
compileall
|
||
----------
|
||
|
||
A new :mod:`compileall` option, :samp:`-j {N}`, allows running *N* workers
|
||
simultaneously to perform parallel bytecode compilation.
|
||
The :func:`~compileall.compile_dir` function has a corresponding ``workers``
|
||
parameter. (Contributed by Claudiu Popa in :issue:`16104`.)
|
||
|
||
Another new option, ``-r``, allows controlling the maximum recursion
|
||
level for subdirectories. (Contributed by Claudiu Popa in :issue:`19628`.)
|
||
|
||
The ``-q`` command line option can now be specified more than once, in
|
||
which case all output, including errors, will be suppressed. The corresponding
|
||
``quiet`` parameter in :func:`~compileall.compile_dir`,
|
||
:func:`~compileall.compile_file`, and :func:`~compileall.compile_path` can now
|
||
accept an integer value indicating the level of output suppression.
|
||
(Contributed by Thomas Kluyver in :issue:`21338`.)
|
||
|
||
|
||
concurrent.futures
|
||
------------------
|
||
|
||
The :meth:`Executor.map() <concurrent.futures.Executor.map>` method now accepts a
|
||
*chunksize* argument to allow batching of tasks to improve performance when
|
||
:meth:`~concurrent.futures.ProcessPoolExecutor` is used.
|
||
(Contributed by Dan O'Reilly in :issue:`11271`.)
|
||
|
||
The number of workers in the :class:`~concurrent.futures.ThreadPoolExecutor`
|
||
constructor is optional now. The default value is 5 times the number of CPUs.
|
||
(Contributed by Claudiu Popa in :issue:`21527`.)
|
||
|
||
|
||
configparser
|
||
------------
|
||
|
||
:mod:`configparser` now provides a way to customize the conversion
|
||
of values by specifying a dictionary of converters in the
|
||
:class:`~configparser.ConfigParser` constructor, or by defining them
|
||
as methods in ``ConfigParser`` subclasses. Converters defined in
|
||
a parser instance are inherited by its section proxies.
|
||
|
||
Example::
|
||
|
||
>>> import configparser
|
||
>>> conv = {}
|
||
>>> conv['list'] = lambda v: [e.strip() for e in v.split() if e.strip()]
|
||
>>> cfg = configparser.ConfigParser(converters=conv)
|
||
>>> cfg.read_string("""
|
||
... [s]
|
||
... list = a b c d e f g
|
||
... """)
|
||
>>> cfg.get('s', 'list')
|
||
'a b c d e f g'
|
||
>>> cfg.getlist('s', 'list')
|
||
['a', 'b', 'c', 'd', 'e', 'f', 'g']
|
||
>>> section = cfg['s']
|
||
>>> section.getlist('list')
|
||
['a', 'b', 'c', 'd', 'e', 'f', 'g']
|
||
|
||
(Contributed by Łukasz Langa in :issue:`18159`.)
|
||
|
||
|
||
contextlib
|
||
----------
|
||
|
||
The new :func:`~contextlib.redirect_stderr` :term:`context manager` (similar to
|
||
:func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to
|
||
handle inflexible APIs that write their output to :data:`sys.stderr` and
|
||
don't provide any options to redirect it::
|
||
|
||
>>> import contextlib, io, logging
|
||
>>> f = io.StringIO()
|
||
>>> with contextlib.redirect_stderr(f):
|
||
... logging.warning('warning')
|
||
...
|
||
>>> f.getvalue()
|
||
'WARNING:root:warning\n'
|
||
|
||
(Contributed by Berker Peksag in :issue:`22389`.)
|
||
|
||
|
||
csv
|
||
---
|
||
|
||
The :meth:`~csv.csvwriter.writerow` method now supports arbitrary iterables,
|
||
not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.)
|
||
|
||
|
||
curses
|
||
------
|
||
|
||
The new :func:`~curses.update_lines_cols` function updates the :envvar:`LINES`
|
||
and :envvar:`COLS` environment variables. This is useful for detecting
|
||
manual screen resizing. (Contributed by Arnon Yaari in :issue:`4254`.)
|
||
|
||
|
||
dbm
|
||
---
|
||
|
||
:func:`dumb.open <dbm.dumb.open>` always creates a new database when the flag
|
||
has the value ``"n"``. (Contributed by Claudiu Popa in :issue:`18039`.)
|
||
|
||
|
||
difflib
|
||
-------
|
||
|
||
The charset of HTML documents generated by
|
||
:meth:`HtmlDiff.make_file() <difflib.HtmlDiff.make_file>`
|
||
can now be customized by using a new *charset* keyword-only argument.
|
||
The default charset of HTML document changed from ``"ISO-8859-1"``
|
||
to ``"utf-8"``.
|
||
(Contributed by Berker Peksag in :issue:`2052`.)
|
||
|
||
The :func:`~difflib.diff_bytes` function can now compare lists of byte
|
||
strings. This fixes a regression from Python 2.
|
||
(Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.)
|
||
|
||
|
||
distutils
|
||
---------
|
||
|
||
Both the ``build`` and ``build_ext`` commands now accept a ``-j`` option to
|
||
enable parallel building of extension modules.
|
||
(Contributed by Antoine Pitrou in :issue:`5309`.)
|
||
|
||
The :mod:`distutils` module now supports ``xz`` compression, and can be
|
||
enabled by passing ``xztar`` as an argument to ``bdist --format``.
|
||
(Contributed by Serhiy Storchaka in :issue:`16314`.)
|
||
|
||
|
||
doctest
|
||
-------
|
||
|
||
The :func:`~doctest.DocTestSuite` function returns an empty
|
||
:class:`unittest.TestSuite` if *module* contains no docstrings, instead of
|
||
raising :exc:`ValueError`. (Contributed by Glenn Jones in :issue:`15916`.)
|
||
|
||
|
||
email
|
||
-----
|
||
|
||
A new policy option :attr:`Policy.mangle_from_ <email.policy.Policy.mangle_from_>`
|
||
controls whether or not lines that start with ``"From "`` in email bodies are
|
||
prefixed with a ``">"`` character by generators. The default is ``True`` for
|
||
:attr:`~email.policy.compat32` and ``False`` for all other policies.
|
||
(Contributed by Milan Oberkirch in :issue:`20098`.)
|
||
|
||
A new
|
||
:meth:`Message.get_content_disposition() <email.message.Message.get_content_disposition>`
|
||
method provides easy access to a canonical value for the
|
||
:mailheader:`Content-Disposition` header.
|
||
(Contributed by Abhilash Raj in :issue:`21083`.)
|
||
|
||
A new policy option :attr:`EmailPolicy.utf8 <email.policy.EmailPolicy.utf8>`
|
||
can be set to ``True`` to encode email headers using the UTF-8 charset instead
|
||
of using encoded words. This allows ``Messages`` to be formatted according to
|
||
:rfc:`6532` and used with an SMTP server that supports the :rfc:`6531`
|
||
``SMTPUTF8`` extension. (Contributed by R. David Murray in
|
||
:issue:`24211`.)
|
||
|
||
The :class:`mime.text.MIMEText <email.mime.text.MIMEText>` constructor now
|
||
accepts a :class:`charset.Charset <email.charset.Charset>` instance.
|
||
(Contributed by Claude Paroz and Berker Peksag in :issue:`16324`.)
|
||
|
||
|
||
enum
|
||
----
|
||
|
||
The :class:`~enum.Enum` callable has a new parameter *start* to
|
||
specify the initial number of enum values if only *names* are provided::
|
||
|
||
>>> Animal = enum.Enum('Animal', 'cat dog', start=10)
|
||
>>> Animal.cat
|
||
<Animal.cat: 10>
|
||
>>> Animal.dog
|
||
<Animal.dog: 11>
|
||
|
||
(Contributed by Ethan Furman in :issue:`21706`.)
|
||
|
||
|
||
faulthandler
|
||
------------
|
||
|
||
The :func:`~faulthandler.enable`, :func:`~faulthandler.register`,
|
||
:func:`~faulthandler.dump_traceback` and
|
||
:func:`~faulthandler.dump_traceback_later` functions now accept file
|
||
descriptors in addition to file-like objects.
|
||
(Contributed by Wei Wu in :issue:`23566`.)
|
||
|
||
|
||
functools
|
||
---------
|
||
|
||
.. _whatsnew-lrucache:
|
||
|
||
Most of the :func:`~functools.lru_cache` machinery is now implemented in C, making
|
||
it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and
|
||
Serhiy Storchaka in :issue:`14373`.)
|
||
|
||
|
||
glob
|
||
----
|
||
|
||
The :func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive
|
||
search in subdirectories, using the ``"**"`` pattern.
|
||
(Contributed by Serhiy Storchaka in :issue:`13968`.)
|
||
|
||
|
||
gzip
|
||
----
|
||
|
||
The *mode* argument of the :class:`~gzip.GzipFile` constructor now
|
||
accepts ``"x"`` to request exclusive creation.
|
||
(Contributed by Tim Heaney in :issue:`19222`.)
|
||
|
||
|
||
heapq
|
||
-----
|
||
|
||
Element comparison in :func:`~heapq.merge` can now be customized by
|
||
passing a :term:`key function` in a new optional *key* keyword argument,
|
||
and a new optional *reverse* keyword argument can be used to reverse element
|
||
comparison::
|
||
|
||
>>> import heapq
|
||
>>> a = ['9', '777', '55555']
|
||
>>> b = ['88', '6666']
|
||
>>> list(heapq.merge(a, b, key=len))
|
||
['9', '88', '777', '6666', '55555']
|
||
>>> list(heapq.merge(reversed(a), reversed(b), key=len, reverse=True))
|
||
['55555', '6666', '777', '88', '9']
|
||
|
||
(Contributed by Raymond Hettinger in :issue:`13742`.)
|
||
|
||
|
||
http
|
||
----
|
||
|
||
A new :class:`HTTPStatus <http.HTTPStatus>` enum that defines a set of
|
||
HTTP status codes, reason phrases and long descriptions written in English.
|
||
(Contributed by Demian Brecht in :issue:`21793`.)
|
||
|
||
|
||
http.client
|
||
-----------
|
||
|
||
:meth:`HTTPConnection.getresponse() <http.client.HTTPConnection.getresponse>`
|
||
now raises a :exc:`~http.client.RemoteDisconnected` exception when a
|
||
remote server connection is closed unexpectedly. Additionally, if a
|
||
:exc:`ConnectionError` (of which ``RemoteDisconnected``
|
||
is a subclass) is raised, the client socket is now closed automatically,
|
||
and will reconnect on the next request::
|
||
|
||
import http.client
|
||
conn = http.client.HTTPConnection('www.python.org')
|
||
for retries in range(3):
|
||
try:
|
||
conn.request('GET', '/')
|
||
resp = conn.getresponse()
|
||
except http.client.RemoteDisconnected:
|
||
pass
|
||
|
||
(Contributed by Martin Panter in :issue:`3566`.)
|
||
|
||
|
||
idlelib and IDLE
|
||
----------------
|
||
|
||
Since idlelib implements the IDLE shell and editor and is not intended for
|
||
import by other programs, it gets improvements with every release. See
|
||
:file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0,
|
||
as well as changes made in future 3.5.x releases. This file is also available
|
||
from the IDLE :menuselection:`Help --> About IDLE` dialog.
|
||
|
||
|
||
imaplib
|
||
-------
|
||
|
||
The :class:`~imaplib.IMAP4` class now supports the :term:`context manager` protocol.
|
||
When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT``
|
||
command will be called automatically at the end of the block.
|
||
(Contributed by Tarek Ziadé and Serhiy Storchaka in :issue:`4972`.)
|
||
|
||
The :mod:`imaplib` module now supports :rfc:`5161` (ENABLE Extension)
|
||
and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable() <imaplib.IMAP4.enable>`
|
||
method. A new :attr:`IMAP4.utf8_enabled <imaplib.IMAP4.utf8_enabled>`
|
||
attribute tracks whether or not :rfc:`6855` support is enabled.
|
||
(Contributed by Milan Oberkirch, R. David Murray, and Maciej Szulik in
|
||
:issue:`21800`.)
|
||
|
||
The :mod:`imaplib` module now automatically encodes non-ASCII string usernames
|
||
and passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan
|
||
Oberkirch in :issue:`21800`.)
|
||
|
||
|
||
imghdr
|
||
------
|
||
|
||
The :func:`~imghdr.what` function now recognizes the
|
||
`OpenEXR <http://www.openexr.com>`_ format
|
||
(contributed by Martin Vignali and Claudiu Popa in :issue:`20295`),
|
||
and the `WebP <https://en.wikipedia.org/wiki/WebP>`_ format
|
||
(contributed by Fabrice Aneche and Claudiu Popa in :issue:`20197`.)
|
||
|
||
|
||
importlib
|
||
---------
|
||
|
||
The :class:`util.LazyLoader <importlib.util.LazyLoader>` class allows for
|
||
lazy loading of modules in applications where startup time is important.
|
||
(Contributed by Brett Cannon in :issue:`17621`.)
|
||
|
||
The :func:`abc.InspectLoader.source_to_code() <importlib.abc.InspectLoader.source_to_code>`
|
||
method is now a static method. This makes it easier to initialize a module
|
||
object with code compiled from a string by running
|
||
``exec(code, module.__dict__)``.
|
||
(Contributed by Brett Cannon in :issue:`21156`.)
|
||
|
||
The new :func:`util.module_from_spec() <importlib.util.module_from_spec>`
|
||
function is now the preferred way to create a new module. As opposed to
|
||
creating a :class:`types.ModuleType` instance directly, this new function
|
||
will set the various import-controlled attributes based on the passed-in
|
||
spec object. (Contributed by Brett Cannon in :issue:`20383`.)
|
||
|
||
|
||
inspect
|
||
-------
|
||
|
||
Both the :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are
|
||
now picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726`
|
||
and :issue:`20334`.)
|
||
|
||
A new
|
||
:meth:`BoundArguments.apply_defaults() <inspect.BoundArguments.apply_defaults>`
|
||
method provides a way to set default values for missing arguments::
|
||
|
||
>>> def foo(a, b='ham', *args): pass
|
||
>>> ba = inspect.signature(foo).bind('spam')
|
||
>>> ba.apply_defaults()
|
||
>>> ba.arguments
|
||
OrderedDict([('a', 'spam'), ('b', 'ham'), ('args', ())])
|
||
|
||
(Contributed by Yury Selivanov in :issue:`24190`.)
|
||
|
||
A new class method
|
||
:meth:`Signature.from_callable() <inspect.Signature.from_callable>` makes
|
||
subclassing of :class:`~inspect.Signature` easier. (Contributed
|
||
by Yury Selivanov and Eric Snow in :issue:`17373`.)
|
||
|
||
The :func:`~inspect.signature` function now accepts a *follow_wrapped*
|
||
optional keyword argument, which, when set to ``False``, disables automatic
|
||
following of ``__wrapped__`` links.
|
||
(Contributed by Yury Selivanov in :issue:`20691`.)
|
||
|
||
A set of new functions to inspect
|
||
:term:`coroutine functions <coroutine function>` and
|
||
:term:`coroutine objects <coroutine>` has been added:
|
||
:func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`,
|
||
:func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`,
|
||
and :func:`~inspect.getcoroutinestate`.
|
||
(Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.)
|
||
|
||
The :func:`~inspect.stack`, :func:`~inspect.trace`,
|
||
:func:`~inspect.getouterframes`, and :func:`~inspect.getinnerframes`
|
||
functions now return a list of named tuples.
|
||
(Contributed by Daniel Shahaf in :issue:`16808`.)
|
||
|
||
|
||
io
|
||
--
|
||
|
||
A new :meth:`BufferedIOBase.readinto1() <io.BufferedIOBase.readinto1>`
|
||
method, that uses at most one call to the underlying raw stream's
|
||
:meth:`RawIOBase.read() <io.RawIOBase.read>` or
|
||
:meth:`RawIOBase.readinto() <io.RawIOBase.readinto>` methods.
|
||
(Contributed by Nikolaus Rath in :issue:`20578`.)
|
||
|
||
|
||
ipaddress
|
||
---------
|
||
|
||
Both the :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes
|
||
now accept an ``(address, netmask)`` tuple argument, so as to easily construct
|
||
network objects from existing addresses::
|
||
|
||
>>> import ipaddress
|
||
>>> ipaddress.IPv4Network(('127.0.0.0', 8))
|
||
IPv4Network('127.0.0.0/8')
|
||
>>> ipaddress.IPv4Network(('127.0.0.0', '255.0.0.0'))
|
||
IPv4Network('127.0.0.0/8')
|
||
|
||
(Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.)
|
||
|
||
A new :attr:`~ipaddress.IPv4Network.reverse_pointer` attribute for the
|
||
:class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes
|
||
returns the name of the reverse DNS PTR record::
|
||
|
||
>>> import ipaddress
|
||
>>> addr = ipaddress.IPv4Address('127.0.0.1')
|
||
>>> addr.reverse_pointer
|
||
'1.0.0.127.in-addr.arpa'
|
||
>>> addr6 = ipaddress.IPv6Address('::1')
|
||
>>> addr6.reverse_pointer
|
||
'1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa'
|
||
|
||
(Contributed by Leon Weber in :issue:`20480`.)
|
||
|
||
|
||
json
|
||
----
|
||
|
||
The :mod:`json.tool` command line interface now preserves the order of keys in
|
||
JSON objects passed in input. The new ``--sort-keys`` option can be used
|
||
to sort the keys alphabetically. (Contributed by Berker Peksag
|
||
in :issue:`21650`.)
|
||
|
||
JSON decoder now raises :exc:`~json.JSONDecodeError` instead of
|
||
:exc:`ValueError` to provide better context information about the error.
|
||
(Contributed by Serhiy Storchaka in :issue:`19361`.)
|
||
|
||
|
||
linecache
|
||
---------
|
||
|
||
A new :func:`~linecache.lazycache` function can be used to capture information
|
||
about a non-file-based module to permit getting its lines later via
|
||
:func:`~linecache.getline`. This avoids doing I/O until a line is actually
|
||
needed, without having to carry the module globals around indefinitely.
|
||
(Contributed by Robert Collins in :issue:`17911`.)
|
||
|
||
|
||
locale
|
||
------
|
||
|
||
A new :func:`~locale.delocalize` function can be used to convert a string into
|
||
a normalized number string, taking the ``LC_NUMERIC`` settings into account::
|
||
|
||
>>> import locale
|
||
>>> locale.setlocale(locale.LC_NUMERIC, 'de_DE.UTF-8')
|
||
'de_DE.UTF-8'
|
||
>>> locale.delocalize('1.234,56')
|
||
'1234.56'
|
||
>>> locale.setlocale(locale.LC_NUMERIC, 'en_US.UTF-8')
|
||
'en_US.UTF-8'
|
||
>>> locale.delocalize('1,234.56')
|
||
'1234.56'
|
||
|
||
(Contributed by Cédric Krier in :issue:`13918`.)
|
||
|
||
|
||
logging
|
||
-------
|
||
|
||
All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`,
|
||
:meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`,
|
||
:meth:`~logging.Logger.debug`, etc.), now accept exception instances
|
||
as an *exc_info* argument, in addition to boolean values and exception
|
||
tuples::
|
||
|
||
>>> import logging
|
||
>>> try:
|
||
... 1/0
|
||
... except ZeroDivisionError as ex:
|
||
... logging.error('exception', exc_info=ex)
|
||
ERROR:root:exception
|
||
|
||
(Contributed by Yury Selivanov in :issue:`20537`.)
|
||
|
||
The :class:`handlers.HTTPHandler <logging.handlers.HTTPHandler>` class now
|
||
accepts an optional :class:`ssl.SSLContext` instance to configure SSL
|
||
settings used in an HTTP connection.
|
||
(Contributed by Alex Gaynor in :issue:`22788`.)
|
||
|
||
The :class:`handlers.QueueListener <logging.handlers.QueueListener>` class now
|
||
takes a *respect_handler_level* keyword argument which, if set to ``True``,
|
||
will pass messages to handlers taking handler levels into account.
|
||
(Contributed by Vinay Sajip.)
|
||
|
||
|
||
lzma
|
||
----
|
||
|
||
The :meth:`LZMADecompressor.decompress() <lzma.LZMADecompressor.decompress>`
|
||
method now accepts an optional *max_length* argument to limit the maximum
|
||
size of decompressed data.
|
||
(Contributed by Martin Panter in :issue:`15955`.)
|
||
|
||
|
||
math
|
||
----
|
||
|
||
Two new constants have been added to the :mod:`math` module: :data:`~math.inf`
|
||
and :data:`~math.nan`. (Contributed by Mark Dickinson in :issue:`23185`.)
|
||
|
||
A new function :func:`~math.isclose` provides a way to test for approximate
|
||
equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.)
|
||
|
||
A new :func:`~math.gcd` function has been added. The :func:`fractions.gcd`
|
||
function is now deprecated. (Contributed by Mark Dickinson and Serhiy
|
||
Storchaka in :issue:`22486`.)
|
||
|
||
|
||
multiprocessing
|
||
---------------
|
||
|
||
:func:`sharedctypes.synchronized() <multiprocessing.sharedctypes.synchronized>`
|
||
objects now support the :term:`context manager` protocol.
|
||
(Contributed by Charles-François Natali in :issue:`21565`.)
|
||
|
||
|
||
operator
|
||
--------
|
||
|
||
:func:`~operator.attrgetter`, :func:`~operator.itemgetter`,
|
||
and :func:`~operator.methodcaller` objects now support pickling.
|
||
(Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.)
|
||
|
||
New :func:`~operator.matmul` and :func:`~operator.imatmul` functions
|
||
to perform matrix multiplication.
|
||
(Contributed by Benjamin Peterson in :issue:`21176`.)
|
||
|
||
|
||
os
|
||
--
|
||
|
||
The new :func:`~os.scandir` function returning an iterator of
|
||
:class:`~os.DirEntry` objects has been added. If possible, :func:`~os.scandir`
|
||
extracts file attributes while scanning a directory, removing the need to
|
||
perform subsequent system calls to determine file type or attributes, which may
|
||
significantly improve performance. (Contributed by Ben Hoyt with the help
|
||
of Victor Stinner in :issue:`22524`.)
|
||
|
||
On Windows, a new
|
||
:attr:`stat_result.st_file_attributes <os.stat_result.st_file_attributes>`
|
||
attribute is now available. It corresponds to the ``dwFileAttributes`` member
|
||
of the ``BY_HANDLE_FILE_INFORMATION`` structure returned by
|
||
``GetFileInformationByHandle()``. (Contributed by Ben Hoyt in :issue:`21719`.)
|
||
|
||
The :func:`~os.urandom` function now uses the ``getrandom()`` syscall on Linux 3.17
|
||
or newer, and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to
|
||
use ``/dev/urandom`` and avoiding failures due to potential file descriptor
|
||
exhaustion. (Contributed by Victor Stinner in :issue:`22181`.)
|
||
|
||
New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow
|
||
getting and setting a file descriptor's blocking mode (:data:`~os.O_NONBLOCK`.)
|
||
(Contributed by Victor Stinner in :issue:`22054`.)
|
||
|
||
The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported
|
||
on Windows. (Contributed by Steve Dower in :issue:`23668`.)
|
||
|
||
There is a new :func:`os.path.commonpath` function returning the longest
|
||
common sub-path of each passed pathname. Unlike the
|
||
:func:`os.path.commonprefix` function, it always returns a valid
|
||
path::
|
||
|
||
>>> os.path.commonprefix(['/usr/lib', '/usr/local/lib'])
|
||
'/usr/l'
|
||
|
||
>>> os.path.commonpath(['/usr/lib', '/usr/local/lib'])
|
||
'/usr'
|
||
|
||
(Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.)
|
||
|
||
|
||
pathlib
|
||
-------
|
||
|
||
The new :meth:`Path.samefile() <pathlib.Path.samefile>` method can be used
|
||
to check whether the path points to the same file as another path, which can
|
||
be either another :class:`~pathlib.Path` object, or a string::
|
||
|
||
>>> import pathlib
|
||
>>> p1 = pathlib.Path('/etc/hosts')
|
||
>>> p2 = pathlib.Path('/etc/../etc/hosts')
|
||
>>> p1.samefile(p2)
|
||
True
|
||
|
||
(Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.)
|
||
|
||
The :meth:`Path.mkdir() <pathlib.Path.mkdir>` method now accepts a new optional
|
||
*exist_ok* argument to match ``mkdir -p`` and :func:`os.makedirs`
|
||
functionality. (Contributed by Berker Peksag in :issue:`21539`.)
|
||
|
||
There is a new :meth:`Path.expanduser() <pathlib.Path.expanduser>` method to
|
||
expand ``~`` and ``~user`` prefixes. (Contributed by Serhiy Storchaka and
|
||
Claudiu Popa in :issue:`19776`.)
|
||
|
||
A new :meth:`Path.home() <pathlib.Path.home>` class method can be used to get
|
||
a :class:`~pathlib.Path` instance representing the user’s home
|
||
directory.
|
||
(Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.)
|
||
|
||
New :meth:`Path.write_text() <pathlib.Path.write_text>`,
|
||
:meth:`Path.read_text() <pathlib.Path.read_text>`,
|
||
:meth:`Path.write_bytes() <pathlib.Path.write_bytes>`,
|
||
:meth:`Path.read_bytes() <pathlib.Path.read_bytes>` methods to simplify
|
||
read/write operations on files.
|
||
|
||
The following code snippet will create or rewrite existing file
|
||
``~/spam42``::
|
||
|
||
>>> import pathlib
|
||
>>> p = pathlib.Path('~/spam42')
|
||
>>> p.expanduser().write_text('ham')
|
||
3
|
||
|
||
(Contributed by Christopher Welborn in :issue:`20218`.)
|
||
|
||
|
||
pickle
|
||
------
|
||
|
||
Nested objects, such as unbound methods or nested classes, can now be pickled
|
||
using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4.
|
||
Protocol version 4 already supports these cases. (Contributed by Serhiy
|
||
Storchaka in :issue:`23611`.)
|
||
|
||
|
||
poplib
|
||
------
|
||
|
||
A new :meth:`POP3.utf8() <poplib.POP3.utf8>` command enables :rfc:`6856`
|
||
(Internationalized Email) support, if a POP server supports it.
|
||
(Contributed by Milan OberKirch in :issue:`21804`.)
|
||
|
||
|
||
re
|
||
--
|
||
|
||
References and conditional references to groups with fixed length are now
|
||
allowed in lookbehind assertions::
|
||
|
||
>>> import re
|
||
>>> pat = re.compile(r'(a|b).(?<=\1)c')
|
||
>>> pat.match('aac')
|
||
<_sre.SRE_Match object; span=(0, 3), match='aac'>
|
||
>>> pat.match('bbc')
|
||
<_sre.SRE_Match object; span=(0, 3), match='bbc'>
|
||
|
||
(Contributed by Serhiy Storchaka in :issue:`9179`.)
|
||
|
||
The number of capturing groups in regular expressions is no longer limited to
|
||
100. (Contributed by Serhiy Storchaka in :issue:`22437`.)
|
||
|
||
The :func:`~re.sub` and :func:`~re.subn` functions now replace unmatched
|
||
groups with empty strings instead of raising an exception.
|
||
(Contributed by Serhiy Storchaka in :issue:`1519638`.)
|
||
|
||
The :class:`re.error` exceptions have new attributes,
|
||
:attr:`~re.error.msg`, :attr:`~re.error.pattern`,
|
||
:attr:`~re.error.pos`, :attr:`~re.error.lineno`,
|
||
and :attr:`~re.error.colno`, that provide better context
|
||
information about the error::
|
||
|
||
>>> re.compile("""
|
||
... (?x)
|
||
... .++
|
||
... """)
|
||
Traceback (most recent call last):
|
||
...
|
||
sre_constants.error: multiple repeat at position 16 (line 3, column 7)
|
||
|
||
(Contributed by Serhiy Storchaka in :issue:`22578`.)
|
||
|
||
|
||
readline
|
||
--------
|
||
|
||
A new :func:`~readline.append_history_file` function can be used to append
|
||
the specified number of trailing elements in history to the given file.
|
||
(Contributed by Bruno Cauet in :issue:`22940`.)
|
||
|
||
|
||
selectors
|
||
---------
|
||
|
||
The new :class:`~selectors.DevpollSelector` supports efficient
|
||
``/dev/poll`` polling on Solaris.
|
||
(Contributed by Giampaolo Rodola' in :issue:`18931`.)
|
||
|
||
|
||
shutil
|
||
------
|
||
|
||
The :func:`~shutil.move` function now accepts a *copy_function* argument,
|
||
allowing, for example, the :func:`~shutil.copy` function to be used instead of
|
||
the default :func:`~shutil.copy2` if there is a need to ignore file metadata
|
||
when moving.
|
||
(Contributed by Claudiu Popa in :issue:`19840`.)
|
||
|
||
The :func:`~shutil.make_archive` function now supports the *xztar* format.
|
||
(Contributed by Serhiy Storchaka in :issue:`5411`.)
|
||
|
||
|
||
signal
|
||
------
|
||
|
||
On Windows, the :func:`~signal.set_wakeup_fd` function now also supports
|
||
socket handles. (Contributed by Victor Stinner in :issue:`22018`.)
|
||
|
||
Various ``SIG*`` constants in the :mod:`signal` module have been converted into
|
||
:mod:`Enums <enum>`. This allows meaningful names to be printed
|
||
during debugging, instead of integer "magic numbers".
|
||
(Contributed by Giampaolo Rodola' in :issue:`21076`.)
|
||
|
||
|
||
smtpd
|
||
-----
|
||
|
||
Both the :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now
|
||
accept a *decode_data* keyword argument to determine if the ``DATA`` portion of
|
||
the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead
|
||
provided to the
|
||
:meth:`SMTPServer.process_message() <smtpd.SMTPServer.process_message>`
|
||
method as a byte string. The default is ``True`` for backward compatibility
|
||
reasons, but will change to ``False`` in Python 3.6. If *decode_data* is set
|
||
to ``False``, the ``process_message`` method must be prepared to accept keyword
|
||
arguments.
|
||
(Contributed by Maciej Szulik in :issue:`19662`.)
|
||
|
||
The :class:`~smtpd.SMTPServer` class now advertises the ``8BITMIME`` extension
|
||
(:rfc:`6152`) if *decode_data* has been set ``True``. If the client
|
||
specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to
|
||
:meth:`SMTPServer.process_message() <smtpd.SMTPServer.process_message>`
|
||
via the *mail_options* keyword.
|
||
(Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.)
|
||
|
||
The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8``
|
||
extension (:rfc:`6531`: Internationalized Email). If the client specified
|
||
``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to
|
||
:meth:`SMTPServer.process_message() <smtpd.SMTPServer.process_message>`
|
||
via the *mail_options* keyword. It is the responsibility of the
|
||
``process_message`` method to correctly handle the ``SMTPUTF8`` data.
|
||
(Contributed by Milan Oberkirch in :issue:`21725`.)
|
||
|
||
It is now possible to provide, directly or via name resolution, IPv6
|
||
addresses in the :class:`~smtpd.SMTPServer` constructor, and have it
|
||
successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.)
|
||
|
||
|
||
smtplib
|
||
-------
|
||
|
||
A new :meth:`SMTP.auth() <smtplib.SMTP.auth>` method provides a convenient way to
|
||
implement custom authentication mechanisms. (Contributed by Milan
|
||
Oberkirch in :issue:`15014`.)
|
||
|
||
The :meth:`SMTP.set_debuglevel() <smtplib.SMTP.set_debuglevel>` method now
|
||
accepts an additional debuglevel (2), which enables timestamps in debug
|
||
messages. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.)
|
||
|
||
Both the :meth:`SMTP.sendmail() <smtplib.SMTP.sendmail>` and
|
||
:meth:`SMTP.send_message() <smtplib.SMTP.send_message>` methods now
|
||
support :rfc:`6531` (SMTPUTF8).
|
||
(Contributed by Milan Oberkirch and R. David Murray in :issue:`22027`.)
|
||
|
||
|
||
sndhdr
|
||
------
|
||
|
||
The :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions now return
|
||
a :func:`~collections.namedtuple`. (Contributed by Claudiu Popa in
|
||
:issue:`18615`.)
|
||
|
||
|
||
socket
|
||
------
|
||
|
||
Functions with timeouts now use a monotonic clock, instead of a system clock.
|
||
(Contributed by Victor Stinner in :issue:`22043`.)
|
||
|
||
A new :meth:`socket.sendfile() <socket.socket.sendfile>` method allows
|
||
sending a file over a socket by using the high-performance :func:`os.sendfile`
|
||
function on UNIX, resulting in uploads being from 2 to 3 times faster than when
|
||
using plain :meth:`socket.send() <socket.socket.send>`.
|
||
(Contributed by Giampaolo Rodola' in :issue:`17552`.)
|
||
|
||
The :meth:`socket.sendall() <socket.socket.sendall>` method no longer resets the
|
||
socket timeout every time bytes are received or sent. The socket timeout is
|
||
now the maximum total duration to send all data.
|
||
(Contributed by Victor Stinner in :issue:`23853`.)
|
||
|
||
The *backlog* argument of the :meth:`socket.listen() <socket.socket.listen>`
|
||
method is now optional. By default it is set to
|
||
:data:`SOMAXCONN <socket.SOMAXCONN>` or to ``128``, whichever is less.
|
||
(Contributed by Charles-François Natali in :issue:`21455`.)
|
||
|
||
|
||
ssl
|
||
---
|
||
|
||
.. _whatsnew-sslmemorybio:
|
||
|
||
Memory BIO Support
|
||
~~~~~~~~~~~~~~~~~~
|
||
|
||
(Contributed by Geert Jansen in :issue:`21965`.)
|
||
|
||
The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol
|
||
support for cases when the network I/O capabilities of :class:`~ssl.SSLSocket`
|
||
are not necessary or are suboptimal. ``SSLObject`` represents
|
||
an SSL protocol instance, but does not implement any network I/O methods, and
|
||
instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO`
|
||
class can be used to pass data between Python and an SSL protocol instance.
|
||
|
||
The memory BIO SSL support is primarily intended to be used in frameworks
|
||
implementing asynchronous I/O for which :class:`~ssl.SSLSocket`'s readiness
|
||
model ("select/poll") is inefficient.
|
||
|
||
A new :meth:`SSLContext.wrap_bio() <ssl.SSLContext.wrap_bio>` method can be used
|
||
to create a new ``SSLObject`` instance.
|
||
|
||
|
||
Application-Layer Protocol Negotiation Support
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
(Contributed by Benjamin Peterson in :issue:`20188`.)
|
||
|
||
Where OpenSSL support is present, the :mod:`ssl` module now implements
|
||
the *Application-Layer Protocol Negotiation* TLS extension as described
|
||
in :rfc:`7301`.
|
||
|
||
The new :meth:`SSLContext.set_alpn_protocols() <ssl.SSLContext.set_alpn_protocols>`
|
||
can be used to specify which protocols a socket should advertise during
|
||
the TLS handshake.
|
||
|
||
The new
|
||
:meth:`SSLSocket.selected_alpn_protocol() <ssl.SSLSocket.selected_alpn_protocol>`
|
||
returns the protocol that was selected during the TLS handshake.
|
||
The :data:`~ssl.HAS_ALPN` flag indicates whether ALPN support is present.
|
||
|
||
|
||
Other Changes
|
||
~~~~~~~~~~~~~
|
||
|
||
There is a new :meth:`SSLSocket.version() <ssl.SSLSocket.version>` method to
|
||
query the actual protocol version in use.
|
||
(Contributed by Antoine Pitrou in :issue:`20421`.)
|
||
|
||
The :class:`~ssl.SSLSocket` class now implements
|
||
a :meth:`SSLSocket.sendfile() <ssl.SSLSocket.sendfile>` method.
|
||
(Contributed by Giampaolo Rodola' in :issue:`17552`.)
|
||
|
||
The :meth:`SSLSocket.send() <ssl.SSLSocket.send>` method now raises either
|
||
the :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a
|
||
non-blocking socket if the operation would block. Previously, it would return
|
||
``0``. (Contributed by Nikolaus Rath in :issue:`20951`.)
|
||
|
||
The :func:`~ssl.cert_time_to_seconds` function now interprets the input time
|
||
as UTC and not as local time, per :rfc:`5280`. Additionally, the return
|
||
value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.)
|
||
|
||
New :meth:`SSLObject.shared_ciphers() <ssl.SSLObject.shared_ciphers>` and
|
||
:meth:`SSLSocket.shared_ciphers() <ssl.SSLSocket.shared_ciphers>` methods return
|
||
the list of ciphers sent by the client during the handshake.
|
||
(Contributed by Benjamin Peterson in :issue:`23186`.)
|
||
|
||
The :meth:`SSLSocket.do_handshake() <ssl.SSLSocket.do_handshake>`,
|
||
:meth:`SSLSocket.read() <ssl.SSLSocket.read>`,
|
||
:meth:`SSLSocket.shutdown() <ssl.SSLSocket.shutdown>`, and
|
||
:meth:`SSLSocket.write() <ssl.SSLSocket.write>` methods of the :class:`~ssl.SSLSocket`
|
||
class no longer reset the socket timeout every time bytes are received or sent.
|
||
The socket timeout is now the maximum total duration of the method.
|
||
(Contributed by Victor Stinner in :issue:`23853`.)
|
||
|
||
The :func:`~ssl.match_hostname` function now supports matching of IP addresses.
|
||
(Contributed by Antoine Pitrou in :issue:`23239`.)
|
||
|
||
|
||
sqlite3
|
||
-------
|
||
|
||
The :class:`~sqlite3.Row` class now fully supports the sequence protocol,
|
||
in particular :func:`reversed` iteration and slice indexing.
|
||
(Contributed by Claudiu Popa in :issue:`10203`; by Lucas Sinclair,
|
||
Jessica McKellar, and Serhiy Storchaka in :issue:`13583`.)
|
||
|
||
|
||
.. _whatsnew-subprocess:
|
||
|
||
subprocess
|
||
----------
|
||
|
||
The new :func:`~subprocess.run` function has been added.
|
||
It runs the specified command and returns a
|
||
:class:`~subprocess.CompletedProcess` object, which describes a finished
|
||
process. The new API is more consistent and is the recommended approach
|
||
to invoking subprocesses in Python code that does not need to maintain
|
||
compatibility with earlier Python versions.
|
||
(Contributed by Thomas Kluyver in :issue:`23342`.)
|
||
|
||
Examples::
|
||
|
||
>>> subprocess.run(["ls", "-l"]) # doesn't capture output
|
||
CompletedProcess(args=['ls', '-l'], returncode=0)
|
||
|
||
>>> subprocess.run("exit 1", shell=True, check=True)
|
||
Traceback (most recent call last):
|
||
...
|
||
subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
|
||
|
||
>>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE)
|
||
CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0,
|
||
stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n')
|
||
|
||
|
||
sys
|
||
---
|
||
|
||
A new :func:`~sys.set_coroutine_wrapper` function allows setting a global
|
||
hook that will be called whenever a :term:`coroutine object <coroutine>`
|
||
is created by an :keyword:`async def` function. A corresponding
|
||
:func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set
|
||
wrapper. Both functions are :term:`provisional <provisional API>`,
|
||
and are intended for debugging purposes only. (Contributed by Yury Selivanov
|
||
in :issue:`24017`.)
|
||
|
||
A new :func:`~sys.is_finalizing` function can be used to check if the Python
|
||
interpreter is :term:`shutting down <interpreter shutdown>`.
|
||
(Contributed by Antoine Pitrou in :issue:`22696`.)
|
||
|
||
|
||
sysconfig
|
||
---------
|
||
|
||
The name of the user scripts directory on Windows now includes the first
|
||
two components of the Python version. (Contributed by Paul Moore
|
||
in :issue:`23437`.)
|
||
|
||
|
||
tarfile
|
||
-------
|
||
|
||
The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"``
|
||
to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.)
|
||
|
||
The :meth:`TarFile.extractall() <tarfile.TarFile.extractall>` and
|
||
:meth:`TarFile.extract() <tarfile.TarFile.extract>` methods now take a keyword
|
||
argument *numeric_owner*. If set to ``True``, the extracted files and
|
||
directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile.
|
||
If set to ``False`` (the default, and the behavior in versions prior to 3.5),
|
||
they will be owned by the named user and group in the tarfile.
|
||
(Contributed by Michael Vogt and Eric Smith in :issue:`23193`.)
|
||
|
||
The :meth:`TarFile.list() <tarfile.TarFile.list>` now accepts an optional
|
||
*members* keyword argument that can be set to a subset of the list returned
|
||
by :meth:`TarFile.getmembers() <tarfile.TarFile.getmembers>`.
|
||
(Contributed by Serhiy Storchaka in :issue:`21549`.)
|
||
|
||
|
||
threading
|
||
---------
|
||
|
||
Both the :meth:`Lock.acquire() <threading.Lock.acquire>` and
|
||
:meth:`RLock.acquire() <threading.RLock.acquire>` methods
|
||
now use a monotonic clock for timeout management.
|
||
(Contributed by Victor Stinner in :issue:`22043`.)
|
||
|
||
|
||
time
|
||
----
|
||
|
||
The :func:`~time.monotonic` function is now always available.
|
||
(Contributed by Victor Stinner in :issue:`22043`.)
|
||
|
||
|
||
timeit
|
||
------
|
||
|
||
A new command line option ``-u`` or :samp:`--unit={U}` can be used to specify the time
|
||
unit for the timer output. Supported options are ``usec``, ``msec``,
|
||
or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.)
|
||
|
||
The :func:`~timeit.timeit` function has a new *globals* parameter for
|
||
specifying the namespace in which the code will be running.
|
||
(Contributed by Ben Roberts in :issue:`2527`.)
|
||
|
||
|
||
tkinter
|
||
-------
|
||
|
||
The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment
|
||
on Windows has been replaced by a private function in the :mod:`_tkinter`
|
||
module which makes no permanent changes to environment variables.
|
||
(Contributed by Zachary Ware in :issue:`20035`.)
|
||
|
||
|
||
.. _whatsnew-traceback:
|
||
|
||
traceback
|
||
---------
|
||
|
||
New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb`
|
||
functions to conveniently traverse frame and traceback objects.
|
||
(Contributed by Robert Collins in :issue:`17911`.)
|
||
|
||
New lightweight classes: :class:`~traceback.TracebackException`,
|
||
:class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`.
|
||
(Contributed by Robert Collins in :issue:`17911`.)
|
||
|
||
Both the :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions
|
||
now support negative values for the *limit* argument.
|
||
(Contributed by Dmitry Kazakov in :issue:`22619`.)
|
||
|
||
|
||
types
|
||
-----
|
||
|
||
A new :func:`~types.coroutine` function to transform
|
||
:term:`generator <generator iterator>` and
|
||
:class:`generator-like <collections.abc.Generator>` objects into
|
||
:term:`awaitables <awaitable>`.
|
||
(Contributed by Yury Selivanov in :issue:`24017`.)
|
||
|
||
A new type called :class:`~types.CoroutineType`, which is used for
|
||
:term:`coroutine` objects created by :keyword:`async def` functions.
|
||
(Contributed by Yury Selivanov in :issue:`24400`.)
|
||
|
||
|
||
unicodedata
|
||
-----------
|
||
|
||
The :mod:`unicodedata` module now uses data from `Unicode 8.0.0
|
||
<http://unicode.org/versions/Unicode8.0.0/>`_.
|
||
|
||
|
||
unittest
|
||
--------
|
||
|
||
The :meth:`TestLoader.loadTestsFromModule() <unittest.TestLoader.loadTestsFromModule>`
|
||
method now accepts a keyword-only argument *pattern* which is passed to
|
||
``load_tests`` as the third argument. Found packages are now checked for
|
||
``load_tests`` regardless of whether their path matches *pattern*, because it
|
||
is impossible for a package name to match the default pattern.
|
||
(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.)
|
||
|
||
Unittest discovery errors now are exposed in the
|
||
:data:`TestLoader.errors <unittest.TestLoader.errors>` attribute of the
|
||
:class:`~unittest.TestLoader` instance.
|
||
(Contributed by Robert Collins in :issue:`19746`.)
|
||
|
||
A new command line option ``--locals`` to show local variables in
|
||
tracebacks. (Contributed by Robert Collins in :issue:`22936`.)
|
||
|
||
|
||
unittest.mock
|
||
-------------
|
||
|
||
The :class:`~unittest.mock.Mock` class has the following improvements:
|
||
|
||
* The class constructor has a new *unsafe* parameter, which causes mock
|
||
objects to raise :exc:`AttributeError` on attribute names starting
|
||
with ``"assert"``.
|
||
(Contributed by Kushal Das in :issue:`21238`.)
|
||
|
||
* A new :meth:`Mock.assert_not_called() <unittest.mock.Mock.assert_not_called>`
|
||
method to check if the mock object was called.
|
||
(Contributed by Kushal Das in :issue:`21262`.)
|
||
|
||
The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`,
|
||
:meth:`__divmod__` and :meth:`__matmul__` operators.
|
||
(Contributed by Johannes Baiter in :issue:`20968`, and Håkan Lövdahl
|
||
in :issue:`23581` and :issue:`23568`.)
|
||
|
||
It is no longer necessary to explicitly pass ``create=True`` to the
|
||
:func:`~unittest.mock.patch` function when patching builtin names.
|
||
(Contributed by Kushal Das in :issue:`17660`.)
|
||
|
||
|
||
urllib
|
||
------
|
||
|
||
A new
|
||
:class:`request.HTTPPasswordMgrWithPriorAuth <urllib.request.HTTPPasswordMgrWithPriorAuth>`
|
||
class allows HTTP Basic Authentication credentials to be managed so as to
|
||
eliminate unnecessary ``401`` response handling, or to unconditionally send
|
||
credentials on the first request in order to communicate with servers that
|
||
return a ``404`` response instead of a ``401`` if the ``Authorization`` header
|
||
is not sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in
|
||
:issue:`7159`.)
|
||
|
||
A new *quote_via* argument for the
|
||
:func:`parse.urlencode() <urllib.parse.urlencode>`
|
||
function provides a way to control the encoding of query parts if needed.
|
||
(Contributed by Samwyse and Arnon Yaari in :issue:`13866`.)
|
||
|
||
The :func:`request.urlopen() <urllib.request.urlopen>` function accepts an
|
||
:class:`ssl.SSLContext` object as a *context* argument, which will be used for
|
||
the HTTPS connection. (Contributed by Alex Gaynor in :issue:`22366`.)
|
||
|
||
The :func:`parse.urljoin() <urllib.parse.urljoin>` was updated to use the
|
||
:rfc:`3986` semantics for the resolution of relative URLs, rather than
|
||
:rfc:`1808` and :rfc:`2396`.
|
||
(Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.)
|
||
|
||
|
||
wsgiref
|
||
-------
|
||
|
||
The *headers* argument of the :class:`headers.Headers <wsgiref.headers.Headers>`
|
||
class constructor is now optional.
|
||
(Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.)
|
||
|
||
|
||
xmlrpc
|
||
------
|
||
|
||
The :class:`client.ServerProxy <xmlrpc.client.ServerProxy>` class now supports
|
||
the :term:`context manager` protocol.
|
||
(Contributed by Claudiu Popa in :issue:`20627`.)
|
||
|
||
The :class:`client.ServerProxy <xmlrpc.client.ServerProxy>` constructor now accepts
|
||
an optional :class:`ssl.SSLContext` instance.
|
||
(Contributed by Alex Gaynor in :issue:`22960`.)
|
||
|
||
|
||
xml.sax
|
||
-------
|
||
|
||
SAX parsers now support a character stream of the
|
||
:class:`xmlreader.InputSource <xml.sax.xmlreader.InputSource>` object.
|
||
(Contributed by Serhiy Storchaka in :issue:`2175`.)
|
||
|
||
:func:`~xml.sax.parseString` now accepts a :class:`str` instance.
|
||
(Contributed by Serhiy Storchaka in :issue:`10590`.)
|
||
|
||
|
||
zipfile
|
||
-------
|
||
|
||
ZIP output can now be written to unseekable streams.
|
||
(Contributed by Serhiy Storchaka in :issue:`23252`.)
|
||
|
||
The *mode* argument of :meth:`ZipFile.open() <zipfile.ZipFile.open>` method now
|
||
accepts ``"x"`` to request exclusive creation.
|
||
(Contributed by Serhiy Storchaka in :issue:`21717`.)
|
||
|
||
|
||
Other module-level changes
|
||
==========================
|
||
|
||
Many functions in the :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`,
|
||
:mod:`ssl`, and :mod:`codecs` modules now accept writable
|
||
:term:`bytes-like objects <bytes-like object>`.
|
||
(Contributed by Serhiy Storchaka in :issue:`23001`.)
|
||
|
||
|
||
Optimizations
|
||
=============
|
||
|
||
The :func:`os.walk` function has been sped up by 3 to 5 times on POSIX systems,
|
||
and by 7 to 20 times on Windows. This was done using the new :func:`os.scandir`
|
||
function, which exposes file information from the underlying ``readdir`` or
|
||
``FindFirstFile``/``FindNextFile`` system calls. (Contributed by
|
||
Ben Hoyt with help from Victor Stinner in :issue:`23605`.)
|
||
|
||
Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less
|
||
memory for large objects. ``calloc()`` is used instead of ``malloc()`` to
|
||
allocate memory for these objects.
|
||
(Contributed by Victor Stinner in :issue:`21233`.)
|
||
|
||
Some operations on :mod:`ipaddress` :class:`~ipaddress.IPv4Network` and
|
||
:class:`~ipaddress.IPv6Network` have been massively sped up, such as
|
||
:meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`,
|
||
:func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`.
|
||
The speed up can range from 3 to 15 times.
|
||
(Contributed by Antoine Pitrou, Michel Albert, and Markus in
|
||
:issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.)
|
||
|
||
Pickling of :mod:`ipaddress` objects was optimized to produce significantly
|
||
smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.)
|
||
|
||
Many operations on :class:`io.BytesIO` are now 50% to 100% faster.
|
||
(Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in
|
||
:issue:`22003`.)
|
||
|
||
The :func:`marshal.dumps` function is now faster: 65--85% with versions 3
|
||
and 4, 20--25% with versions 0 to 2 on typical data, and up to 5 times in
|
||
best cases.
|
||
(Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.)
|
||
|
||
The UTF-32 encoder is now 3 to 7 times faster.
|
||
(Contributed by Serhiy Storchaka in :issue:`15027`.)
|
||
|
||
Regular expressions are now parsed up to 10% faster.
|
||
(Contributed by Serhiy Storchaka in :issue:`19380`.)
|
||
|
||
The :func:`json.dumps` function was optimized to run with
|
||
``ensure_ascii=False`` as fast as with ``ensure_ascii=True``.
|
||
(Contributed by Naoki Inada in :issue:`23206`.)
|
||
|
||
The :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass`
|
||
functions have been sped up in the common case that the second argument
|
||
has :class:`type` as its metaclass.
|
||
(Contributed Georg Brandl by in :issue:`22540`.)
|
||
|
||
Method caching was slightly improved, yielding up to 5% performance
|
||
improvement in some benchmarks.
|
||
(Contributed by Antoine Pitrou in :issue:`22847`.)
|
||
|
||
Objects from the :mod:`random` module now use 50% less memory on 64-bit
|
||
builds. (Contributed by Serhiy Storchaka in :issue:`23488`.)
|
||
|
||
The :func:`property` getter calls are up to 25% faster.
|
||
(Contributed by Joe Jevnik in :issue:`23910`.)
|
||
|
||
Instantiation of :class:`fractions.Fraction` is now up to 30% faster.
|
||
(Contributed by Stefan Behnel in :issue:`22464`.)
|
||
|
||
String methods :meth:`~str.find`, :meth:`~str.rfind`, :meth:`~str.split`,
|
||
:meth:`~str.partition` and the :keyword:`in` string operator are now significantly
|
||
faster for searching 1-character substrings.
|
||
(Contributed by Serhiy Storchaka in :issue:`23573`.)
|
||
|
||
|
||
Build and C API Changes
|
||
=======================
|
||
|
||
New ``calloc`` functions were added:
|
||
|
||
* :c:func:`PyMem_RawCalloc`,
|
||
* :c:func:`PyMem_Calloc`,
|
||
* :c:func:`PyObject_Calloc`.
|
||
|
||
(Contributed by Victor Stinner in :issue:`21233`.)
|
||
|
||
New encoding/decoding helper functions:
|
||
|
||
* :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``),
|
||
* :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``).
|
||
|
||
(Contributed by Victor Stinner in :issue:`18395`.)
|
||
|
||
A new :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode
|
||
encode error with ``\N{...}`` escapes.
|
||
(Contributed by Serhiy Storchaka in :issue:`19676`.)
|
||
|
||
A new :c:func:`PyErr_FormatV` function similar to :c:func:`PyErr_Format`,
|
||
but accepts a ``va_list`` argument.
|
||
(Contributed by Antoine Pitrou in :issue:`18711`.)
|
||
|
||
A new :c:data:`PyExc_RecursionError` exception.
|
||
(Contributed by Georg Brandl in :issue:`19235`.)
|
||
|
||
New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`,
|
||
and :c:func:`PyModule_ExecDef` functions introduced by :pep:`489` --
|
||
multi-phase extension module initialization.
|
||
(Contributed by Petr Viktorin in :issue:`24268`.)
|
||
|
||
New :c:func:`PyNumber_MatrixMultiply` and
|
||
:c:func:`PyNumber_InPlaceMatrixMultiply` functions to perform matrix
|
||
multiplication.
|
||
(Contributed by Benjamin Peterson in :issue:`21176`. See also :pep:`465`
|
||
for details.)
|
||
|
||
The :c:member:`PyTypeObject.tp_finalize` slot is now part of the stable ABI.
|
||
|
||
Windows builds now require Microsoft Visual C++ 14.0, which
|
||
is available as part of `Visual Studio 2015 <https://www.visualstudio.com/>`_.
|
||
|
||
Extension modules now include a platform information tag in their filename on
|
||
some platforms (the tag is optional, and CPython will import extensions without
|
||
it, although if the tag is present and mismatched, the extension won't be
|
||
loaded):
|
||
|
||
* On Linux, extension module filenames end with
|
||
``.cpython-<major><minor>m-<architecture>-<os>.pyd``:
|
||
|
||
* ``<major>`` is the major number of the Python version;
|
||
for Python 3.5 this is ``3``.
|
||
|
||
* ``<minor>`` is the minor number of the Python version;
|
||
for Python 3.5 this is ``5``.
|
||
|
||
* ``<architecture>`` is the hardware architecture the extension module
|
||
was built to run on. It's most commonly either ``i386`` for 32-bit Intel
|
||
platforms or ``x86_64`` for 64-bit Intel (and AMD) platforms.
|
||
|
||
* ``<os>`` is always ``linux-gnu``, except for extensions built to
|
||
talk to the 32-bit ABI on 64-bit platforms, in which case it is
|
||
``linux-gnu32`` (and ``<architecture>`` will be ``x86_64``).
|
||
|
||
* On Windows, extension module filenames end with
|
||
``<debug>.cp<major><minor>-<platform>.pyd``:
|
||
|
||
* ``<major>`` is the major number of the Python version;
|
||
for Python 3.5 this is ``3``.
|
||
|
||
* ``<minor>`` is the minor number of the Python version;
|
||
for Python 3.5 this is ``5``.
|
||
|
||
* ``<platform>`` is the platform the extension module was built for,
|
||
either ``win32`` for Win32, ``win_amd64`` for Win64, ``win_ia64`` for
|
||
Windows Itanium 64, and ``win_arm`` for Windows on ARM.
|
||
|
||
* If built in debug mode, ``<debug>`` will be ``_d``,
|
||
otherwise it will be blank.
|
||
|
||
* On OS X platforms, extension module filenames now end with ``-darwin.so``.
|
||
|
||
* On all other platforms, extension module filenames are the same as they were
|
||
with Python 3.4.
|
||
|
||
|
||
Deprecated
|
||
==========
|
||
|
||
New Keywords
|
||
------------
|
||
|
||
``async`` and ``await`` are not recommended to be used as variable, class,
|
||
function or module names. Introduced by :pep:`492` in Python 3.5, they will
|
||
become proper keywords in Python 3.7.
|
||
|
||
|
||
Deprecated Python Behavior
|
||
--------------------------
|
||
|
||
Raising the :exc:`StopIteration` exception inside a generator will now generate a silent
|
||
:exc:`PendingDeprecationWarning`, which will become a non-silent deprecation
|
||
warning in Python 3.6 and will trigger a :exc:`RuntimeError` in Python 3.7.
|
||
See :ref:`PEP 479: Change StopIteration handling inside generators <whatsnew-pep-479>`
|
||
for details.
|
||
|
||
|
||
Unsupported Operating Systems
|
||
-----------------------------
|
||
|
||
Windows XP is no longer supported by Microsoft, thus, per :PEP:`11`, CPython
|
||
3.5 is no longer officially supported on this OS.
|
||
|
||
|
||
Deprecated Python modules, functions and methods
|
||
------------------------------------------------
|
||
|
||
The :mod:`formatter` module has now graduated to full deprecation and is still
|
||
slated for removal in Python 3.6.
|
||
|
||
The :func:`asyncio.async` function is deprecated in favor of
|
||
:func:`~asyncio.ensure_future`.
|
||
|
||
The :mod:`smtpd` module has in the past always decoded the DATA portion of
|
||
email messages using the ``utf-8`` codec. This can now be controlled by the
|
||
new *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is
|
||
``True``, but this default is deprecated. Specify the *decode_data* keyword
|
||
with an appropriate value to avoid the deprecation warning.
|
||
|
||
Directly assigning values to the :attr:`~http.cookies.Morsel.key`,
|
||
:attr:`~http.cookies.Morsel.value` and
|
||
:attr:`~http.cookies.Morsel.coded_value` of :class:`http.cookies.Morsel`
|
||
objects is deprecated. Use the :meth:`~http.cookies.Morsel.set` method
|
||
instead. In addition, the undocumented *LegalChars* parameter of
|
||
:meth:`~http.cookies.Morsel.set` is deprecated, and is now ignored.
|
||
|
||
Passing a format string as keyword argument *format_string* to the
|
||
:meth:`~string.Formatter.format` method of the :class:`string.Formatter`
|
||
class has been deprecated.
|
||
(Contributed by Serhiy Storchaka in :issue:`23671`.)
|
||
|
||
The :func:`platform.dist` and :func:`platform.linux_distribution` functions
|
||
are now deprecated. Linux distributions use too many different ways of
|
||
describing themselves, so the functionality is left to a package.
|
||
(Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.)
|
||
|
||
The previously undocumented ``from_function`` and ``from_builtin`` methods of
|
||
:class:`inspect.Signature` are deprecated. Use the new
|
||
:meth:`Signature.from_callable() <inspect.Signature.from_callable>`
|
||
method instead. (Contributed by Yury Selivanov in :issue:`24248`.)
|
||
|
||
The :func:`inspect.getargspec` function is deprecated and scheduled to be
|
||
removed in Python 3.6. (See :issue:`20438` for details.)
|
||
|
||
The :mod:`inspect` :func:`~inspect.getfullargspec`,
|
||
:func:`~inspect.getcallargs`, and :func:`~inspect.formatargspec` functions are
|
||
deprecated in favor of the :func:`inspect.signature` API. (Contributed by Yury
|
||
Selivanov in :issue:`20438`.)
|
||
|
||
:func:`~inspect.getargvalues` and :func:`~inspect.formatargvalues` functions
|
||
were inadvertently marked as deprecated with the release of Python 3.5.0.
|
||
|
||
Use of :const:`re.LOCALE` flag with str patterns or :const:`re.ASCII` is now
|
||
deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.)
|
||
|
||
Use of unrecognized special sequences consisting of ``'\'`` and an ASCII letter
|
||
in regular expression patterns and replacement patterns now raises a
|
||
deprecation warning and will be forbidden in Python 3.6.
|
||
(Contributed by Serhiy Storchaka in :issue:`23622`.)
|
||
|
||
The undocumented and unofficial *use_load_tests* default argument of the
|
||
:meth:`unittest.TestLoader.loadTestsFromModule` method now is
|
||
deprecated and ignored.
|
||
(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.)
|
||
|
||
|
||
Removed
|
||
=======
|
||
|
||
API and Feature Removals
|
||
------------------------
|
||
|
||
The following obsolete and previously deprecated APIs and features have been
|
||
removed:
|
||
|
||
* The ``__version__`` attribute has been dropped from the email package. The
|
||
email code hasn't been shipped separately from the stdlib for a long time,
|
||
and the ``__version__`` string was not updated in the last few releases.
|
||
|
||
* The internal ``Netrc`` class in the :mod:`ftplib` module was deprecated in
|
||
3.4, and has now been removed.
|
||
(Contributed by Matt Chaput in :issue:`6623`.)
|
||
|
||
* The concept of ``.pyo`` files has been removed.
|
||
|
||
* The JoinableQueue class in the provisional :mod:`asyncio` module was
|
||
deprecated in 3.4.4 and is now removed.
|
||
(Contributed by A. Jesse Jiryu Davis in :issue:`23464`.)
|
||
|
||
|
||
Porting to Python 3.5
|
||
=====================
|
||
|
||
This section lists previously described changes and other bugfixes
|
||
that may require changes to your code.
|
||
|
||
|
||
Changes in Python behavior
|
||
--------------------------
|
||
|
||
* Due to an oversight, earlier Python versions erroneously accepted the
|
||
following syntax::
|
||
|
||
f(1 for x in [1], *args)
|
||
f(1 for x in [1], **kwargs)
|
||
|
||
Python 3.5 now correctly raises a :exc:`SyntaxError`, as generator
|
||
expressions must be put in parentheses if not a sole argument to a function.
|
||
|
||
|
||
Changes in the Python API
|
||
-------------------------
|
||
|
||
* :pep:`475`: System calls are now retried when interrupted by a signal instead
|
||
of raising :exc:`InterruptedError` if the Python signal handler does not
|
||
raise an exception.
|
||
|
||
* Before Python 3.5, a :class:`datetime.time` object was considered to be false
|
||
if it represented midnight in UTC. This behavior was considered obscure and
|
||
error-prone and has been removed in Python 3.5. See :issue:`13936` for full
|
||
details.
|
||
|
||
* The :meth:`ssl.SSLSocket.send()` method now raises either
|
||
:exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError`
|
||
on a non-blocking socket if the operation would block. Previously,
|
||
it would return ``0``. (Contributed by Nikolaus Rath in :issue:`20951`.)
|
||
|
||
* The ``__name__`` attribute of generators is now set from the function name,
|
||
instead of being set from the code name. Use ``gen.gi_code.co_name`` to
|
||
retrieve the code name. Generators also have a new ``__qualname__``
|
||
attribute, the qualified name, which is now used for the representation
|
||
of a generator (``repr(gen)``).
|
||
(Contributed by Victor Stinner in :issue:`21205`.)
|
||
|
||
* The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`,
|
||
:meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been
|
||
removed. (Contributed by Ezio Melotti in :issue:`15114`.)
|
||
The *convert_charrefs* argument of :class:`~html.parser.HTMLParser` is
|
||
now ``True`` by default. (Contributed by Berker Peksag in :issue:`21047`.)
|
||
|
||
* Although it is not formally part of the API, it is worth noting for porting
|
||
purposes (ie: fixing tests) that error messages that were previously of the
|
||
form "'sometype' does not support the buffer protocol" are now of the form "a
|
||
:term:`bytes-like object` is required, not 'sometype'".
|
||
(Contributed by Ezio Melotti in :issue:`16518`.)
|
||
|
||
* If the current directory is set to a directory that no longer exists then
|
||
:exc:`FileNotFoundError` will no longer be raised and instead
|
||
:meth:`~importlib.machinery.FileFinder.find_spec` will return ``None``
|
||
**without** caching ``None`` in :data:`sys.path_importer_cache`, which is
|
||
different than the typical case (:issue:`22834`).
|
||
|
||
* HTTP status code and messages from :mod:`http.client` and :mod:`http.server`
|
||
were refactored into a common :class:`~http.HTTPStatus` enum. The values in
|
||
:mod:`http.client` and :mod:`http.server` remain available for backwards
|
||
compatibility. (Contributed by Demian Brecht in :issue:`21793`.)
|
||
|
||
* When an import loader defines :meth:`importlib.machinery.Loader.exec_module`
|
||
it is now expected to also define
|
||
:meth:`~importlib.machinery.Loader.create_module` (raises a
|
||
:exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader
|
||
inherits from :class:`importlib.abc.Loader` then there is nothing to do, else
|
||
simply define :meth:`~importlib.machinery.Loader.create_module` to return
|
||
``None``. (Contributed by Brett Cannon in :issue:`23014`.)
|
||
|
||
* The :func:`re.split` function always ignored empty pattern matches, so the
|
||
``"x*"`` pattern worked the same as ``"x+"``, and the ``"\b"`` pattern never
|
||
worked. Now :func:`re.split` raises a warning if the pattern could match
|
||
an empty string. For compatibility, use patterns that never match an empty
|
||
string (e.g. ``"x+"`` instead of ``"x*"``). Patterns that could only match
|
||
an empty string (such as ``"\b"``) now raise an error.
|
||
(Contributed by Serhiy Storchaka in :issue:`22818`.)
|
||
|
||
* The :class:`http.cookies.Morsel` dict-like interface has been made self
|
||
consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key`
|
||
and :attr:`~http.cookies.Morsel.value` into account,
|
||
:meth:`~http.cookies.Morsel.copy` now results in a
|
||
:class:`~http.cookies.Morsel` instance rather than a :class:`dict`, and
|
||
:meth:`~http.cookies.Morsel.update` will now raise an exception if any of the
|
||
keys in the update dictionary are invalid. In addition, the undocumented
|
||
*LegalChars* parameter of :func:`~http.cookies.Morsel.set` is deprecated and
|
||
is now ignored. (Contributed by Demian Brecht in :issue:`2211`.)
|
||
|
||
* :pep:`488` has removed ``.pyo`` files from Python and introduced the optional
|
||
``opt-`` tag in ``.pyc`` file names. The
|
||
:func:`importlib.util.cache_from_source` has gained an *optimization*
|
||
parameter to help control the ``opt-`` tag. Because of this, the
|
||
*debug_override* parameter of the function is now deprecated. `.pyo` files
|
||
are also no longer supported as a file argument to the Python interpreter and
|
||
thus serve no purpose when distributed on their own (i.e. sourceless code
|
||
distribution). Due to the fact that the magic number for bytecode has changed
|
||
in Python 3.5, all old `.pyo` files from previous versions of Python are
|
||
invalid regardless of this PEP.
|
||
|
||
* The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES`
|
||
constant on linux 3.6 and greater.
|
||
|
||
* The :func:`ssl.cert_time_to_seconds` function now interprets the input time
|
||
as UTC and not as local time, per :rfc:`5280`. Additionally, the return
|
||
value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.)
|
||
|
||
* The ``pygettext.py`` Tool now uses the standard +NNNN format for timezones in
|
||
the POT-Creation-Date header.
|
||
|
||
* The :mod:`smtplib` module now uses :data:`sys.stderr` instead of the previous
|
||
module-level :data:`stderr` variable for debug output. If your (test)
|
||
program depends on patching the module-level variable to capture the debug
|
||
output, you will need to update it to capture sys.stderr instead.
|
||
|
||
* The :meth:`str.startswith` and :meth:`str.endswith` methods no longer return
|
||
``True`` when finding the empty string and the indexes are completely out of
|
||
range. (Contributed by Serhiy Storchaka in :issue:`24284`.)
|
||
|
||
* The :func:`inspect.getdoc` function now returns documentation strings
|
||
inherited from base classes. Documentation strings no longer need to be
|
||
duplicated if the inherited documentation is appropriate. To suppress an
|
||
inherited string, an empty string must be specified (or the documentation
|
||
may be filled in). This change affects the output of the :mod:`pydoc`
|
||
module and the :func:`help` function.
|
||
(Contributed by Serhiy Storchaka in :issue:`15582`.)
|
||
|
||
* Nested :func:`functools.partial` calls are now flattened. If you were
|
||
relying on the previous behavior, you can now either add an attribute to a
|
||
:func:`functools.partial` object or you can create a subclass of
|
||
:func:`functools.partial`.
|
||
(Contributed by Alexander Belopolsky in :issue:`7830`.)
|
||
|
||
Changes in the C API
|
||
--------------------
|
||
|
||
* The undocumented :c:member:`~PyMemoryViewObject.format` member of the
|
||
(non-public) :c:type:`PyMemoryViewObject` structure has been removed.
|
||
All extensions relying on the relevant parts in ``memoryobject.h``
|
||
must be rebuilt.
|
||
|
||
* The :c:type:`PyMemAllocator` structure was renamed to
|
||
:c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added.
|
||
|
||
* Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references.
|
||
Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions
|
||
to format the :func:`repr` of the object.
|
||
(Contributed by Serhiy Storchaka in :issue:`22453`.)
|
||
|
||
* Because the lack of the :attr:`__module__` attribute breaks pickling and
|
||
introspection, a deprecation warning is now raised for builtin types without
|
||
the :attr:`__module__` attribute. This would be an AttributeError in
|
||
the future.
|
||
(Contributed by Serhiy Storchaka in :issue:`20204`.)
|
||
|
||
* As part of the :pep:`492` implementation, the ``tp_reserved`` slot of
|
||
:c:type:`PyTypeObject` was replaced with a
|
||
:c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for
|
||
new types, structures and functions.
|
||
|
||
|
||
Notable changes in Python 3.5.4
|
||
===============================
|
||
|
||
New ``make regen-all`` build target
|
||
-----------------------------------
|
||
|
||
To simplify cross-compilation, and to ensure that CPython can reliably be
|
||
compiled without requiring an existing version of Python to already be
|
||
available, the autotools-based build system no longer attempts to implicitly
|
||
recompile generated files based on file modification times.
|
||
|
||
Instead, a new ``make regen-all`` command has been added to force regeneration
|
||
of these files when desired (e.g. after an initial version of Python has
|
||
already been built based on the pregenerated versions).
|
||
|
||
More selective regeneration targets are also defined - see
|
||
:source:`Makefile.pre.in` for details.
|
||
|
||
(Contributed by Victor Stinner in :issue:`23404`.)
|
||
|
||
.. versionadded:: 3.5.4
|
||
|
||
|
||
Removal of ``make touch`` build target
|
||
--------------------------------------
|
||
|
||
The ``make touch`` build target previously used to request implicit regeneration
|
||
of generated files by updating their modification times has been removed.
|
||
|
||
It has been replaced by the new ``make regen-all`` target.
|
||
|
||
(Contributed by Victor Stinner in :issue:`23404`.)
|
||
|
||
.. versionchanged:: 3.5.4
|
||
|