2572 lines
98 KiB
ReStructuredText
2572 lines
98 KiB
ReStructuredText
****************************
|
|
What's New In Python 3.7
|
|
****************************
|
|
|
|
:Editor: Elvis Pranskevichus <elvis@magic.io>
|
|
|
|
.. Rules for maintenance:
|
|
|
|
* Anyone can add text to this document. Do not spend very much time
|
|
on the wording of your changes, because your text will probably
|
|
get rewritten to some degree.
|
|
|
|
* The maintainer will go through Misc/NEWS periodically and add
|
|
changes; it's therefore more important to add your changes to
|
|
Misc/NEWS than to this file.
|
|
|
|
* This is not a complete list of every single change; completeness
|
|
is the purpose of Misc/NEWS. Some changes I consider too small
|
|
or esoteric to include. If such a change is added to the text,
|
|
I'll just remove it. (This is another reason you shouldn't spend
|
|
too much time on writing your addition.)
|
|
|
|
* If you want to draw your new text to the attention of the
|
|
maintainer, add 'XXX' to the beginning of the paragraph or
|
|
section.
|
|
|
|
* It's OK to just add a fragmentary note about a change. For
|
|
example: "XXX Describe the transmogrify() function added to the
|
|
socket module." The maintainer will research the change and
|
|
write the necessary text.
|
|
|
|
* You can comment out your additions if you like, but it's not
|
|
necessary (especially when a final release is some months away).
|
|
|
|
* Credit the author of a patch or bugfix. Just the name is
|
|
sufficient; the e-mail address isn't necessary.
|
|
|
|
* It's helpful to add the bug/patch number as a comment:
|
|
|
|
XXX Describe the transmogrify() function added to the socket
|
|
module.
|
|
(Contributed by P.Y. Developer in :issue:`12345`.)
|
|
|
|
This saves the maintainer the effort of going through the Git log
|
|
when researching a change.
|
|
|
|
This article explains the new features in Python 3.7, compared to 3.6.
|
|
Python 3.7 was released on June 27, 2018.
|
|
For full details, see the :ref:`changelog <changelog>`.
|
|
|
|
|
|
Summary -- Release Highlights
|
|
=============================
|
|
|
|
.. This section singles out the most important changes in Python 3.7.
|
|
Brevity is key.
|
|
|
|
New syntax features:
|
|
|
|
* :ref:`PEP 563 <whatsnew37-pep563>`, postponed evaluation of type annotations.
|
|
|
|
Backwards incompatible syntax changes:
|
|
|
|
* :keyword:`async` and :keyword:`await` are now reserved keywords.
|
|
|
|
New library modules:
|
|
|
|
* :mod:`contextvars`: :ref:`PEP 567 -- Context Variables <whatsnew37-pep567>`
|
|
* :mod:`dataclasses`: :ref:`PEP 557 -- Data Classes <whatsnew37-pep557>`
|
|
* :ref:`whatsnew37_importlib_resources`
|
|
|
|
New built-in features:
|
|
|
|
* :ref:`PEP 553 <whatsnew37-pep553>`, the new :func:`breakpoint` function.
|
|
|
|
Python data model improvements:
|
|
|
|
* :ref:`PEP 562 <whatsnew37-pep562>`, customization of access to
|
|
module attributes.
|
|
|
|
* :ref:`PEP 560 <whatsnew37-pep560>`, core support for typing module and
|
|
generic types.
|
|
|
|
* the insertion-order preservation nature of :ref:`dict <typesmapping>`
|
|
objects `has been declared`_ to be an official
|
|
part of the Python language spec.
|
|
|
|
.. _has been declared: https://mail.python.org/pipermail/python-dev/2017-December/151283.html
|
|
|
|
Significant improvements in the standard library:
|
|
|
|
* The :mod:`asyncio` module has received new features, significant
|
|
:ref:`usability and performance improvements <whatsnew37_asyncio>`.
|
|
|
|
* The :mod:`time` module gained support for
|
|
:ref:`functions with nanosecond resolution <whatsnew37-pep564>`.
|
|
|
|
CPython implementation improvements:
|
|
|
|
* Avoiding the use of ASCII as a default text encoding:
|
|
|
|
* :ref:`PEP 538 <whatsnew37-pep538>`, legacy C locale coercion
|
|
* :ref:`PEP 540 <whatsnew37-pep540>`, forced UTF-8 runtime mode
|
|
* :ref:`PEP 552 <whatsnew37-pep552>`, deterministic .pycs
|
|
* :ref:`New Python Development Mode <whatsnew37-devmode>`
|
|
* :ref:`PEP 565 <whatsnew37-pep565>`, improved :exc:`DeprecationWarning`
|
|
handling
|
|
|
|
C API improvements:
|
|
|
|
* :ref:`PEP 539 <whatsnew37-pep539>`, new C API for thread-local storage
|
|
|
|
Documentation improvements:
|
|
|
|
* :ref:`PEP 545 <whatsnew37-pep545>`, Python documentation translations
|
|
* New documentation translations: `Japanese <https://docs.python.org/ja/>`_,
|
|
`French <https://docs.python.org/fr/>`_, and
|
|
`Korean <https://docs.python.org/ko/>`_.
|
|
|
|
This release features notable performance improvements in many areas.
|
|
The :ref:`whatsnew37-perf` section lists them in detail.
|
|
|
|
For a list of changes that may affect compatibility with previous Python
|
|
releases please refer to the :ref:`porting-to-python-37` section.
|
|
|
|
|
|
New Features
|
|
============
|
|
|
|
.. _whatsnew37-pep563:
|
|
|
|
PEP 563: Postponed Evaluation of Annotations
|
|
--------------------------------------------
|
|
|
|
The advent of type hints in Python uncovered two glaring usability issues
|
|
with the functionality of annotations added in :pep:`3107` and refined
|
|
further in :pep:`526`:
|
|
|
|
* annotations could only use names which were already available in the
|
|
current scope, in other words they didn't support forward references
|
|
of any kind; and
|
|
|
|
* annotating source code had adverse effects on startup time of Python
|
|
programs.
|
|
|
|
Both of these issues are fixed by postponing the evaluation of
|
|
annotations. Instead of compiling code which executes expressions in
|
|
annotations at their definition time, the compiler stores the annotation
|
|
in a string form equivalent to the AST of the expression in question.
|
|
If needed, annotations can be resolved at runtime using
|
|
:func:`typing.get_type_hints`. In the common case where this is not
|
|
required, the annotations are cheaper to store (since short strings
|
|
are interned by the interpreter) and make startup time faster.
|
|
|
|
Usability-wise, annotations now support forward references, making the
|
|
following syntax valid::
|
|
|
|
class C:
|
|
@classmethod
|
|
def from_string(cls, source: str) -> C:
|
|
...
|
|
|
|
def validate_b(self, obj: B) -> bool:
|
|
...
|
|
|
|
class B:
|
|
...
|
|
|
|
Since this change breaks compatibility, the new behavior needs to be enabled
|
|
on a per-module basis in Python 3.7 using a :mod:`__future__` import::
|
|
|
|
from __future__ import annotations
|
|
|
|
It will become the default in Python 3.10.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`563` -- Postponed evaluation of annotations
|
|
PEP written and implemented by Łukasz Langa.
|
|
|
|
|
|
.. _whatsnew37-pep538:
|
|
|
|
PEP 538: Legacy C Locale Coercion
|
|
---------------------------------
|
|
|
|
An ongoing challenge within the Python 3 series has been determining a sensible
|
|
default strategy for handling the "7-bit ASCII" text encoding assumption
|
|
currently implied by the use of the default C or POSIX locale on non-Windows
|
|
platforms.
|
|
|
|
:pep:`538` updates the default interpreter command line interface to
|
|
automatically coerce that locale to an available UTF-8 based locale as
|
|
described in the documentation of the new :envvar:`PYTHONCOERCECLOCALE`
|
|
environment variable. Automatically setting ``LC_CTYPE`` this way means that
|
|
both the core interpreter and locale-aware C extensions (such as
|
|
:mod:`readline`) will assume the use of UTF-8 as the default text encoding,
|
|
rather than ASCII.
|
|
|
|
The platform support definition in :pep:`11` has also been updated to limit
|
|
full text handling support to suitably configured non-ASCII based locales.
|
|
|
|
As part of this change, the default error handler for :data:`~sys.stdin` and
|
|
:data:`~sys.stdout` is now ``surrogateescape`` (rather than ``strict``) when
|
|
using any of the defined coercion target locales (currently ``C.UTF-8``,
|
|
``C.utf8``, and ``UTF-8``). The default error handler for :data:`~sys.stderr`
|
|
continues to be ``backslashreplace``, regardless of locale.
|
|
|
|
Locale coercion is silent by default, but to assist in debugging potentially
|
|
locale related integration problems, explicit warnings (emitted directly on
|
|
:data:`~sys.stderr`) can be requested by setting ``PYTHONCOERCECLOCALE=warn``.
|
|
This setting will also cause the Python runtime to emit a warning if the
|
|
legacy C locale remains active when the core interpreter is initialized.
|
|
|
|
While :pep:`538`'s locale coercion has the benefit of also affecting extension
|
|
modules (such as GNU ``readline``), as well as child processes (including those
|
|
running non-Python applications and older versions of Python), it has the
|
|
downside of requiring that a suitable target locale be present on the running
|
|
system. To better handle the case where no suitable target locale is available
|
|
(as occurs on RHEL/CentOS 7, for example), Python 3.7 also implements
|
|
:ref:`whatsnew37-pep540`.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`538` -- Coercing the legacy C locale to a UTF-8 based locale
|
|
PEP written and implemented by Nick Coghlan.
|
|
|
|
|
|
.. _whatsnew37-pep540:
|
|
|
|
PEP 540: Forced UTF-8 Runtime Mode
|
|
-----------------------------------
|
|
|
|
The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8`
|
|
environment variable can be used to enable the CPython *UTF-8 mode*.
|
|
|
|
When in UTF-8 mode, CPython ignores the locale settings, and uses the
|
|
UTF-8 encoding by default. The error handlers for :data:`sys.stdin` and
|
|
:data:`sys.stdout` streams are set to ``surrogateescape``.
|
|
|
|
The forced UTF-8 mode can be used to change the text handling behavior in
|
|
an embedded Python interpreter without changing the locale settings of
|
|
an embedding application.
|
|
|
|
While :pep:`540`'s UTF-8 mode has the benefit of working regardless of which
|
|
locales are available on the running system, it has the downside of having no
|
|
effect on extension modules (such as GNU ``readline``), child processes running
|
|
non-Python applications, and child processes running older versions of Python.
|
|
To reduce the risk of corrupting text data when communicating with such
|
|
components, Python 3.7 also implements :ref:`whatsnew37-pep540`).
|
|
|
|
The UTF-8 mode is enabled by default when the locale is ``C`` or ``POSIX``, and
|
|
the :pep:`538` locale coercion feature fails to change it to a UTF-8 based
|
|
alternative (whether that failure is due to ``PYTHONCOERCECLOCALE=0`` being set,
|
|
``LC_ALL`` being set, or the lack of a suitable target locale).
|
|
|
|
.. seealso::
|
|
|
|
:pep:`540` -- Add a new UTF-8 mode
|
|
PEP written and implemented by Victor Stinner
|
|
|
|
|
|
.. _whatsnew37-pep553:
|
|
|
|
PEP 553: Built-in ``breakpoint()``
|
|
----------------------------------
|
|
|
|
Python 3.7 includes the new built-in :func:`breakpoint` function as
|
|
an easy and consistent way to enter the Python debugger.
|
|
|
|
Built-in ``breakpoint()`` calls :func:`sys.breakpointhook`. By default, the
|
|
latter imports :mod:`pdb` and then calls ``pdb.set_trace()``, but by binding
|
|
``sys.breakpointhook()`` to the function of your choosing, ``breakpoint()`` can
|
|
enter any debugger. Additionally, the environment variable
|
|
:envvar:`PYTHONBREAKPOINT` can be set to the callable of your debugger of
|
|
choice. Set ``PYTHONBREAKPOINT=0`` to completely disable built-in
|
|
``breakpoint()``.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`553` -- Built-in breakpoint()
|
|
PEP written and implemented by Barry Warsaw
|
|
|
|
|
|
.. _whatsnew37-pep539:
|
|
|
|
PEP 539: New C API for Thread-Local Storage
|
|
-------------------------------------------
|
|
|
|
While Python provides a C API for thread-local storage support; the existing
|
|
:ref:`Thread Local Storage (TLS) API <thread-local-storage-api>` has used
|
|
:c:type:`int` to represent TLS keys across all platforms. This has not
|
|
generally been a problem for officially-support platforms, but that is neither
|
|
POSIX-compliant, nor portable in any practical sense.
|
|
|
|
:pep:`539` changes this by providing a new :ref:`Thread Specific Storage (TSS)
|
|
API <thread-specific-storage-api>` to CPython which supersedes use of the
|
|
existing TLS API within the CPython interpreter, while deprecating the existing
|
|
API. The TSS API uses a new type :c:type:`Py_tss_t` instead of :c:type:`int`
|
|
to represent TSS keys--an opaque type the definition of which may depend on
|
|
the underlying TLS implementation. Therefore, this will allow to build CPython
|
|
on platforms where the native TLS key is defined in a way that cannot be safely
|
|
cast to :c:type:`int`.
|
|
|
|
Note that on platforms where the native TLS key is defined in a way that cannot
|
|
be safely cast to :c:type:`int`, all functions of the existing TLS API will be
|
|
no-op and immediately return failure. This indicates clearly that the old API
|
|
is not supported on platforms where it cannot be used reliably, and that no
|
|
effort will be made to add such support.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`539` -- A New C-API for Thread-Local Storage in CPython
|
|
PEP written by Erik M. Bray; implementation by Masayuki Yamamoto.
|
|
|
|
|
|
.. _whatsnew37-pep562:
|
|
|
|
PEP 562: Customization of Access to Module Attributes
|
|
-----------------------------------------------------
|
|
|
|
Python 3.7 allows defining :meth:`__getattr__` on modules and will call
|
|
it whenever a module attribute is otherwise not found. Defining
|
|
:meth:`__dir__` on modules is now also allowed.
|
|
|
|
A typical example of where this may be useful is module attribute deprecation
|
|
and lazy loading.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`562` -- Module ``__getattr__`` and ``__dir__``
|
|
PEP written and implemented by Ivan Levkivskyi
|
|
|
|
|
|
.. _whatsnew37-pep564:
|
|
|
|
PEP 564: New Time Functions With Nanosecond Resolution
|
|
------------------------------------------------------
|
|
|
|
The resolution of clocks in modern systems can exceed the limited precision
|
|
of a floating point number returned by the :func:`time.time` function
|
|
and its variants. To avoid loss of precision, :pep:`564` adds six new
|
|
"nanosecond" variants of the existing timer functions to the :mod:`time`
|
|
module:
|
|
|
|
* :func:`time.clock_gettime_ns`
|
|
* :func:`time.clock_settime_ns`
|
|
* :func:`time.monotonic_ns`
|
|
* :func:`time.perf_counter_ns`
|
|
* :func:`time.process_time_ns`
|
|
* :func:`time.time_ns`
|
|
|
|
The new functions return the number of nanoseconds as an integer value.
|
|
|
|
`Measurements <https://www.python.org/dev/peps/pep-0564/#annex-clocks-resolution-in-python>`_
|
|
show that on Linux and Windows the resolution of :func:`time.time_ns` is
|
|
approximately 3 times better than that of :func:`time.time`.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`564` -- Add new time functions with nanosecond resolution
|
|
PEP written and implemented by Victor Stinner
|
|
|
|
|
|
.. _whatsnew37-pep565:
|
|
|
|
PEP 565: Show DeprecationWarning in ``__main__``
|
|
------------------------------------------------
|
|
|
|
The default handling of :exc:`DeprecationWarning` has been changed such that
|
|
these warnings are once more shown by default, but only when the code
|
|
triggering them is running directly in the :mod:`__main__` module. As a result,
|
|
developers of single file scripts and those using Python interactively should
|
|
once again start seeing deprecation warnings for the APIs they use, but
|
|
deprecation warnings triggered by imported application, library and framework
|
|
modules will continue to be hidden by default.
|
|
|
|
As a result of this change, the standard library now allows developers to choose
|
|
between three different deprecation warning behaviours:
|
|
|
|
* :exc:`FutureWarning`: always displayed by default, recommended for warnings
|
|
intended to be seen by application end users (e.g. for deprecated application
|
|
configuration settings).
|
|
* :exc:`DeprecationWarning`: displayed by default only in :mod:`__main__` and when
|
|
running tests, recommended for warnings intended to be seen by other Python
|
|
developers where a version upgrade may result in changed behaviour or an
|
|
error.
|
|
* :exc:`PendingDeprecationWarning`: displayed by default only when running
|
|
tests, intended for cases where a future version upgrade will change the
|
|
warning category to :exc:`DeprecationWarning` or :exc:`FutureWarning`.
|
|
|
|
Previously both :exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning`
|
|
were only visible when running tests, which meant that developers primarily
|
|
writing single file scripts or using Python interactively could be surprised
|
|
by breaking changes in the APIs they used.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`565` -- Show DeprecationWarning in ``__main__``
|
|
PEP written and implemented by Nick Coghlan
|
|
|
|
|
|
.. _whatsnew37-pep560:
|
|
|
|
PEP 560: Core Support for ``typing`` module and Generic Types
|
|
-------------------------------------------------------------
|
|
|
|
Initially :pep:`484` was designed in such way that it would not introduce *any*
|
|
changes to the core CPython interpreter. Now type hints and the :mod:`typing`
|
|
module are extensively used by the community, so this restriction is removed.
|
|
The PEP introduces two special methods :meth:`__class_getitem__` and
|
|
``__mro_entries__``, these methods are now used by most classes and special
|
|
constructs in :mod:`typing`. As a result, the speed of various operations
|
|
with types increased up to 7 times, the generic types can be used without
|
|
metaclass conflicts, and several long standing bugs in :mod:`typing` module are
|
|
fixed.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`560` -- Core support for typing module and generic types
|
|
PEP written and implemented by Ivan Levkivskyi
|
|
|
|
|
|
.. _whatsnew37-pep552:
|
|
|
|
PEP 552: Hash-based .pyc Files
|
|
------------------------------
|
|
|
|
Python has traditionally checked the up-to-dateness of bytecode cache files
|
|
(i.e., ``.pyc`` files) by comparing the source metadata (last-modified timestamp
|
|
and size) with source metadata saved in the cache file header when it was
|
|
generated. While effective, this invalidation method has its drawbacks. When
|
|
filesystem timestamps are too coarse, Python can miss source updates, leading to
|
|
user confusion. Additionally, having a timestamp in the cache file is
|
|
problematic for `build reproducibility <https://reproducible-builds.org/>`_ and
|
|
content-based build systems.
|
|
|
|
:pep:`552` extends the pyc format to allow the hash of the source file to be
|
|
used for invalidation instead of the source timestamp. Such ``.pyc`` files are
|
|
called "hash-based". By default, Python still uses timestamp-based invalidation
|
|
and does not generate hash-based ``.pyc`` files at runtime. Hash-based ``.pyc``
|
|
files may be generated with :mod:`py_compile` or :mod:`compileall`.
|
|
|
|
Hash-based ``.pyc`` files come in two variants: checked and unchecked. Python
|
|
validates checked hash-based ``.pyc`` files against the corresponding source
|
|
files at runtime but doesn't do so for unchecked hash-based pycs. Unchecked
|
|
hash-based ``.pyc`` files are a useful performance optimization for environments
|
|
where a system external to Python (e.g., the build system) is responsible for
|
|
keeping ``.pyc`` files up-to-date.
|
|
|
|
See :ref:`pyc-invalidation` for more information.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`552` -- Deterministic pycs
|
|
PEP written and implemented by Benjamin Peterson
|
|
|
|
|
|
.. _whatsnew37-pep545:
|
|
|
|
PEP 545: Python Documentation Translations
|
|
------------------------------------------
|
|
|
|
:pep:`545` describes the process of creating and maintaining Python
|
|
documentation translations.
|
|
|
|
Three new translations have been added:
|
|
|
|
- Japanese: https://docs.python.org/ja/
|
|
- French: https://docs.python.org/fr/
|
|
- Korean: https://docs.python.org/ko/
|
|
|
|
.. seealso::
|
|
|
|
:pep:`545` -- Python Documentation Translations
|
|
PEP written and implemented by Julien Palard, Inada Naoki, and
|
|
Victor Stinner.
|
|
|
|
|
|
.. _whatsnew37-devmode:
|
|
|
|
Python Development Mode (-X dev)
|
|
--------------------------------
|
|
|
|
The new :option:`-X` ``dev`` command line option or the new
|
|
:envvar:`PYTHONDEVMODE` environment variable can be used to enable
|
|
:ref:`Python Development Mode <devmode>`. When in development mode, Python performs
|
|
additional runtime checks that are too expensive to be enabled by default.
|
|
See :ref:`Python Development Mode <devmode>` documentation for the full
|
|
description.
|
|
|
|
|
|
Other Language Changes
|
|
======================
|
|
|
|
* An :keyword:`await` expression and comprehensions containing an
|
|
:keyword:`async for` clause were illegal in the expressions in
|
|
:ref:`formatted string literals <f-strings>` due to a problem with the
|
|
implementation. In Python 3.7 this restriction was lifted.
|
|
|
|
* More than 255 arguments can now be passed to a function, and a function can
|
|
now have more than 255 parameters. (Contributed by Serhiy Storchaka in
|
|
:issue:`12844` and :issue:`18896`.)
|
|
|
|
* :meth:`bytes.fromhex` and :meth:`bytearray.fromhex` now ignore all ASCII
|
|
whitespace, not only spaces. (Contributed by Robert Xiao in :issue:`28927`.)
|
|
|
|
* :class:`str`, :class:`bytes`, and :class:`bytearray` gained support for
|
|
the new :meth:`isascii() <str.isascii>` method, which can be used to
|
|
test if a string or bytes contain only the ASCII characters.
|
|
(Contributed by INADA Naoki in :issue:`32677`.)
|
|
|
|
* :exc:`ImportError` now displays module name and module ``__file__`` path when
|
|
``from ... import ...`` fails. (Contributed by Matthias Bussonnier in
|
|
:issue:`29546`.)
|
|
|
|
* Circular imports involving absolute imports with binding a submodule to
|
|
a name are now supported.
|
|
(Contributed by Serhiy Storchaka in :issue:`30024`.)
|
|
|
|
* ``object.__format__(x, '')`` is now equivalent to ``str(x)`` rather than
|
|
``format(str(self), '')``.
|
|
(Contributed by Serhiy Storchaka in :issue:`28974`.)
|
|
|
|
* In order to better support dynamic creation of stack traces,
|
|
:class:`types.TracebackType` can now be instantiated from Python code, and
|
|
the ``tb_next`` attribute on :ref:`tracebacks <traceback-objects>` is now
|
|
writable.
|
|
(Contributed by Nathaniel J. Smith in :issue:`30579`.)
|
|
|
|
* When using the :option:`-m` switch, ``sys.path[0]`` is now eagerly expanded
|
|
to the full starting directory path, rather than being left as the empty
|
|
directory (which allows imports from the *current* working directory at the
|
|
time when an import occurs)
|
|
(Contributed by Nick Coghlan in :issue:`33053`.)
|
|
|
|
* The new :option:`-X` ``importtime`` option or the
|
|
:envvar:`PYTHONPROFILEIMPORTTIME` environment variable can be used to show
|
|
the timing of each module import.
|
|
(Contributed by Victor Stinner in :issue:`31415`.)
|
|
|
|
|
|
New Modules
|
|
===========
|
|
|
|
.. _whatsnew37-pep567:
|
|
|
|
contextvars
|
|
-----------
|
|
|
|
The new :mod:`contextvars` module and a set of
|
|
:ref:`new C APIs <contextvarsobjects>` introduce
|
|
support for *context variables*. Context variables are conceptually
|
|
similar to thread-local variables. Unlike TLS, context variables
|
|
support asynchronous code correctly.
|
|
|
|
The :mod:`asyncio` and :mod:`decimal` modules have been updated to use
|
|
and support context variables out of the box. Particularly the active
|
|
decimal context is now stored in a context variable, which allows
|
|
decimal operations to work with the correct context in asynchronous code.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`567` -- Context Variables
|
|
PEP written and implemented by Yury Selivanov
|
|
|
|
|
|
.. _whatsnew37-pep557:
|
|
|
|
dataclasses
|
|
-----------
|
|
|
|
The new :func:`~dataclasses.dataclass` decorator provides a way to declare
|
|
*data classes*. A data class describes its attributes using class variable
|
|
annotations. Its constructor and other magic methods, such as
|
|
:meth:`~object.__repr__`, :meth:`~object.__eq__`, and
|
|
:meth:`~object.__hash__` are generated automatically.
|
|
|
|
Example::
|
|
|
|
@dataclass
|
|
class Point:
|
|
x: float
|
|
y: float
|
|
z: float = 0.0
|
|
|
|
p = Point(1.5, 2.5)
|
|
print(p) # produces "Point(x=1.5, y=2.5, z=0.0)"
|
|
|
|
.. seealso::
|
|
|
|
:pep:`557` -- Data Classes
|
|
PEP written and implemented by Eric V. Smith
|
|
|
|
|
|
.. _whatsnew37_importlib_resources:
|
|
|
|
importlib.resources
|
|
-------------------
|
|
|
|
The new :mod:`importlib.resources` module provides several new APIs and one
|
|
new ABC for access to, opening, and reading *resources* inside packages.
|
|
Resources are roughly similar to files inside packages, but they needn't
|
|
be actual files on the physical file system. Module loaders can provide a
|
|
:meth:`get_resource_reader()` function which returns
|
|
a :class:`importlib.abc.ResourceReader` instance to support this
|
|
new API. Built-in file path loaders and zip file loaders both support this.
|
|
|
|
Contributed by Barry Warsaw and Brett Cannon in :issue:`32248`.
|
|
|
|
.. seealso::
|
|
|
|
`importlib_resources <http://importlib-resources.readthedocs.io/en/latest/>`_
|
|
-- a PyPI backport for earlier Python versions.
|
|
|
|
|
|
Improved Modules
|
|
================
|
|
|
|
|
|
argparse
|
|
--------
|
|
|
|
The new :meth:`ArgumentParser.parse_intermixed_args()
|
|
<argparse.ArgumentParser.parse_intermixed_args>`
|
|
method allows intermixing options and positional arguments.
|
|
(Contributed by paul.j3 in :issue:`14191`.)
|
|
|
|
|
|
.. _whatsnew37_asyncio:
|
|
|
|
asyncio
|
|
-------
|
|
|
|
The :mod:`asyncio` module has received many new features, usability and
|
|
:ref:`performance improvements <whatsnew37-asyncio-perf>`. Notable changes
|
|
include:
|
|
|
|
* The new :term:`provisional <provisional API>` :func:`asyncio.run` function can
|
|
be used to run a coroutine from synchronous code by automatically creating and
|
|
destroying the event loop.
|
|
(Contributed by Yury Selivanov in :issue:`32314`.)
|
|
|
|
* asyncio gained support for :mod:`contextvars`.
|
|
:meth:`loop.call_soon() <asyncio.loop.call_soon>`,
|
|
:meth:`loop.call_soon_threadsafe() <asyncio.loop.call_soon_threadsafe>`,
|
|
:meth:`loop.call_later() <asyncio.loop.call_later>`,
|
|
:meth:`loop.call_at() <asyncio.loop.call_at>`, and
|
|
:meth:`Future.add_done_callback() <asyncio.Future.add_done_callback>`
|
|
have a new optional keyword-only *context* parameter.
|
|
:class:`Tasks <asyncio.Task>` now track their context automatically.
|
|
See :pep:`567` for more details.
|
|
(Contributed by Yury Selivanov in :issue:`32436`.)
|
|
|
|
* The new :func:`asyncio.create_task` function has been added as a shortcut
|
|
to ``asyncio.get_event_loop().create_task()``.
|
|
(Contributed by Andrew Svetlov in :issue:`32311`.)
|
|
|
|
* The new :meth:`loop.start_tls() <asyncio.loop.start_tls>`
|
|
method can be used to upgrade an existing connection to TLS.
|
|
(Contributed by Yury Selivanov in :issue:`23749`.)
|
|
|
|
* The new :meth:`loop.sock_recv_into() <asyncio.loop.sock_recv_into>`
|
|
method allows reading data from a socket directly into a provided buffer making
|
|
it possible to reduce data copies.
|
|
(Contributed by Antoine Pitrou in :issue:`31819`.)
|
|
|
|
* The new :func:`asyncio.current_task` function returns the currently running
|
|
:class:`~asyncio.Task` instance, and the new :func:`asyncio.all_tasks`
|
|
function returns a set of all existing ``Task`` instances in a given loop.
|
|
The :meth:`Task.current_task() <asyncio.Task.current_task>` and
|
|
:meth:`Task.all_tasks() <asyncio.Task.all_tasks>` methods have been deprecated.
|
|
(Contributed by Andrew Svetlov in :issue:`32250`.)
|
|
|
|
* The new *provisional* :class:`~asyncio.BufferedProtocol` class allows
|
|
implementing streaming protocols with manual control over the receive buffer.
|
|
(Contributed by Yury Selivanov in :issue:`32251`.)
|
|
|
|
* The new :func:`asyncio.get_running_loop` function returns the currently
|
|
running loop, and raises a :exc:`RuntimeError` if no loop is running.
|
|
This is in contrast with :func:`asyncio.get_event_loop`, which will *create*
|
|
a new event loop if none is running.
|
|
(Contributed by Yury Selivanov in :issue:`32269`.)
|
|
|
|
* The new :meth:`StreamWriter.wait_closed() <asyncio.StreamWriter.wait_closed>`
|
|
coroutine method allows waiting until the stream writer is closed. The new
|
|
:meth:`StreamWriter.is_closing() <asyncio.StreamWriter.is_closing>` method
|
|
can be used to determine if the writer is closing.
|
|
(Contributed by Andrew Svetlov in :issue:`32391`.)
|
|
|
|
* The new :meth:`loop.sock_sendfile() <asyncio.loop.sock_sendfile>`
|
|
coroutine method allows sending files using :mod:`os.sendfile` when possible.
|
|
(Contributed by Andrew Svetlov in :issue:`32410`.)
|
|
|
|
* The new :meth:`Future.get_loop() <asyncio.Future.get_loop>` and
|
|
``Task.get_loop()`` methods return the instance of the loop on which a task or
|
|
a future were created.
|
|
:meth:`Server.get_loop() <asyncio.Server.get_loop>` allows doing the same for
|
|
:class:`asyncio.Server` objects.
|
|
(Contributed by Yury Selivanov in :issue:`32415` and
|
|
Srinivas Reddy Thatiparthy in :issue:`32418`.)
|
|
|
|
* It is now possible to control how instances of :class:`asyncio.Server` begin
|
|
serving. Previously, the server would start serving immediately when created.
|
|
The new *start_serving* keyword argument to
|
|
:meth:`loop.create_server() <asyncio.loop.create_server>` and
|
|
:meth:`loop.create_unix_server() <asyncio.loop.create_unix_server>`,
|
|
as well as :meth:`Server.start_serving() <asyncio.Server.start_serving>`, and
|
|
:meth:`Server.serve_forever() <asyncio.Server.serve_forever>`
|
|
can be used to decouple server instantiation and serving. The new
|
|
:meth:`Server.is_serving() <asyncio.Server.is_serving>` method returns ``True``
|
|
if the server is serving. :class:`~asyncio.Server` objects are now
|
|
asynchronous context managers::
|
|
|
|
srv = await loop.create_server(...)
|
|
|
|
async with srv:
|
|
# some code
|
|
|
|
# At this point, srv is closed and no longer accepts new connections.
|
|
|
|
(Contributed by Yury Selivanov in :issue:`32662`.)
|
|
|
|
* Callback objects returned by
|
|
:func:`loop.call_later() <asyncio.loop.call_later>`
|
|
gained the new :meth:`when() <asyncio.TimerHandle.when>` method which
|
|
returns an absolute scheduled callback timestamp.
|
|
(Contributed by Andrew Svetlov in :issue:`32741`.)
|
|
|
|
* The :meth:`loop.create_datagram_endpoint() \
|
|
<asyncio.loop.create_datagram_endpoint>` method
|
|
gained support for Unix sockets.
|
|
(Contributed by Quentin Dawans in :issue:`31245`.)
|
|
|
|
* The :func:`asyncio.open_connection`, :func:`asyncio.start_server` functions,
|
|
:meth:`loop.create_connection() <asyncio.loop.create_connection>`,
|
|
:meth:`loop.create_server() <asyncio.loop.create_server>`,
|
|
:meth:`loop.create_accepted_socket() <asyncio.loop.connect_accepted_socket>`
|
|
methods and their corresponding UNIX socket variants now accept the
|
|
*ssl_handshake_timeout* keyword argument.
|
|
(Contributed by Neil Aspinall in :issue:`29970`.)
|
|
|
|
* The new :meth:`Handle.cancelled() <asyncio.Handle.cancelled>` method returns
|
|
``True`` if the callback was cancelled.
|
|
(Contributed by Marat Sharafutdinov in :issue:`31943`.)
|
|
|
|
* The asyncio source has been converted to use the
|
|
:keyword:`async`/:keyword:`await` syntax.
|
|
(Contributed by Andrew Svetlov in :issue:`32193`.)
|
|
|
|
* The new :meth:`ReadTransport.is_reading() <asyncio.ReadTransport.is_reading>`
|
|
method can be used to determine the reading state of the transport.
|
|
Additionally, calls to
|
|
:meth:`ReadTransport.resume_reading() <asyncio.ReadTransport.resume_reading>`
|
|
and :meth:`ReadTransport.pause_reading() <asyncio.ReadTransport.pause_reading>`
|
|
are now idempotent.
|
|
(Contributed by Yury Selivanov in :issue:`32356`.)
|
|
|
|
* Loop methods which accept socket paths now support passing
|
|
:term:`path-like objects <path-like object>`.
|
|
(Contributed by Yury Selivanov in :issue:`32066`.)
|
|
|
|
* In :mod:`asyncio` TCP sockets on Linux are now created with ``TCP_NODELAY``
|
|
flag set by default.
|
|
(Contributed by Yury Selivanov and Victor Stinner in :issue:`27456`.)
|
|
|
|
* Exceptions occurring in cancelled tasks are no longer logged.
|
|
(Contributed by Yury Selivanov in :issue:`30508`.)
|
|
|
|
* New ``WindowsSelectorEventLoopPolicy`` and
|
|
``WindowsProactorEventLoopPolicy`` classes.
|
|
(Contributed by Yury Selivanov in :issue:`33792`.)
|
|
|
|
Several ``asyncio`` APIs have been
|
|
:ref:`deprecated <whatsnew37-asyncio-deprecated>`.
|
|
|
|
|
|
binascii
|
|
--------
|
|
|
|
The :func:`~binascii.b2a_uu` function now accepts an optional *backtick*
|
|
keyword argument. When it's true, zeros are represented by ``'`'``
|
|
instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.)
|
|
|
|
|
|
calendar
|
|
--------
|
|
|
|
The :class:`~calendar.HTMLCalendar` class has new class attributes which ease
|
|
the customization of CSS classes in the produced HTML calendar.
|
|
(Contributed by Oz Tiram in :issue:`30095`.)
|
|
|
|
|
|
collections
|
|
-----------
|
|
|
|
``collections.namedtuple()`` now supports default values.
|
|
(Contributed by Raymond Hettinger in :issue:`32320`.)
|
|
|
|
|
|
compileall
|
|
----------
|
|
|
|
:func:`compileall.compile_dir` learned the new *invalidation_mode* parameter,
|
|
which can be used to enable
|
|
:ref:`hash-based .pyc invalidation <whatsnew37-pep552>`. The invalidation
|
|
mode can also be specified on the command line using the new
|
|
``--invalidation-mode`` argument.
|
|
(Contributed by Benjamin Peterson in :issue:`31650`.)
|
|
|
|
|
|
concurrent.futures
|
|
------------------
|
|
|
|
:class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>` and
|
|
:class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>` now
|
|
support the new *initializer* and *initargs* constructor arguments.
|
|
(Contributed by Antoine Pitrou in :issue:`21423`.)
|
|
|
|
The :class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>`
|
|
can now take the multiprocessing context via the new *mp_context* argument.
|
|
(Contributed by Thomas Moreau in :issue:`31540`.)
|
|
|
|
|
|
contextlib
|
|
----------
|
|
|
|
The new :func:`~contextlib.nullcontext` is a simpler and faster no-op
|
|
context manager than :class:`~contextlib.ExitStack`.
|
|
(Contributed by Jesse-Bakker in :issue:`10049`.)
|
|
|
|
The new :func:`~contextlib.asynccontextmanager`,
|
|
:class:`~contextlib.AbstractAsyncContextManager`, and
|
|
:class:`~contextlib.AsyncExitStack` have been added to
|
|
complement their synchronous counterparts. (Contributed
|
|
by Jelle Zijlstra in :issue:`29679` and :issue:`30241`,
|
|
and by Alexander Mohr and Ilya Kulakov in :issue:`29302`.)
|
|
|
|
|
|
cProfile
|
|
--------
|
|
|
|
The :mod:`cProfile` command line now accepts ``-m module_name`` as an
|
|
alternative to script path. (Contributed by Sanyam Khurana in :issue:`21862`.)
|
|
|
|
|
|
crypt
|
|
-----
|
|
|
|
The :mod:`crypt` module now supports the Blowfish hashing method.
|
|
(Contributed by Serhiy Storchaka in :issue:`31664`.)
|
|
|
|
The :func:`~crypt.mksalt` function now allows specifying the number of rounds
|
|
for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.)
|
|
|
|
|
|
datetime
|
|
--------
|
|
|
|
The new :meth:`datetime.fromisoformat() <datetime.datetime.fromisoformat>`
|
|
method constructs a :class:`~datetime.datetime` object from a string
|
|
in one of the formats output by
|
|
:meth:`datetime.isoformat() <datetime.datetime.isoformat>`.
|
|
(Contributed by Paul Ganssle in :issue:`15873`.)
|
|
|
|
The :class:`tzinfo <datetime.tzinfo>` class now supports sub-minute offsets.
|
|
(Contributed by Alexander Belopolsky in :issue:`5288`.)
|
|
|
|
|
|
dbm
|
|
---
|
|
|
|
:mod:`dbm.dumb` now supports reading read-only files and no longer writes the
|
|
index file when it is not changed.
|
|
|
|
|
|
decimal
|
|
-------
|
|
|
|
The :mod:`decimal` module now uses :ref:`context variables <whatsnew37-pep567>`
|
|
to store the decimal context.
|
|
(Contributed by Yury Selivanov in :issue:`32630`.)
|
|
|
|
|
|
dis
|
|
---
|
|
|
|
The :func:`~dis.dis` function is now able to
|
|
disassemble nested code objects (the code of comprehensions, generator
|
|
expressions and nested functions, and the code used for building nested
|
|
classes). The maximum depth of disassembly recursion is controlled by
|
|
the new *depth* parameter.
|
|
(Contributed by Serhiy Storchaka in :issue:`11822`.)
|
|
|
|
|
|
distutils
|
|
---------
|
|
|
|
``README.rst`` is now included in the list of distutils standard READMEs and
|
|
therefore included in source distributions.
|
|
(Contributed by Ryan Gonzalez in :issue:`11913`.)
|
|
|
|
|
|
enum
|
|
----
|
|
|
|
The :class:`Enum <enum.Enum>` learned the new ``_ignore_`` class property,
|
|
which allows listing the names of properties which should not become
|
|
enum members.
|
|
(Contributed by Ethan Furman in :issue:`31801`.)
|
|
|
|
In Python 3.8, attempting to check for non-Enum objects in :class:`Enum`
|
|
classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly,
|
|
attempting to check for non-Flag objects in a :class:`Flag` member will
|
|
raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations
|
|
return :const:`False` instead and are deprecated.
|
|
(Contributed by Ethan Furman in :issue:`33217`.)
|
|
|
|
|
|
functools
|
|
---------
|
|
|
|
:func:`functools.singledispatch` now supports registering implementations
|
|
using type annotations.
|
|
(Contributed by Łukasz Langa in :issue:`32227`.)
|
|
|
|
|
|
gc
|
|
--
|
|
|
|
The new :func:`gc.freeze` function allows freezing all objects tracked
|
|
by the garbage collector and excluding them from future collections.
|
|
This can be used before a POSIX ``fork()`` call to make the GC copy-on-write
|
|
friendly or to speed up collection. The new :func:`gc.unfreeze` functions
|
|
reverses this operation. Additionally, :func:`gc.get_freeze_count` can
|
|
be used to obtain the number of frozen objects.
|
|
(Contributed by Li Zekun in :issue:`31558`.)
|
|
|
|
|
|
hmac
|
|
----
|
|
|
|
The :mod:`hmac` module now has an optimized one-shot :func:`~hmac.digest`
|
|
function, which is up to three times faster than :func:`~hmac.HMAC`.
|
|
(Contributed by Christian Heimes in :issue:`32433`.)
|
|
|
|
|
|
http.client
|
|
-----------
|
|
|
|
:class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection`
|
|
now support the new *blocksize* argument for improved upload throughput.
|
|
(Contributed by Nir Soffer in :issue:`31945`.)
|
|
|
|
|
|
http.server
|
|
-----------
|
|
|
|
:class:`~http.server.SimpleHTTPRequestHandler` now supports the HTTP
|
|
``If-Modified-Since`` header. The server returns the 304 response status if
|
|
the target file was not modified after the time specified in the header.
|
|
(Contributed by Pierre Quentel in :issue:`29654`.)
|
|
|
|
:class:`~http.server.SimpleHTTPRequestHandler` accepts the new *directory*
|
|
argument, in addition to the new ``--directory`` command line argument.
|
|
With this parameter, the server serves the specified directory, by default it
|
|
uses the current working directory.
|
|
(Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.)
|
|
|
|
The new :class:`ThreadingHTTPServer <http.server.ThreadingHTTPServer>` class
|
|
uses threads to handle requests using :class:`~socketserver.ThreadingMixin`.
|
|
It is used when ``http.server`` is run with ``-m``.
|
|
(Contributed by Julien Palard in :issue:`31639`.)
|
|
|
|
|
|
idlelib and IDLE
|
|
----------------
|
|
|
|
Multiple fixes for autocompletion. (Contributed by Louie Lu in :issue:`15786`.)
|
|
|
|
Module Browser (on the File menu, formerly called Class Browser),
|
|
now displays nested functions and classes in addition to top-level
|
|
functions and classes.
|
|
(Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy
|
|
in :issue:`1612262`.)
|
|
|
|
The Settings dialog (Options, Configure IDLE) has been partly rewritten
|
|
to improve both appearance and function.
|
|
(Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.)
|
|
|
|
The font sample now includes a selection of non-Latin characters so that
|
|
users can better see the effect of selecting a particular font.
|
|
(Contributed by Terry Jan Reedy in :issue:`13802`.)
|
|
The sample can be edited to include other characters.
|
|
(Contributed by Serhiy Storchaka in :issue:`31860`.)
|
|
|
|
The IDLE features formerly implemented as extensions have been reimplemented
|
|
as normal features. Their settings have been moved from the Extensions tab
|
|
to other dialog tabs.
|
|
(Contributed by Charles Wohlganger and Terry Jan Reedy in :issue:`27099`.)
|
|
|
|
Editor code context option revised. Box displays all context lines up to
|
|
maxlines. Clicking on a context line jumps the editor to that line. Context
|
|
colors for custom themes is added to Highlights tab of Settings dialog.
|
|
(Contributed by Cheryl Sabella and Terry Jan Reedy in :issue:`33642`,
|
|
:issue:`33768`, and :issue:`33679`.)
|
|
|
|
On Windows, a new API call tells Windows that tk scales for DPI. On Windows
|
|
8.1+ or 10, with DPI compatibility properties of the Python binary
|
|
unchanged, and a monitor resolution greater than 96 DPI, this should
|
|
make text and lines sharper. It should otherwise have no effect.
|
|
(Contributed by Terry Jan Reedy in :issue:`33656`.)
|
|
|
|
New in 3.7.1:
|
|
|
|
Output over N lines (50 by default) is squeezed down to a button.
|
|
N can be changed in the PyShell section of the General page of the
|
|
Settings dialog. Fewer, but possibly extra long, lines can be squeezed by
|
|
right clicking on the output. Squeezed output can be expanded in place
|
|
by double-clicking the button or into the clipboard or a separate window
|
|
by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.)
|
|
|
|
The changes above have been backported to 3.6 maintenance releases.
|
|
|
|
NEW in 3.7.4:
|
|
|
|
Add "Run Customized" to the Run menu to run a module with customized
|
|
settings. Any command line arguments entered are added to sys.argv.
|
|
They re-appear in the box for the next customized run. One can also
|
|
suppress the normal Shell main module restart. (Contributed by Cheryl
|
|
Sabella, Terry Jan Reedy, and others in :issue:`5680` and :issue:`37627`.)
|
|
|
|
New in 3.7.5:
|
|
|
|
Add optional line numbers for IDLE editor windows. Windows
|
|
open without line numbers unless set otherwise in the General
|
|
tab of the configuration dialog. Line numbers for an existing
|
|
window are shown and hidden in the Options menu.
|
|
(Contributed by Tal Einat and Saimadhav Heblikar in :issue:`17535`.)
|
|
|
|
|
|
importlib
|
|
---------
|
|
|
|
The :class:`importlib.abc.ResourceReader` ABC was introduced to
|
|
support the loading of resources from packages. See also
|
|
:ref:`whatsnew37_importlib_resources`.
|
|
(Contributed by Barry Warsaw, Brett Cannon in :issue:`32248`.)
|
|
|
|
:func:`importlib.reload` now raises :exc:`ModuleNotFoundError` if the module
|
|
lacks a spec.
|
|
(Contributed by Garvit Khatri in :issue:`29851`.)
|
|
|
|
:func:`importlib.find_spec` now raises :exc:`ModuleNotFoundError` instead of
|
|
:exc:`AttributeError` if the specified parent module is not a package (i.e.
|
|
lacks a ``__path__`` attribute).
|
|
(Contributed by Milan Oberkirch in :issue:`30436`.)
|
|
|
|
The new :func:`importlib.source_hash` can be used to compute the hash of
|
|
the passed source. A :ref:`hash-based .pyc file <whatsnew37-pep552>`
|
|
embeds the value returned by this function.
|
|
|
|
|
|
io
|
|
--
|
|
|
|
The new :meth:`TextIOWrapper.reconfigure() <io.TextIOWrapper.reconfigure>`
|
|
method can be used to reconfigure the text stream with the new settings.
|
|
(Contributed by Antoine Pitrou in :issue:`30526` and
|
|
INADA Naoki in :issue:`15216`.)
|
|
|
|
|
|
ipaddress
|
|
---------
|
|
|
|
The new ``subnet_of()`` and ``supernet_of()`` methods of
|
|
:class:`ipaddress.IPv6Network` and :class:`ipaddress.IPv4Network` can
|
|
be used for network containment tests.
|
|
(Contributed by Michel Albert and Cheryl Sabella in :issue:`20825`.)
|
|
|
|
|
|
itertools
|
|
---------
|
|
|
|
:func:`itertools.islice` now accepts
|
|
:meth:`integer-like objects <object.__index__>` as start, stop,
|
|
and slice arguments.
|
|
(Contributed by Will Roberts in :issue:`30537`.)
|
|
|
|
|
|
locale
|
|
------
|
|
|
|
The new *monetary* argument to :func:`locale.format_string` can be used
|
|
to make the conversion use monetary thousands separators and
|
|
grouping strings. (Contributed by Garvit in :issue:`10379`.)
|
|
|
|
The :func:`locale.getpreferredencoding` function now always returns ``'UTF-8'``
|
|
on Android or when in the :ref:`forced UTF-8 mode <whatsnew37-pep540>`.
|
|
|
|
|
|
logging
|
|
-------
|
|
|
|
:class:`~logging.Logger` instances can now be pickled.
|
|
(Contributed by Vinay Sajip in :issue:`30520`.)
|
|
|
|
The new :meth:`StreamHandler.setStream() <logging.StreamHandler.setStream>`
|
|
method can be used to replace the logger stream after handler creation.
|
|
(Contributed by Vinay Sajip in :issue:`30522`.)
|
|
|
|
It is now possible to specify keyword arguments to handler constructors in
|
|
configuration passed to :func:`logging.config.fileConfig`.
|
|
(Contributed by Preston Landers in :issue:`31080`.)
|
|
|
|
|
|
math
|
|
----
|
|
|
|
The new :func:`math.remainder` function implements the IEEE 754-style remainder
|
|
operation. (Contributed by Mark Dickinson in :issue:`29962`.)
|
|
|
|
|
|
mimetypes
|
|
---------
|
|
|
|
The MIME type of .bmp has been changed from ``'image/x-ms-bmp'`` to
|
|
``'image/bmp'``.
|
|
(Contributed by Nitish Chandra in :issue:`22589`.)
|
|
|
|
|
|
msilib
|
|
------
|
|
|
|
The new :meth:`Database.Close() <msilib.Database.Close>` method can be used
|
|
to close the :abbr:`MSI` database.
|
|
(Contributed by Berker Peksag in :issue:`20486`.)
|
|
|
|
|
|
multiprocessing
|
|
---------------
|
|
|
|
The new :meth:`Process.close() <multiprocessing.Process.close>` method
|
|
explicitly closes the process object and releases all resources associated
|
|
with it. :exc:`ValueError` is raised if the underlying process is still
|
|
running.
|
|
(Contributed by Antoine Pitrou in :issue:`30596`.)
|
|
|
|
The new :meth:`Process.kill() <multiprocessing.Process.kill>` method can
|
|
be used to terminate the process using the :data:`SIGKILL` signal on Unix.
|
|
(Contributed by Vitor Pereira in :issue:`30794`.)
|
|
|
|
Non-daemonic threads created by :class:`~multiprocessing.Process` are now
|
|
joined on process exit.
|
|
(Contributed by Antoine Pitrou in :issue:`18966`.)
|
|
|
|
|
|
os
|
|
--
|
|
|
|
:func:`os.fwalk` now accepts the *path* argument as :class:`bytes`.
|
|
(Contributed by Serhiy Storchaka in :issue:`28682`.)
|
|
|
|
:func:`os.scandir` gained support for :ref:`file descriptors <path_fd>`.
|
|
(Contributed by Serhiy Storchaka in :issue:`25996`.)
|
|
|
|
The new :func:`~os.register_at_fork` function allows registering Python
|
|
callbacks to be executed at process fork.
|
|
(Contributed by Antoine Pitrou in :issue:`16500`.)
|
|
|
|
Added :func:`os.preadv` (combine the functionality of :func:`os.readv` and
|
|
:func:`os.pread`) and :func:`os.pwritev` functions (combine the functionality
|
|
of :func:`os.writev` and :func:`os.pwrite`). (Contributed by Pablo Galindo in
|
|
:issue:`31368`.)
|
|
|
|
The mode argument of :func:`os.makedirs` no longer affects the file
|
|
permission bits of newly-created intermediate-level directories.
|
|
(Contributed by Serhiy Storchaka in :issue:`19930`.)
|
|
|
|
:func:`os.dup2` now returns the new file descriptor. Previously, ``None``
|
|
was always returned.
|
|
(Contributed by Benjamin Peterson in :issue:`32441`.)
|
|
|
|
The structure returned by :func:`os.stat` now contains the
|
|
:attr:`~os.stat_result.st_fstype` attribute on Solaris and its derivatives.
|
|
(Contributed by Jesús Cea Avión in :issue:`32659`.)
|
|
|
|
|
|
pathlib
|
|
-------
|
|
|
|
The new :meth:`Path.is_mount() <pathlib.Path.is_mount>` method is now available
|
|
on POSIX systems and can be used to determine whether a path is a mount point.
|
|
(Contributed by Cooper Ry Lees in :issue:`30897`.)
|
|
|
|
|
|
pdb
|
|
---
|
|
|
|
:func:`pdb.set_trace` now takes an optional *header* keyword-only
|
|
argument. If given, it is printed to the console just before debugging
|
|
begins. (Contributed by Barry Warsaw in :issue:`31389`.)
|
|
|
|
:mod:`pdb` command line now accepts ``-m module_name`` as an alternative to
|
|
script file. (Contributed by Mario Corchero in :issue:`32206`.)
|
|
|
|
|
|
py_compile
|
|
----------
|
|
|
|
:func:`py_compile.compile` -- and by extension, :mod:`compileall` -- now
|
|
respects the :envvar:`SOURCE_DATE_EPOCH` environment variable by
|
|
unconditionally creating ``.pyc`` files for hash-based validation.
|
|
This allows for guaranteeing
|
|
`reproducible builds <https://reproducible-builds.org/>`_ of ``.pyc``
|
|
files when they are created eagerly. (Contributed by Bernhard M. Wiedemann
|
|
in :issue:`29708`.)
|
|
|
|
|
|
pydoc
|
|
-----
|
|
|
|
The pydoc server can now bind to an arbitrary hostname specified by the
|
|
new ``-n`` command-line argument.
|
|
(Contributed by Feanil Patel in :issue:`31128`.)
|
|
|
|
|
|
queue
|
|
-----
|
|
|
|
The new :class:`~queue.SimpleQueue` class is an unbounded :abbr:`FIFO` queue.
|
|
(Contributed by Antoine Pitrou in :issue:`14976`.)
|
|
|
|
|
|
re
|
|
--
|
|
|
|
The flags :const:`re.ASCII`, :const:`re.LOCALE` and :const:`re.UNICODE`
|
|
can be set within the scope of a group.
|
|
(Contributed by Serhiy Storchaka in :issue:`31690`.)
|
|
|
|
:func:`re.split` now supports splitting on a pattern like ``r'\b'``,
|
|
``'^$'`` or ``(?=-)`` that matches an empty string.
|
|
(Contributed by Serhiy Storchaka in :issue:`25054`.)
|
|
|
|
Regular expressions compiled with the :const:`re.LOCALE` flag no longer
|
|
depend on the locale at compile time. Locale settings are applied only
|
|
when the compiled regular expression is used.
|
|
(Contributed by Serhiy Storchaka in :issue:`30215`.)
|
|
|
|
:exc:`FutureWarning` is now emitted if a regular expression contains
|
|
character set constructs that will change semantically in the future,
|
|
such as nested sets and set operations.
|
|
(Contributed by Serhiy Storchaka in :issue:`30349`.)
|
|
|
|
Compiled regular expression and match objects can now be copied
|
|
using :func:`copy.copy` and :func:`copy.deepcopy`.
|
|
(Contributed by Serhiy Storchaka in :issue:`10076`.)
|
|
|
|
|
|
signal
|
|
------
|
|
|
|
The new *warn_on_full_buffer* argument to the :func:`signal.set_wakeup_fd`
|
|
function makes it possible to specify whether Python prints a warning on
|
|
stderr when the wakeup buffer overflows.
|
|
(Contributed by Nathaniel J. Smith in :issue:`30050`.)
|
|
|
|
|
|
socket
|
|
------
|
|
|
|
The new :func:`socket.getblocking() <socket.socket.getblocking>` method
|
|
returns ``True`` if the socket is in blocking mode and ``False`` otherwise.
|
|
(Contributed by Yury Selivanov in :issue:`32373`.)
|
|
|
|
The new :func:`socket.close` function closes the passed socket file descriptor.
|
|
This function should be used instead of :func:`os.close` for better
|
|
compatibility across platforms.
|
|
(Contributed by Christian Heimes in :issue:`32454`.)
|
|
|
|
The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION`
|
|
(Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and
|
|
:data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants.
|
|
(Contributed by Omar Sandoval in :issue:`26273` and
|
|
Nathaniel J. Smith in :issue:`29728`.)
|
|
|
|
Support for :data:`socket.AF_VSOCK` sockets has been added to allow
|
|
communication between virtual machines and their hosts.
|
|
(Contributed by Cathy Avery in :issue:`27584`.)
|
|
|
|
Sockets now auto-detect family, type and protocol from file descriptor
|
|
by default.
|
|
(Contributed by Christian Heimes in :issue:`28134`.)
|
|
|
|
|
|
socketserver
|
|
------------
|
|
|
|
:meth:`socketserver.ThreadingMixIn.server_close` now waits until all non-daemon
|
|
threads complete. :meth:`socketserver.ForkingMixIn.server_close` now waits
|
|
until all child processes complete.
|
|
|
|
Add a new :attr:`socketserver.ForkingMixIn.block_on_close` class attribute to
|
|
:class:`socketserver.ForkingMixIn` and :class:`socketserver.ThreadingMixIn`
|
|
classes. Set the class attribute to ``False`` to get the pre-3.7 behaviour.
|
|
|
|
|
|
sqlite3
|
|
-------
|
|
|
|
:class:`sqlite3.Connection` now exposes the :meth:`~sqlite3.Connection.backup`
|
|
method when the underlying SQLite library is at version 3.6.11 or higher.
|
|
(Contributed by Lele Gaifax in :issue:`27645`.)
|
|
|
|
The *database* argument of :func:`sqlite3.connect` now accepts any
|
|
:term:`path-like object`, instead of just a string.
|
|
(Contributed by Anders Lorentsen in :issue:`31843`.)
|
|
|
|
|
|
ssl
|
|
---
|
|
|
|
The :mod:`ssl` module now uses OpenSSL's builtin API instead of
|
|
:func:`~ssl.match_hostname` to check a host name or an IP address. Values
|
|
are validated during TLS handshake. Any certificate validation error
|
|
including failing the host name check now raises
|
|
:exc:`~ssl.SSLCertVerificationError` and aborts the handshake with a proper
|
|
TLS Alert message. The new exception contains additional information.
|
|
Host name validation can be customized with
|
|
:attr:`SSLContext.hostname_checks_common_name <ssl.SSLContext.hostname_checks_common_name>`.
|
|
(Contributed by Christian Heimes in :issue:`31399`.)
|
|
|
|
.. note::
|
|
The improved host name check requires a *libssl* implementation compatible
|
|
with OpenSSL 1.0.2 or 1.1. Consequently, OpenSSL 0.9.8 and 1.0.1 are no
|
|
longer supported (see :ref:`37-platform-support-removals` for more details).
|
|
The ssl module is mostly compatible with LibreSSL 2.7.2 and newer.
|
|
|
|
The ``ssl`` module no longer sends IP addresses in SNI TLS extension.
|
|
(Contributed by Christian Heimes in :issue:`32185`.)
|
|
|
|
:func:`~ssl.match_hostname` no longer supports partial wildcards like
|
|
``www*.example.org``.
|
|
(Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in
|
|
:issue:`31399`.)
|
|
|
|
The default cipher suite selection of the ``ssl`` module now uses a blacklist
|
|
approach rather than a hard-coded whitelist. Python no longer re-enables
|
|
ciphers that have been blocked by OpenSSL security updates. Default cipher
|
|
suite selection can be configured at compile time.
|
|
(Contributed by Christian Heimes in :issue:`31429`.)
|
|
|
|
Validation of server certificates containing internationalized domain names
|
|
(IDNs) is now supported. As part of this change, the
|
|
:attr:`SSLSocket.server_hostname <ssl.SSLSocket.server_hostname>` attribute
|
|
now stores the expected hostname in A-label form (``"xn--pythn-mua.org"``),
|
|
rather than the U-label form (``"pythön.org"``). (Contributed by
|
|
Nathaniel J. Smith and Christian Heimes in :issue:`28414`.)
|
|
|
|
The ``ssl`` module has preliminary and experimental support for TLS 1.3 and
|
|
OpenSSL 1.1.1. At the time of Python 3.7.0 release, OpenSSL 1.1.1 is still
|
|
under development and TLS 1.3 hasn't been finalized yet. The TLS 1.3
|
|
handshake and protocol behaves slightly differently than TLS 1.2 and earlier,
|
|
see :ref:`ssl-tlsv1_3`.
|
|
(Contributed by Christian Heimes in :issue:`32947`, :issue:`20995`,
|
|
:issue:`29136`, :issue:`30622` and :issue:`33618`)
|
|
|
|
:class:`~ssl.SSLSocket` and :class:`~ssl.SSLObject` no longer have a public
|
|
constructor. Direct instantiation was never a documented and supported
|
|
feature. Instances must be created with :class:`~ssl.SSLContext` methods
|
|
:meth:`~ssl.SSLContext.wrap_socket` and :meth:`~ssl.SSLContext.wrap_bio`.
|
|
(Contributed by Christian Heimes in :issue:`32951`)
|
|
|
|
OpenSSL 1.1 APIs for setting the minimum and maximum TLS protocol version are
|
|
available as :attr:`SSLContext.minimum_version <ssl.SSLContext.minimum_version>`
|
|
and :attr:`SSLContext.maximum_version <ssl.SSLContext.maximum_version>`.
|
|
Supported protocols are indicated by several new flags, such as
|
|
:data:`~ssl.HAS_TLSv1_1`.
|
|
(Contributed by Christian Heimes in :issue:`32609`.)
|
|
|
|
|
|
string
|
|
------
|
|
|
|
:class:`string.Template` now lets you to optionally modify the regular
|
|
expression pattern for braced placeholders and non-braced placeholders
|
|
separately. (Contributed by Barry Warsaw in :issue:`1198569`.)
|
|
|
|
|
|
subprocess
|
|
----------
|
|
|
|
The :func:`subprocess.run` function accepts the new *capture_output*
|
|
keyword argument. When true, stdout and stderr will be captured.
|
|
This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and
|
|
*stderr* arguments.
|
|
(Contributed by Bo Bayles in :issue:`32102`.)
|
|
|
|
The ``subprocess.run`` function and the :class:`subprocess.Popen` constructor
|
|
now accept the *text* keyword argument as an alias
|
|
to *universal_newlines*.
|
|
(Contributed by Andrew Clegg in :issue:`31756`.)
|
|
|
|
On Windows the default for *close_fds* was changed from ``False`` to
|
|
``True`` when redirecting the standard handles. It's now possible to set
|
|
*close_fds* to true when redirecting the standard handles. See
|
|
:class:`subprocess.Popen`. This means that *close_fds* now defaults to
|
|
``True`` on all supported platforms.
|
|
(Contributed by Segev Finer in :issue:`19764`.)
|
|
|
|
The subprocess module is now more graceful when handling
|
|
:exc:`KeyboardInterrupt` during :func:`subprocess.call`,
|
|
:func:`subprocess.run`, or in a :class:`~subprocess.Popen`
|
|
context manager. It now waits a short amount of time for the child
|
|
to exit, before continuing the handling of the ``KeyboardInterrupt``
|
|
exception.
|
|
(Contributed by Gregory P. Smith in :issue:`25942`.)
|
|
|
|
|
|
sys
|
|
---
|
|
|
|
The new :func:`sys.breakpointhook` hook function is called by the
|
|
built-in :func:`breakpoint`.
|
|
(Contributed by Barry Warsaw in :issue:`31353`.)
|
|
|
|
On Android, the new :func:`sys.getandroidapilevel` returns the build-time
|
|
Android API version.
|
|
(Contributed by Victor Stinner in :issue:`28740`.)
|
|
|
|
The new :func:`sys.get_coroutine_origin_tracking_depth` function returns
|
|
the current coroutine origin tracking depth, as set by
|
|
the new :func:`sys.set_coroutine_origin_tracking_depth`. :mod:`asyncio`
|
|
has been converted to use this new API instead of
|
|
the deprecated :func:`sys.set_coroutine_wrapper`.
|
|
(Contributed by Nathaniel J. Smith in :issue:`32591`.)
|
|
|
|
|
|
time
|
|
----
|
|
|
|
:pep:`564` adds six new functions with nanosecond resolution to the
|
|
:mod:`time` module:
|
|
|
|
* :func:`time.clock_gettime_ns`
|
|
* :func:`time.clock_settime_ns`
|
|
* :func:`time.monotonic_ns`
|
|
* :func:`time.perf_counter_ns`
|
|
* :func:`time.process_time_ns`
|
|
* :func:`time.time_ns`
|
|
|
|
New clock identifiers have been added:
|
|
|
|
* :data:`time.CLOCK_BOOTTIME` (Linux): Identical to
|
|
:data:`time.CLOCK_MONOTONIC`, except it also includes any time that the
|
|
system is suspended.
|
|
* :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution
|
|
per-process CPU timer.
|
|
* :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is
|
|
the time the system has been running and not suspended, providing accurate
|
|
uptime measurement.
|
|
|
|
The new :func:`time.thread_time` and :func:`time.thread_time_ns` functions
|
|
can be used to get per-thread CPU time measurements.
|
|
(Contributed by Antoine Pitrou in :issue:`32025`.)
|
|
|
|
The new :func:`time.pthread_getcpuclockid` function returns the clock ID
|
|
of the thread-specific CPU-time clock.
|
|
|
|
|
|
tkinter
|
|
-------
|
|
|
|
The new :class:`tkinter.ttk.Spinbox` class is now available.
|
|
(Contributed by Alan Moore in :issue:`32585`.)
|
|
|
|
|
|
tracemalloc
|
|
-----------
|
|
|
|
:class:`tracemalloc.Traceback` behaves more like regular tracebacks,
|
|
sorting the frames from oldest to most recent.
|
|
:meth:`Traceback.format() <tracemalloc.Traceback.format>`
|
|
now accepts negative *limit*, truncating the result to the
|
|
``abs(limit)`` oldest frames. To get the old behaviour, use
|
|
the new *most_recent_first* argument to ``Traceback.format()``.
|
|
(Contributed by Jesse Bakker in :issue:`32121`.)
|
|
|
|
|
|
types
|
|
-----
|
|
|
|
The new :class:`~types.WrapperDescriptorType`,
|
|
:class:`~types.MethodWrapperType`, :class:`~types.MethodDescriptorType`,
|
|
and :class:`~types.ClassMethodDescriptorType` classes are now available.
|
|
(Contributed by Manuel Krebber and Guido van Rossum in :issue:`29377`,
|
|
and Serhiy Storchaka in :issue:`32265`.)
|
|
|
|
The new :func:`types.resolve_bases` function resolves MRO entries
|
|
dynamically as specified by :pep:`560`.
|
|
(Contributed by Ivan Levkivskyi in :issue:`32717`.)
|
|
|
|
|
|
unicodedata
|
|
-----------
|
|
|
|
The internal :mod:`unicodedata` database has been upgraded to use `Unicode 11
|
|
<http://www.unicode.org/versions/Unicode11.0.0/>`_. (Contributed by Benjamin
|
|
Peterson.)
|
|
|
|
|
|
unittest
|
|
--------
|
|
|
|
The new ``-k`` command-line option allows filtering tests by a name
|
|
substring or a Unix shell-like pattern.
|
|
For example, ``python -m unittest -k foo`` runs
|
|
``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``,
|
|
but not ``bar_tests.FooTest.test_something``.
|
|
(Contributed by Jonas Haag in :issue:`32071`.)
|
|
|
|
|
|
unittest.mock
|
|
-------------
|
|
|
|
The :const:`~unittest.mock.sentinel` attributes now preserve their identity
|
|
when they are :mod:`copied <copy>` or :mod:`pickled <pickle>`. (Contributed by
|
|
Serhiy Storchaka in :issue:`20804`.)
|
|
|
|
The new :func:`~unittest.mock.seal` function allows sealing
|
|
:class:`~unittest.mock.Mock` instances, which will disallow further creation
|
|
of attribute mocks. The seal is applied recursively to all attributes that
|
|
are themselves mocks.
|
|
(Contributed by Mario Corchero in :issue:`30541`.)
|
|
|
|
|
|
urllib.parse
|
|
------------
|
|
|
|
:func:`urllib.parse.quote` has been updated from :rfc:`2396` to :rfc:`3986`,
|
|
adding ``~`` to the set of characters that are never quoted by default.
|
|
(Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.)
|
|
|
|
|
|
uu
|
|
--
|
|
|
|
The :func:`uu.encode` function now accepts an optional *backtick*
|
|
keyword argument. When it's true, zeros are represented by ``'`'``
|
|
instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.)
|
|
|
|
|
|
uuid
|
|
----
|
|
|
|
The new :attr:`UUID.is_safe <uuid.UUID.is_safe>` attribute relays information
|
|
from the platform about whether generated UUIDs are generated with a
|
|
multiprocessing-safe method.
|
|
(Contributed by Barry Warsaw in :issue:`22807`.)
|
|
|
|
:func:`uuid.getnode` now prefers universally administered
|
|
MAC addresses over locally administered MAC addresses.
|
|
This makes a better guarantee for global uniqueness of UUIDs returned
|
|
from :func:`uuid.uuid1`. If only locally administered MAC addresses are
|
|
available, the first such one found is returned.
|
|
(Contributed by Barry Warsaw in :issue:`32107`.)
|
|
|
|
|
|
warnings
|
|
--------
|
|
|
|
The initialization of the default warnings filters has changed as follows:
|
|
|
|
* warnings enabled via command line options (including those for :option:`-b`
|
|
and the new CPython-specific :option:`-X` ``dev`` option) are always passed
|
|
to the warnings machinery via the :data:`sys.warnoptions` attribute.
|
|
|
|
* warnings filters enabled via the command line or the environment now have the
|
|
following order of precedence:
|
|
|
|
* the ``BytesWarning`` filter for :option:`-b` (or ``-bb``)
|
|
* any filters specified with the :option:`-W` option
|
|
* any filters specified with the :envvar:`PYTHONWARNINGS` environment
|
|
variable
|
|
* any other CPython specific filters (e.g. the ``default`` filter added
|
|
for the new ``-X dev`` mode)
|
|
* any implicit filters defined directly by the warnings machinery
|
|
|
|
* in CPython debug builds, all warnings are now displayed by default (the
|
|
implicit filter list is empty)
|
|
|
|
(Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`,
|
|
:issue:`32043`, and :issue:`32230`.)
|
|
|
|
Deprecation warnings are once again shown by default in single-file scripts and
|
|
at the interactive prompt. See :ref:`whatsnew37-pep565` for details.
|
|
(Contributed by Nick Coghlan in :issue:`31975`.)
|
|
|
|
|
|
xml.etree
|
|
---------
|
|
|
|
:ref:`ElementPath <elementtree-xpath>` predicates in the :meth:`find`
|
|
methods can now compare text of the current node with ``[. = "text"]``,
|
|
not only text in children. Predicates also allow adding spaces for
|
|
better readability. (Contributed by Stefan Behnel in :issue:`31648`.)
|
|
|
|
|
|
xmlrpc.server
|
|
-------------
|
|
|
|
:meth:`SimpleXMLRPCDispatcher.register_function <xmlrpc.server.SimpleXMLRPCDispatcher>`
|
|
can now be used as a decorator. (Contributed by Xiang Zhang in
|
|
:issue:`7769`.)
|
|
|
|
|
|
zipapp
|
|
------
|
|
|
|
Function :func:`~zipapp.create_archive` now accepts an optional *filter*
|
|
argument to allow the user to select which files should be included in the
|
|
archive. (Contributed by Irmen de Jong in :issue:`31072`.)
|
|
|
|
Function :func:`~zipapp.create_archive` now accepts an optional *compressed*
|
|
argument to generate a compressed archive. A command line option
|
|
``--compress`` has also been added to support compression.
|
|
(Contributed by Zhiming Wang in :issue:`31638`.)
|
|
|
|
|
|
zipfile
|
|
-------
|
|
|
|
:class:`~zipfile.ZipFile` now accepts the new *compresslevel* parameter to
|
|
control the compression level.
|
|
(Contributed by Bo Bayles in :issue:`21417`.)
|
|
|
|
Subdirectories in archives created by ``ZipFile`` are now stored in
|
|
alphabetical order.
|
|
(Contributed by Bernhard M. Wiedemann in :issue:`30693`.)
|
|
|
|
|
|
C API Changes
|
|
=============
|
|
|
|
A new API for thread-local storage has been implemented. See
|
|
:ref:`whatsnew37-pep539` for an overview and
|
|
:ref:`thread-specific-storage-api` for a complete reference.
|
|
(Contributed by Masayuki Yamamoto in :issue:`25658`.)
|
|
|
|
The new :ref:`context variables <whatsnew37-pep567>` functionality
|
|
exposes a number of :ref:`new C APIs <contextvarsobjects>`.
|
|
|
|
The new :c:func:`PyImport_GetModule` function returns the previously
|
|
imported module with the given name.
|
|
(Contributed by Eric Snow in :issue:`28411`.)
|
|
|
|
The new :c:macro:`Py_RETURN_RICHCOMPARE` macro eases writing rich
|
|
comparison functions.
|
|
(Contributed by Petr Victorin in :issue:`23699`.)
|
|
|
|
The new :c:macro:`Py_UNREACHABLE` macro can be used to mark unreachable
|
|
code paths.
|
|
(Contributed by Barry Warsaw in :issue:`31338`.)
|
|
|
|
The :mod:`tracemalloc` now exposes a C API through the new
|
|
:c:func:`PyTraceMalloc_Track` and :c:func:`PyTraceMalloc_Untrack`
|
|
functions.
|
|
(Contributed by Victor Stinner in :issue:`30054`.)
|
|
|
|
The new :c:func:`import__find__load__start` and
|
|
:c:func:`import__find__load__done` static markers can be used to trace
|
|
module imports.
|
|
(Contributed by Christian Heimes in :issue:`31574`.)
|
|
|
|
The fields :c:member:`name` and :c:member:`doc` of structures
|
|
:c:type:`PyMemberDef`, :c:type:`PyGetSetDef`,
|
|
:c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`,
|
|
and :c:type:`wrapperbase` are now of type ``const char *`` rather of
|
|
``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.)
|
|
|
|
The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8`
|
|
is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy
|
|
Storchaka in :issue:`28769`.)
|
|
|
|
The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and
|
|
:c:func:`PyMapping_Items` is now always a list, rather than a list or a
|
|
tuple. (Contributed by Oren Milman in :issue:`28280`.)
|
|
|
|
Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`.
|
|
(Contributed by Serhiy Storchaka in :issue:`27867`.)
|
|
|
|
:c:func:`PyOS_AfterFork` is deprecated in favour of the new functions
|
|
:c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and
|
|
:c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in
|
|
:issue:`16500`.)
|
|
|
|
The ``PyExc_RecursionErrorInst`` singleton that was part of the public API
|
|
has been removed as its members being never cleared may cause a segfault
|
|
during finalization of the interpreter. Contributed by Xavier de Gaye in
|
|
:issue:`22898` and :issue:`30697`.
|
|
|
|
Added C API support for timezones with timezone constructors
|
|
:c:func:`PyTimeZone_FromOffset` and :c:func:`PyTimeZone_FromOffsetAndName`,
|
|
and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`.
|
|
Contributed by Paul Ganssle in :issue:`10381`.
|
|
|
|
The type of results of :c:func:`PyThread_start_new_thread` and
|
|
:c:func:`PyThread_get_thread_ident`, and the *id* parameter of
|
|
:c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to
|
|
:c:type:`unsigned long`.
|
|
(Contributed by Serhiy Storchaka in :issue:`6532`.)
|
|
|
|
:c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the
|
|
second argument is ``NULL`` and the :c:type:`wchar_t*` string contains null
|
|
characters. (Contributed by Serhiy Storchaka in :issue:`30708`.)
|
|
|
|
Changes to the startup sequence and the management of dynamic memory
|
|
allocators mean that the long documented requirement to call
|
|
:c:func:`Py_Initialize` before calling most C API functions is now
|
|
relied on more heavily, and failing to abide by it may lead to segfaults in
|
|
embedding applications. See the :ref:`porting-to-python-37` section in this
|
|
document and the :ref:`pre-init-safe` section in the C API documentation
|
|
for more details.
|
|
|
|
The new :c:func:`PyInterpreterState_GetID` returns the unique ID for a
|
|
given interpreter.
|
|
(Contributed by Eric Snow in :issue:`29102`.)
|
|
|
|
:c:func:`Py_DecodeLocale`, :c:func:`Py_EncodeLocale` now use the UTF-8
|
|
encoding when the :ref:`UTF-8 mode <whatsnew37-pep540>` is enabled.
|
|
(Contributed by Victor Stinner in :issue:`29240`.)
|
|
|
|
:c:func:`PyUnicode_DecodeLocaleAndSize` and :c:func:`PyUnicode_EncodeLocale`
|
|
now use the current locale encoding for ``surrogateescape`` error handler.
|
|
(Contributed by Victor Stinner in :issue:`29240`.)
|
|
|
|
The *start* and *end* parameters of :c:func:`PyUnicode_FindChar` are
|
|
now adjusted to behave like string slices.
|
|
(Contributed by Xiang Zhang in :issue:`28822`.)
|
|
|
|
|
|
Build Changes
|
|
=============
|
|
|
|
Support for building ``--without-threads`` has been removed. The
|
|
:mod:`threading` module is now always available.
|
|
(Contributed by Antoine Pitrou in :issue:`31370`.).
|
|
|
|
A full copy of libffi is no longer bundled for use when building the
|
|
:mod:`_ctypes <ctypes>` module on non-OSX UNIX platforms. An installed copy
|
|
of libffi is now required when building ``_ctypes`` on such platforms.
|
|
(Contributed by Zachary Ware in :issue:`27979`.)
|
|
|
|
The Windows build process no longer depends on Subversion to pull in external
|
|
sources, a Python script is used to download zipfiles from GitHub instead.
|
|
If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to
|
|
download a copy of 32-bit Python for this purpose. (Contributed by Zachary
|
|
Ware in :issue:`30450`.)
|
|
|
|
The :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl.
|
|
OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer
|
|
supported. LibreSSL is temporarily not supported as well. LibreSSL releases
|
|
up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs.
|
|
|
|
|
|
.. _whatsnew37-perf:
|
|
|
|
Optimizations
|
|
=============
|
|
|
|
The overhead of calling many methods of various standard library classes
|
|
implemented in C has been significantly reduced by porting more code
|
|
to use the ``METH_FASTCALL`` convention.
|
|
(Contributed by Victor Stinner in :issue:`29300`, :issue:`29507`,
|
|
:issue:`29452`, and :issue:`29286`.)
|
|
|
|
Various optimizations have reduced Python startup time by 10% on Linux and
|
|
up to 30% on macOS.
|
|
(Contributed by Victor Stinner, INADA Naoki in :issue:`29585`, and
|
|
Ivan Levkivskyi in :issue:`31333`.)
|
|
|
|
Method calls are now up to 20% faster due to the bytecode changes which
|
|
avoid creating bound method instances.
|
|
(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.)
|
|
|
|
.. _whatsnew37-asyncio-perf:
|
|
|
|
The :mod:`asyncio` module received a number of notable optimizations for
|
|
commonly used functions:
|
|
|
|
* The :func:`asyncio.get_event_loop` function has been reimplemented in C to
|
|
make it up to 15 times faster.
|
|
(Contributed by Yury Selivanov in :issue:`32296`.)
|
|
|
|
* :class:`asyncio.Future` callback management has been optimized.
|
|
(Contributed by Yury Selivanov in :issue:`32348`.)
|
|
|
|
* :func:`asyncio.gather` is now up to 15% faster.
|
|
(Contributed by Yury Selivanov in :issue:`32355`.)
|
|
|
|
* :func:`asyncio.sleep` is now up to 2 times faster when the *delay*
|
|
argument is zero or negative.
|
|
(Contributed by Andrew Svetlov in :issue:`32351`.)
|
|
|
|
* The performance overhead of asyncio debug mode has been reduced.
|
|
(Contributed by Antoine Pitrou in :issue:`31970`.)
|
|
|
|
As a result of :ref:`PEP 560 work <whatsnew37-pep560>`, the import time
|
|
of :mod:`typing` has been reduced by a factor of 7, and many typing operations
|
|
are now faster.
|
|
(Contributed by Ivan Levkivskyi in :issue:`32226`.)
|
|
|
|
:func:`sorted` and :meth:`list.sort` have been optimized for common cases
|
|
to be up to 40-75% faster.
|
|
(Contributed by Elliot Gorokhovsky in :issue:`28685`.)
|
|
|
|
:meth:`dict.copy` is now up to 5.5 times faster.
|
|
(Contributed by Yury Selivanov in :issue:`31179`.)
|
|
|
|
:func:`hasattr` and :func:`getattr` are now about 4 times faster when
|
|
*name* is not found and *obj* does not override :meth:`object.__getattr__`
|
|
or :meth:`object.__getattribute__`.
|
|
(Contributed by INADA Naoki in :issue:`32544`.)
|
|
|
|
Searching for certain Unicode characters (like Ukrainian capital "Є")
|
|
in a string was up to 25 times slower than searching for other characters.
|
|
It is now only 3 times slower in the worst case.
|
|
(Contributed by Serhiy Storchaka in :issue:`24821`.)
|
|
|
|
The :func:`collections.namedtuple` factory has been reimplemented to
|
|
make the creation of named tuples 4 to 6 times faster.
|
|
(Contributed by Jelle Zijlstra with further improvements by INADA Naoki,
|
|
Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.)
|
|
|
|
:meth:`date.fromordinal` and :meth:`date.fromtimestamp` are now up to
|
|
30% faster in the common case.
|
|
(Contributed by Paul Ganssle in :issue:`32403`.)
|
|
|
|
The :func:`os.fwalk` function is now up to 2 times faster thanks to
|
|
the use of :func:`os.scandir`.
|
|
(Contributed by Serhiy Storchaka in :issue:`25996`.)
|
|
|
|
The speed of the :func:`shutil.rmtree` function has been improved by
|
|
20--40% thanks to the use of the :func:`os.scandir` function.
|
|
(Contributed by Serhiy Storchaka in :issue:`28564`.)
|
|
|
|
Optimized case-insensitive matching and searching of :mod:`regular
|
|
expressions <re>`. Searching some patterns can now be up to 20 times faster.
|
|
(Contributed by Serhiy Storchaka in :issue:`30285`.)
|
|
|
|
:func:`re.compile` now converts ``flags`` parameter to int object if
|
|
it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than
|
|
Python 3.6 by about 10% depending on the pattern.
|
|
(Contributed by INADA Naoki in :issue:`31671`.)
|
|
|
|
The :meth:`~selectors.BaseSelector.modify` methods of classes
|
|
:class:`selectors.EpollSelector`, :class:`selectors.PollSelector`
|
|
and :class:`selectors.DevpollSelector` may be around 10% faster under
|
|
heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`)
|
|
|
|
Constant folding has been moved from the peephole optimizer to the new AST
|
|
optimizer, which is able perform optimizations more consistently.
|
|
(Contributed by Eugene Toder and INADA Naoki in :issue:`29469` and
|
|
:issue:`11549`.)
|
|
|
|
Most functions and methods in :mod:`abc` have been rewritten in C.
|
|
This makes creation of abstract base classes, and calling :func:`isinstance`
|
|
and :func:`issubclass` on them 1.5x faster. This also reduces Python
|
|
start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki
|
|
in :issue:`31333`)
|
|
|
|
Significant speed improvements to alternate constructors for
|
|
:class:`datetime.date` and :class:`datetime.datetime` by using fast-path
|
|
constructors when not constructing subclasses. (Contributed by Paul Ganssle
|
|
in :issue:`32403`)
|
|
|
|
The speed of comparison of :class:`array.array` instances has been
|
|
improved considerably in certain cases. It is now from 10x to 70x faster
|
|
when comparing arrays holding values of the same integer type.
|
|
(Contributed by Adrian Wielgosik in :issue:`24700`.)
|
|
|
|
The :func:`math.erf` and :func:`math.erfc` functions now use the (faster)
|
|
C library implementation on most platforms.
|
|
(Contributed by Serhiy Storchaka in :issue:`26121`.)
|
|
|
|
|
|
Other CPython Implementation Changes
|
|
====================================
|
|
|
|
* Trace hooks may now opt out of receiving the ``line`` and opt into receiving
|
|
the ``opcode`` events from the interpreter by setting the corresponding new
|
|
``f_trace_lines`` and ``f_trace_opcodes`` attributes on the
|
|
frame being traced. (Contributed by Nick Coghlan in :issue:`31344`.)
|
|
|
|
* Fixed some consistency problems with namespace package module attributes.
|
|
Namespace module objects now have an ``__file__`` that is set to ``None``
|
|
(previously unset), and their ``__spec__.origin`` is also set to ``None``
|
|
(previously the string ``"namespace"``). See :issue:`32305`. Also, the
|
|
namespace module object's ``__spec__.loader`` is set to the same value as
|
|
``__loader__`` (previously, the former was set to ``None``). See
|
|
:issue:`32303`.
|
|
|
|
* The :func:`locals` dictionary now displays in the lexical order that
|
|
variables were defined. Previously, the order was undefined.
|
|
(Contributed by Raymond Hettinger in :issue:`32690`.)
|
|
|
|
* The :mod:`distutils` ``upload`` command no longer tries to change CR
|
|
end-of-line characters to CRLF. This fixes a corruption issue with sdists
|
|
that ended with a byte equivalent to CR.
|
|
(Contributed by Bo Bayles in :issue:`32304`.)
|
|
|
|
|
|
Deprecated Python Behavior
|
|
==========================
|
|
|
|
Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated
|
|
in comprehensions and generator expressions (aside from the iterable expression
|
|
in the leftmost :keyword:`!for` clause). This ensures that comprehensions
|
|
always immediately return a container of the appropriate type (rather than
|
|
potentially returning a :term:`generator iterator` object), while generator
|
|
expressions won't attempt to interleave their implicit output with the output
|
|
from any explicit yield expressions. In Python 3.7, such expressions emit
|
|
:exc:`DeprecationWarning` when compiled, in Python 3.8 this will be a
|
|
:exc:`SyntaxError`.
|
|
(Contributed by Serhiy Storchaka in :issue:`10544`.)
|
|
|
|
Returning a subclass of :class:`complex` from :meth:`object.__complex__` is
|
|
deprecated and will be an error in future Python versions. This makes
|
|
``__complex__()`` consistent with :meth:`object.__int__` and
|
|
:meth:`object.__float__`.
|
|
(Contributed by Serhiy Storchaka in :issue:`28894`.)
|
|
|
|
|
|
|
|
Deprecated Python modules, functions and methods
|
|
================================================
|
|
|
|
aifc
|
|
----
|
|
|
|
:func:`aifc.openfp` has been deprecated and will be removed in Python 3.9.
|
|
Use :func:`aifc.open` instead.
|
|
(Contributed by Brian Curtin in :issue:`31985`.)
|
|
|
|
|
|
.. _whatsnew37-asyncio-deprecated:
|
|
|
|
asyncio
|
|
-------
|
|
|
|
Support for directly ``await``-ing instances of :class:`asyncio.Lock` and
|
|
other asyncio synchronization primitives has been deprecated. An
|
|
asynchronous context manager must be used in order to acquire and release
|
|
the synchronization resource.
|
|
(Contributed by Andrew Svetlov in :issue:`32253`.)
|
|
|
|
The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks`
|
|
methods have been deprecated.
|
|
(Contributed by Andrew Svetlov in :issue:`32250`.)
|
|
|
|
|
|
collections
|
|
-----------
|
|
|
|
In Python 3.8, the abstract base classes in :mod:`collections.abc` will no
|
|
longer be exposed in the regular :mod:`collections` module. This will help
|
|
create a clearer distinction between the concrete classes and the abstract
|
|
base classes.
|
|
(Contributed by Serhiy Storchaka in :issue:`25988`.)
|
|
|
|
|
|
dbm
|
|
---
|
|
|
|
:mod:`dbm.dumb` now supports reading read-only files and no longer writes the
|
|
index file when it is not changed. A deprecation warning is now emitted
|
|
if the index file is missing and recreated in the ``'r'`` and ``'w'``
|
|
modes (this will be an error in future Python releases).
|
|
(Contributed by Serhiy Storchaka in :issue:`28847`.)
|
|
|
|
|
|
enum
|
|
----
|
|
|
|
In Python 3.8, attempting to check for non-Enum objects in :class:`Enum`
|
|
classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly,
|
|
attempting to check for non-Flag objects in a :class:`Flag` member will
|
|
raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations
|
|
return :const:`False` instead.
|
|
(Contributed by Ethan Furman in :issue:`33217`.)
|
|
|
|
|
|
gettext
|
|
-------
|
|
|
|
Using non-integer value for selecting a plural form in :mod:`gettext` is
|
|
now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka
|
|
in :issue:`28692`.)
|
|
|
|
|
|
importlib
|
|
---------
|
|
|
|
Methods
|
|
:meth:`MetaPathFinder.find_module() <importlib.abc.MetaPathFinder.find_module>`
|
|
(replaced by
|
|
:meth:`MetaPathFinder.find_spec() <importlib.abc.MetaPathFinder.find_spec>`)
|
|
and
|
|
:meth:`PathEntryFinder.find_loader() <importlib.abc.PathEntryFinder.find_loader>`
|
|
(replaced by
|
|
:meth:`PathEntryFinder.find_spec() <importlib.abc.PathEntryFinder.find_spec>`)
|
|
both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`.
|
|
(Contributed by Matthias Bussonnier in :issue:`29576`)
|
|
|
|
The :class:`importlib.abc.ResourceLoader` ABC has been deprecated in
|
|
favour of :class:`importlib.abc.ResourceReader`.
|
|
|
|
|
|
locale
|
|
------
|
|
|
|
:func:`locale.format` has been deprecated, use :meth:`locale.format_string`
|
|
instead. (Contributed by Garvit in :issue:`10379`.)
|
|
|
|
|
|
macpath
|
|
-------
|
|
|
|
The :mod:`macpath` is now deprecated and will be removed in Python 3.8.
|
|
(Contributed by Chi Hsuan Yen in :issue:`9850`.)
|
|
|
|
|
|
threading
|
|
---------
|
|
|
|
:mod:`dummy_threading` and :mod:`_dummy_thread` have been deprecated. It is
|
|
no longer possible to build Python with threading disabled.
|
|
Use :mod:`threading` instead.
|
|
(Contributed by Antoine Pitrou in :issue:`31370`.)
|
|
|
|
|
|
socket
|
|
------
|
|
|
|
The silent argument value truncation in :func:`socket.htons` and
|
|
:func:`socket.ntohs` has been deprecated. In future versions of Python,
|
|
if the passed argument is larger than 16 bits, an exception will be raised.
|
|
(Contributed by Oren Milman in :issue:`28332`.)
|
|
|
|
|
|
ssl
|
|
---
|
|
|
|
:func:`ssl.wrap_socket` is deprecated. Use
|
|
:meth:`ssl.SSLContext.wrap_socket` instead.
|
|
(Contributed by Christian Heimes in :issue:`28124`.)
|
|
|
|
|
|
sunau
|
|
-----
|
|
|
|
:func:`sunau.openfp` has been deprecated and will be removed in Python 3.9.
|
|
Use :func:`sunau.open` instead.
|
|
(Contributed by Brian Curtin in :issue:`31985`.)
|
|
|
|
|
|
sys
|
|
---
|
|
|
|
Deprecated :func:`sys.set_coroutine_wrapper` and
|
|
:func:`sys.get_coroutine_wrapper`.
|
|
|
|
The undocumented ``sys.callstats()`` function has been deprecated and
|
|
will be removed in a future Python version.
|
|
(Contributed by Victor Stinner in :issue:`28799`.)
|
|
|
|
|
|
wave
|
|
----
|
|
|
|
:func:`wave.openfp` has been deprecated and will be removed in Python 3.9.
|
|
Use :func:`wave.open` instead.
|
|
(Contributed by Brian Curtin in :issue:`31985`.)
|
|
|
|
|
|
Deprecated functions and types of the C API
|
|
===========================================
|
|
|
|
Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with
|
|
a macro if ``Py_LIMITED_API`` is not set or set to a value in the range
|
|
between ``0x03050400`` and ``0x03060000`` (not inclusive), or is ``0x03060100``
|
|
or higher. (Contributed by Serhiy Storchaka in :issue:`27867`.)
|
|
|
|
:c:func:`PyOS_AfterFork` has been deprecated. Use :c:func:`PyOS_BeforeFork`,
|
|
:c:func:`PyOS_AfterFork_Parent` or :c:func:`PyOS_AfterFork_Child()` instead.
|
|
(Contributed by Antoine Pitrou in :issue:`16500`.)
|
|
|
|
|
|
.. _37-platform-support-removals:
|
|
|
|
Platform Support Removals
|
|
=========================
|
|
|
|
* FreeBSD 9 and older are no longer officially supported.
|
|
* For full Unicode support, including within extension modules, \*nix platforms
|
|
are now expected to provide at least one of ``C.UTF-8`` (full locale),
|
|
``C.utf8`` (full locale) or ``UTF-8`` (``LC_CTYPE``-only locale) as an
|
|
alternative to the legacy ``ASCII``-based ``C`` locale.
|
|
* OpenSSL 0.9.8 and 1.0.1 are no longer supported, which means building CPython
|
|
3.7 with SSL/TLS support on older platforms still using these versions
|
|
requires custom build options that link to a more recent version of OpenSSL.
|
|
|
|
Notably, this issue affects the Debian 8 (aka "jessie") and Ubuntu 14.04
|
|
(aka "Trusty") LTS Linux distributions, as they still use OpenSSL 1.0.1 by
|
|
default.
|
|
|
|
Debian 9 ("stretch") and Ubuntu 16.04 ("xenial"), as well as recent releases
|
|
of other LTS Linux releases (e.g. RHEL/CentOS 7.5, SLES 12-SP3), use OpenSSL
|
|
1.0.2 or later, and remain supported in the default build configuration.
|
|
|
|
CPython's own :source:`CI configuration file <.travis.yml>` provides an
|
|
example of using the SSL
|
|
:source:`compatibility testing infrastructure <Tools/ssl/multissltests.py>` in
|
|
CPython's test suite to build and link against OpenSSL 1.1.0 rather than an
|
|
outdated system provided OpenSSL.
|
|
|
|
|
|
API and Feature Removals
|
|
========================
|
|
|
|
The following features and APIs have been removed from Python 3.7:
|
|
|
|
* The ``os.stat_float_times()`` function has been removed. It was introduced in
|
|
Python 2.3 for backward compatibility with Python 2.2, and was deprecated
|
|
since Python 3.1.
|
|
|
|
* Unknown escapes consisting of ``'\'`` and an ASCII letter in replacement
|
|
templates for :func:`re.sub` were deprecated in Python 3.5, and will now
|
|
cause an error.
|
|
|
|
* Removed support of the *exclude* argument in :meth:`tarfile.TarFile.add`.
|
|
It was deprecated in Python 2.7 and 3.2. Use the *filter* argument instead.
|
|
|
|
* The ``splitunc()`` function in the :mod:`ntpath` module was deprecated in
|
|
Python 3.1, and has now been removed. Use the :func:`~os.path.splitdrive`
|
|
function instead.
|
|
|
|
* :func:`collections.namedtuple` no longer supports the *verbose* parameter
|
|
or ``_source`` attribute which showed the generated source code for the
|
|
named tuple class. This was part of an optimization designed to speed-up
|
|
class creation. (Contributed by Jelle Zijlstra with further improvements
|
|
by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.)
|
|
|
|
* Functions :func:`bool`, :func:`float`, :func:`list` and :func:`tuple` no
|
|
longer take keyword arguments. The first argument of :func:`int` can now
|
|
be passed only as positional argument.
|
|
|
|
* Removed previously deprecated in Python 2.4 classes ``Plist``, ``Dict`` and
|
|
``_InternalDict`` in the :mod:`plistlib` module. Dict values in the result
|
|
of functions :func:`~plistlib.readPlist` and
|
|
:func:`~plistlib.readPlistFromBytes` are now normal dicts. You no longer
|
|
can use attribute access to access items of these dictionaries.
|
|
|
|
* The ``asyncio.windows_utils.socketpair()`` function has been
|
|
removed. Use the :func:`socket.socketpair` function instead,
|
|
it is available on all platforms since Python 3.5.
|
|
``asyncio.windows_utils.socketpair`` was just an alias to
|
|
``socket.socketpair`` on Python 3.5 and newer.
|
|
|
|
* :mod:`asyncio` no longer exports the :mod:`selectors` and
|
|
:mod:`_overlapped` modules as ``asyncio.selectors`` and
|
|
``asyncio._overlapped``. Replace ``from asyncio import selectors`` with
|
|
``import selectors``.
|
|
|
|
* Direct instantiation of :class:`ssl.SSLSocket` and :class:`ssl.SSLObject`
|
|
objects is now prohibited. The constructors were never documented, tested,
|
|
or designed as public constructors. Users were supposed to use
|
|
:func:`ssl.wrap_socket` or :class:`ssl.SSLContext`.
|
|
(Contributed by Christian Heimes in :issue:`32951`.)
|
|
|
|
* The unused :mod:`distutils` ``install_misc`` command has been removed.
|
|
(Contributed by Eric N. Vander Weele in :issue:`29218`.)
|
|
|
|
|
|
Module Removals
|
|
===============
|
|
|
|
The ``fpectl`` module has been removed. It was never enabled by
|
|
default, never worked correctly on x86-64, and it changed the Python
|
|
ABI in ways that caused unexpected breakage of C extensions.
|
|
(Contributed by Nathaniel J. Smith in :issue:`29137`.)
|
|
|
|
|
|
Windows-only Changes
|
|
====================
|
|
|
|
The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without**
|
|
having to specify a minor version as well. So ``py -3-32`` and ``py -3-64``
|
|
become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms
|
|
are now accepted to force 64 bit python even if 32 bit would have otherwise
|
|
been used. If the specified version is not available py.exe will error exit.
|
|
(Contributed by Steve Barnes in :issue:`30291`.)
|
|
|
|
The launcher can be run as ``py -0`` to produce a list of the installed pythons,
|
|
*with default marked with an asterisk*. Running ``py -0p`` will include the paths.
|
|
If py is run with a version specifier that cannot be matched it will also print
|
|
the *short form* list of available specifiers.
|
|
(Contributed by Steve Barnes in :issue:`30362`.)
|
|
|
|
|
|
.. _porting-to-python-37:
|
|
|
|
Porting to Python 3.7
|
|
=====================
|
|
|
|
This section lists previously described changes and other bugfixes
|
|
that may require changes to your code.
|
|
|
|
|
|
Changes in Python Behavior
|
|
--------------------------
|
|
|
|
* :keyword:`async` and :keyword:`await` names are now reserved keywords.
|
|
Code using these names as identifiers will now raise a :exc:`SyntaxError`.
|
|
(Contributed by Jelle Zijlstra in :issue:`30406`.)
|
|
|
|
* :pep:`479` is enabled for all code in Python 3.7, meaning that
|
|
:exc:`StopIteration` exceptions raised directly or indirectly in
|
|
coroutines and generators are transformed into :exc:`RuntimeError`
|
|
exceptions.
|
|
(Contributed by Yury Selivanov in :issue:`32670`.)
|
|
|
|
* :meth:`object.__aiter__` methods can no longer be declared as
|
|
asynchronous. (Contributed by Yury Selivanov in :issue:`31709`.)
|
|
|
|
* Due to an oversight, earlier Python versions erroneously accepted the
|
|
following syntax::
|
|
|
|
f(1 for x in [1],)
|
|
|
|
class C(1 for x in [1]):
|
|
pass
|
|
|
|
Python 3.7 now correctly raises a :exc:`SyntaxError`, as a generator
|
|
expression always needs to be directly inside a set of parentheses
|
|
and cannot have a comma on either side, and the duplication of the
|
|
parentheses can be omitted only on calls.
|
|
(Contributed by Serhiy Storchaka in :issue:`32012` and :issue:`32023`.)
|
|
|
|
* When using the :option:`-m` switch, the initial working directory is now added
|
|
to :data:`sys.path`, rather than an empty string (which dynamically denoted
|
|
the current working directory at the time of each import). Any programs that
|
|
are checking for the empty string, or otherwise relying on the previous
|
|
behaviour, will need to be updated accordingly (e.g. by also checking for
|
|
``os.getcwd()`` or ``os.path.dirname(__main__.__file__)``, depending on why
|
|
the code was checking for the empty string in the first place).
|
|
|
|
|
|
Changes in the Python API
|
|
-------------------------
|
|
|
|
* :meth:`socketserver.ThreadingMixIn.server_close` now waits until all
|
|
non-daemon threads complete. Set the new
|
|
:attr:`socketserver.ThreadingMixIn.block_on_close` class attribute to
|
|
``False`` to get the pre-3.7 behaviour.
|
|
(Contributed by Victor Stinner in :issue:`31233` and :issue:`33540`.)
|
|
|
|
* :meth:`socketserver.ForkingMixIn.server_close` now waits until all
|
|
child processes complete. Set the new
|
|
:attr:`socketserver.ForkingMixIn.block_on_close` class attribute to ``False``
|
|
to get the pre-3.7 behaviour.
|
|
(Contributed by Victor Stinner in :issue:`31151` and :issue:`33540`.)
|
|
|
|
* The :func:`locale.localeconv` function now temporarily sets the ``LC_CTYPE``
|
|
locale to the value of ``LC_NUMERIC`` in some cases.
|
|
(Contributed by Victor Stinner in :issue:`31900`.)
|
|
|
|
* :meth:`pkgutil.walk_packages` now raises a :exc:`ValueError` if *path* is
|
|
a string. Previously an empty list was returned.
|
|
(Contributed by Sanyam Khurana in :issue:`24744`.)
|
|
|
|
* A format string argument for :meth:`string.Formatter.format`
|
|
is now :ref:`positional-only <positional-only_parameter>`.
|
|
Passing it as a keyword argument was deprecated in Python 3.5. (Contributed
|
|
by Serhiy Storchaka in :issue:`29193`.)
|
|
|
|
* Attributes :attr:`~http.cookies.Morsel.key`,
|
|
:attr:`~http.cookies.Morsel.value` and
|
|
:attr:`~http.cookies.Morsel.coded_value` of class
|
|
:class:`http.cookies.Morsel` are now read-only.
|
|
Assigning to them was deprecated in Python 3.5.
|
|
Use the :meth:`~http.cookies.Morsel.set` method for setting them.
|
|
(Contributed by Serhiy Storchaka in :issue:`29192`.)
|
|
|
|
* The *mode* argument of :func:`os.makedirs` no longer affects the file
|
|
permission bits of newly-created intermediate-level directories.
|
|
To set their file permission bits you can set the umask before invoking
|
|
``makedirs()``.
|
|
(Contributed by Serhiy Storchaka in :issue:`19930`.)
|
|
|
|
* The :attr:`struct.Struct.format` type is now :class:`str` instead of
|
|
:class:`bytes`. (Contributed by Victor Stinner in :issue:`21071`.)
|
|
|
|
* :func:`~cgi.parse_multipart` now accepts the *encoding* and *errors*
|
|
arguments and returns the same results as
|
|
:class:`~FieldStorage`: for non-file fields, the value associated to a key
|
|
is a list of strings, not bytes.
|
|
(Contributed by Pierre Quentel in :issue:`29979`.)
|
|
|
|
* Due to internal changes in :mod:`socket`, calling :func:`socket.fromshare`
|
|
on a socket created by :func:`socket.share <socket.socket.share>` in older
|
|
Python versions is not supported.
|
|
|
|
* ``repr`` for :exc:`BaseException` has changed to not include the trailing
|
|
comma. Most exceptions are affected by this change.
|
|
(Contributed by Serhiy Storchaka in :issue:`30399`.)
|
|
|
|
* ``repr`` for :class:`datetime.timedelta` has changed to include the keyword
|
|
arguments in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.)
|
|
|
|
* Because :func:`shutil.rmtree` is now implemented using the :func:`os.scandir`
|
|
function, the user specified handler *onerror* is now called with the first
|
|
argument ``os.scandir`` instead of ``os.listdir`` when listing the directory
|
|
is failed.
|
|
|
|
* Support for nested sets and set operations in regular expressions as in
|
|
`Unicode Technical Standard #18`_ might be added in the future. This would
|
|
change the syntax. To facilitate this future change a :exc:`FutureWarning`
|
|
will be raised in ambiguous cases for the time being.
|
|
That include sets starting with a literal ``'['`` or containing literal
|
|
character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To
|
|
avoid a warning, escape them with a backslash.
|
|
(Contributed by Serhiy Storchaka in :issue:`30349`.)
|
|
|
|
.. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/
|
|
|
|
* The result of splitting a string on a :mod:`regular expression <re>`
|
|
that could match an empty string has been changed. For example
|
|
splitting on ``r'\s*'`` will now split not only on whitespaces as it
|
|
did previously, but also on empty strings before all non-whitespace
|
|
characters and just before the end of the string.
|
|
The previous behavior can be restored by changing the pattern
|
|
to ``r'\s+'``. A :exc:`FutureWarning` was emitted for such patterns since
|
|
Python 3.5.
|
|
|
|
For patterns that match both empty and non-empty strings, the result of
|
|
searching for all matches may also be changed in other cases. For example
|
|
in the string ``'a\n\n'``, the pattern ``r'(?m)^\s*?$'`` will not only
|
|
match empty strings at positions 2 and 3, but also the string ``'\n'`` at
|
|
positions 2--3. To match only blank lines, the pattern should be rewritten
|
|
as ``r'(?m)^[^\S\n]*$'``.
|
|
|
|
:func:`re.sub()` now replaces empty matches adjacent to a previous
|
|
non-empty match. For example ``re.sub('x*', '-', 'abxd')`` returns now
|
|
``'-a-b--d-'`` instead of ``'-a-b-d-'`` (the first minus between 'b' and
|
|
'd' replaces 'x', and the second minus replaces an empty string between
|
|
'x' and 'd').
|
|
|
|
(Contributed by Serhiy Storchaka in :issue:`25054` and :issue:`32308`.)
|
|
|
|
* Change :func:`re.escape` to only escape regex special characters instead
|
|
of escaping all characters other than ASCII letters, numbers, and ``'_'``.
|
|
(Contributed by Serhiy Storchaka in :issue:`29995`.)
|
|
|
|
* :class:`tracemalloc.Traceback` frames are now sorted from oldest to most
|
|
recent to be more consistent with :mod:`traceback`.
|
|
(Contributed by Jesse Bakker in :issue:`32121`.)
|
|
|
|
* On OSes that support :const:`socket.SOCK_NONBLOCK` or
|
|
:const:`socket.SOCK_CLOEXEC` bit flags, the
|
|
:attr:`socket.type <socket.socket.type>` no longer has them applied.
|
|
Therefore, checks like ``if sock.type == socket.SOCK_STREAM``
|
|
work as expected on all platforms.
|
|
(Contributed by Yury Selivanov in :issue:`32331`.)
|
|
|
|
* On Windows the default for the *close_fds* argument of
|
|
:class:`subprocess.Popen` was changed from :const:`False` to :const:`True`
|
|
when redirecting the standard handles. If you previously depended on handles
|
|
being inherited when using :class:`subprocess.Popen` with standard io
|
|
redirection, you will have to pass ``close_fds=False`` to preserve the
|
|
previous behaviour, or use
|
|
:attr:`STARTUPINFO.lpAttributeList <subprocess.STARTUPINFO.lpAttributeList>`.
|
|
|
|
* :meth:`importlib.machinery.PathFinder.invalidate_caches` -- which implicitly
|
|
affects :func:`importlib.invalidate_caches` -- now deletes entries
|
|
in :data:`sys.path_importer_cache` which are set to ``None``.
|
|
(Contributed by Brett Cannon in :issue:`33169`.)
|
|
|
|
* In :mod:`asyncio`,
|
|
:meth:`loop.sock_recv() <asyncio.loop.sock_recv>`,
|
|
:meth:`loop.sock_sendall() <asyncio.loop.sock_sendall>`,
|
|
:meth:`loop.sock_accept() <asyncio.loop.sock_accept>`,
|
|
:meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`,
|
|
:meth:`loop.getnameinfo() <asyncio.loop.getnameinfo>`
|
|
have been changed to be proper coroutine methods to match their
|
|
documentation. Previously, these methods returned :class:`asyncio.Future`
|
|
instances.
|
|
(Contributed by Yury Selivanov in :issue:`32327`.)
|
|
|
|
* :attr:`asyncio.Server.sockets` now returns a copy of the internal list
|
|
of server sockets, instead of returning it directly.
|
|
(Contributed by Yury Selivanov in :issue:`32662`.)
|
|
|
|
* :attr:`Struct.format <struct.Struct.format>` is now a :class:`str` instance
|
|
instead of a :class:`bytes` instance.
|
|
(Contributed by Victor Stinner in :issue:`21071`.)
|
|
|
|
* :mod:`argparse` subparsers can now be made mandatory by passing ``required=True``
|
|
to :meth:`ArgumentParser.add_subparsers() <argparse.ArgumentParser.add_subparsers>`.
|
|
(Contributed by Anthony Sottile in :issue:`26510`.)
|
|
|
|
* :meth:`ast.literal_eval()` is now stricter. Addition and subtraction of
|
|
arbitrary numbers are no longer allowed.
|
|
(Contributed by Serhiy Storchaka in :issue:`31778`.)
|
|
|
|
* :meth:`Calendar.itermonthdates <calendar.Calendar.itermonthdates>`
|
|
will now consistently raise an exception when a date falls outside of the
|
|
``0001-01-01`` through ``9999-12-31`` range. To support applications that
|
|
cannot tolerate such exceptions, the new
|
|
:meth:`Calendar.itermonthdays3 <calendar.Calendar.itermonthdays3>` and
|
|
:meth:`Calendar.itermonthdays4 <calendar.Calendar.itermonthdays4>` can be used.
|
|
The new methods return tuples and are not restricted by the range supported by
|
|
:class:`datetime.date`.
|
|
(Contributed by Alexander Belopolsky in :issue:`28292`.)
|
|
|
|
* :class:`collections.ChainMap` now preserves the order of the underlying
|
|
mappings. (Contributed by Raymond Hettinger in :issue:`32792`.)
|
|
|
|
* The ``submit()`` method of :class:`concurrent.futures.ThreadPoolExecutor`
|
|
and :class:`concurrent.futures.ProcessPoolExecutor` now raises
|
|
a :exc:`RuntimeError` if called during interpreter shutdown.
|
|
(Contributed by Mark Nemec in :issue:`33097`.)
|
|
|
|
* The :class:`configparser.ConfigParser` constructor now uses ``read_dict()``
|
|
to process the default values, making its behavior consistent with the
|
|
rest of the parser. Non-string keys and values in the defaults
|
|
dictionary are now being implicitly converted to strings.
|
|
(Contributed by James Tocknell in :issue:`23835`.)
|
|
|
|
* Several undocumented internal imports were removed.
|
|
One example is that ``os.errno`` is no longer available; use ``import errno``
|
|
directly instead.
|
|
Note that such undocumented internal imports may be removed any time without
|
|
notice, even in micro version releases.
|
|
|
|
|
|
Changes in the C API
|
|
--------------------
|
|
|
|
The function :c:func:`PySlice_GetIndicesEx` is considered unsafe for
|
|
resizable sequences. If the slice indices are not instances of :class:`int`,
|
|
but objects that implement the :meth:`!__index__` method, the sequence can be
|
|
resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This
|
|
can lead to returning indices out of the length of the sequence. For
|
|
avoiding possible problems use new functions :c:func:`PySlice_Unpack` and
|
|
:c:func:`PySlice_AdjustIndices`.
|
|
(Contributed by Serhiy Storchaka in :issue:`27867`.)
|
|
|
|
|
|
CPython bytecode changes
|
|
------------------------
|
|
|
|
There are two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`.
|
|
(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.)
|
|
|
|
The :opcode:`STORE_ANNOTATION` opcode has been removed.
|
|
(Contributed by Mark Shannon in :issue:`32550`.)
|
|
|
|
|
|
Windows-only Changes
|
|
--------------------
|
|
|
|
The file used to override :data:`sys.path` is now called
|
|
``<python-executable>._pth`` instead of ``'sys.path'``.
|
|
See :ref:`finding_modules` for more information.
|
|
(Contributed by Steve Dower in :issue:`28137`.)
|
|
|
|
|
|
Other CPython implementation changes
|
|
------------------------------------
|
|
|
|
In preparation for potential future changes to the public CPython runtime
|
|
initialization API (see :pep:`432` for an initial, but somewhat outdated,
|
|
draft), CPython's internal startup
|
|
and configuration management logic has been significantly refactored. While
|
|
these updates are intended to be entirely transparent to both embedding
|
|
applications and users of the regular CPython CLI, they're being mentioned
|
|
here as the refactoring changes the internal order of various operations
|
|
during interpreter startup, and hence may uncover previously latent defects,
|
|
either in embedding applications, or in CPython itself.
|
|
(Initially contributed by Nick Coghlan and Eric Snow as part of
|
|
:issue:`22257`, and further updated by Nick, Eric, and Victor Stinner in a
|
|
number of other issues). Some known details affected:
|
|
|
|
* :c:func:`PySys_AddWarnOptionUnicode` is not currently usable by embedding
|
|
applications due to the requirement to create a Unicode object prior to
|
|
calling `Py_Initialize`. Use :c:func:`PySys_AddWarnOption` instead.
|
|
|
|
* warnings filters added by an embedding application with
|
|
:c:func:`PySys_AddWarnOption` should now more consistently take precedence
|
|
over the default filters set by the interpreter
|
|
|
|
Due to changes in the way the default warnings filters are configured,
|
|
setting :c:data:`Py_BytesWarningFlag` to a value greater than one is no longer
|
|
sufficient to both emit :exc:`BytesWarning` messages and have them converted
|
|
to exceptions. Instead, the flag must be set (to cause the warnings to be
|
|
emitted in the first place), and an explicit ``error::BytesWarning``
|
|
warnings filter added to convert them to exceptions.
|
|
|
|
Due to a change in the way docstrings are handled by the compiler, the
|
|
implicit ``return None`` in a function body consisting solely of a docstring
|
|
is now marked as occurring on the same line as the docstring, not on the
|
|
function's header line.
|
|
|
|
The current exception state has been moved from the frame object to the co-routine.
|
|
This simplified the interpreter and fixed a couple of obscure bugs caused by
|
|
having swap exception state when entering or exiting a generator.
|
|
(Contributed by Mark Shannon in :issue:`25612`.)
|
|
|
|
Notable changes in Python 3.7.1
|
|
===============================
|
|
|
|
Starting in 3.7.1, :c:func:`Py_Initialize` now consistently reads and respects
|
|
all of the same environment settings as :c:func:`Py_Main` (in earlier Python
|
|
versions, it respected an ill-defined subset of those environment variables,
|
|
while in Python 3.7.0 it didn't read any of them due to :issue:`34247`). If
|
|
this behavior is unwanted, set :c:data:`Py_IgnoreEnvironmentFlag` to 1 before
|
|
calling :c:func:`Py_Initialize`.
|
|
|
|
In 3.7.1 the C API for Context Variables
|
|
:ref:`was updated <contextvarsobjects_pointertype_change>` to use
|
|
:c:type:`PyObject` pointers. See also :issue:`34762`.
|
|
|
|
In 3.7.1 the :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token
|
|
when provided with input that does not have a trailing new line. This behavior
|
|
now matches what the C tokenizer does internally.
|
|
(Contributed by Ammar Askar in :issue:`33899`.)
|
|
|
|
Notable changes in Python 3.7.2
|
|
===============================
|
|
|
|
In 3.7.2, :mod:`venv` on Windows no longer copies the original binaries, but
|
|
creates redirector scripts named ``python.exe`` and ``pythonw.exe`` instead.
|
|
This resolves a long standing issue where all virtual environments would have
|
|
to be upgraded or recreated with each Python update. However, note that this
|
|
release will still require recreation of virtual environments in order to get
|
|
the new scripts.
|
|
|
|
Notable changes in Python 3.7.6
|
|
===============================
|
|
|
|
Due to significant security concerns, the *reuse_address* parameter of
|
|
:meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is
|
|
because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more
|
|
details, see the documentation for ``loop.create_datagram_endpoint()``.
|
|
(Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in
|
|
:issue:`37228`.)
|
|
|
|
Notable changes in Python 3.7.10
|
|
================================
|
|
|
|
Earlier Python versions allowed using both ``;`` and ``&`` as
|
|
query parameter separators in :func:`urllib.parse.parse_qs` and
|
|
:func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with
|
|
newer W3C recommendations, this has been changed to allow only a single
|
|
separator key, with ``&`` as the default. This change also affects
|
|
:func:`cgi.parse` and :func:`cgi.parse_multipart` as they use the affected
|
|
functions internally. For more details, please see their respective
|
|
documentation.
|
|
(Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.)
|