****************************

What's New In Python 3.6

****************************

:Editors: Elvis Pranskevichus <elvis@magic.io>, Yury Selivanov <yury@magic.io>

.. Rules for maintenance:

* Anyone can add text to this document. Do not spend very much time

on the wording of your changes, because your text will probably

get rewritten to some degree.

* The maintainer will go through Misc/NEWS periodically and add

changes; it's therefore more important to add your changes to

Misc/NEWS than to this file.

* This is not a complete list of every single change; completeness

is the purpose of Misc/NEWS. Some changes I consider too small

or esoteric to include. If such a change is added to the text,

I'll just remove it. (This is another reason you shouldn't spend

too much time on writing your addition.)

* If you want to draw your new text to the attention of the

maintainer, add 'XXX' to the beginning of the paragraph or

section.

* It's OK to just add a fragmentary note about a change. For

example: "XXX Describe the transmogrify() function added to the

socket module." The maintainer will research the change and

write the necessary text.

* You can comment out your additions if you like, but it's not

necessary (especially when a final release is some months away).

* Credit the author of a patch or bugfix. Just the name is

sufficient; the e-mail address isn't necessary.

* It's helpful to add the bug/patch number as a comment:

XXX Describe the transmogrify() function added to the socket

module.

(Contributed by P.Y. Developer in :issue:`12345`.)

This saves the maintainer the effort of going through the Mercurial log

when researching a change.

This article explains the new features in Python 3.6, compared to 3.5.

Python 3.6 was released on December 23, 2016. See the

`changelog <https://docs.python.org/3.6/whatsnew/changelog.html>`_ for a full

list of changes.

.. seealso::

:pep:`494` - Python 3.6 Release Schedule

Summary -- Release highlights

=============================

New syntax features:

* :ref:`PEP 498 <whatsnew36-pep498>`, formatted string literals.

* :ref:`PEP 515 <whatsnew36-pep515>`, underscores in numeric literals.

* :ref:`PEP 526 <whatsnew36-pep526>`, syntax for variable annotations.

* :ref:`PEP 525 <whatsnew36-pep525>`, asynchronous generators.

* :ref:`PEP 530 <whatsnew36-pep530>`: asynchronous comprehensions.

New library modules:

* :mod:`secrets`: :ref:`PEP 506 -- Adding A Secrets Module To The Standard Library <whatsnew36-pep506>`.

CPython implementation improvements:

* The :ref:`dict <typesmapping>` type has been reimplemented to use

a :ref:`more compact representation <whatsnew36-compactdict>`

based on `a proposal by Raymond Hettinger

<https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_

and similar to the `PyPy dict implementation`_. This resulted in dictionaries

using 20% to 25% less memory when compared to Python 3.5.

* Customization of class creation has been simplified with the

:ref:`new protocol <whatsnew36-pep487>`.

* The class attribute definition order is

:ref:`now preserved <whatsnew36-pep520>`.

* The order of elements in ``**kwargs`` now

:ref:`corresponds to the order <whatsnew36-pep468>` in which keyword

arguments were passed to the function.

* DTrace and SystemTap :ref:`probing support <whatsnew36-tracing>` has

been added.

* The new :ref:`PYTHONMALLOC <whatsnew36-pythonmalloc>` environment variable

can now be used to debug the interpreter memory allocation and access

errors.

Significant improvements in the standard library:

* The :mod:`asyncio` module has received new features, significant

usability and performance improvements, and a fair amount of bug fixes.

Starting with Python 3.6 the ``asyncio`` module is no longer provisional

and its API is considered stable.

* A new :ref:`file system path protocol <whatsnew36-pep519>` has been

implemented to support :term:`path-like objects <path-like object>`.

All standard library functions operating on paths have been updated to

work with the new protocol.

* The :mod:`datetime` module has gained support for

:ref:`Local Time Disambiguation <whatsnew36-pep495>`.

* The :mod:`typing` module received a number of

:ref:`improvements <whatsnew36-typing>`.

* The :mod:`tracemalloc` module has been significantly reworked

and is now used to provide better output for :exc:`ResourceWarning`

as well as provide better diagnostics for memory allocation errors.

See the :ref:`PYTHONMALLOC section <whatsnew36-pythonmalloc>` for more

information.

Security improvements:

* The new :mod:`secrets` module has been added to simplify the generation of

cryptographically strong pseudo-random numbers suitable for

managing secrets such as account authentication, tokens, and similar.

* On Linux, :func:`os.urandom` now blocks until the system urandom entropy

pool is initialized to increase the security. See the :pep:`524` for the

rationale.

* The :mod:`hashlib` and :mod:`ssl` modules now support OpenSSL 1.1.0.

* The default settings and feature set of the :mod:`ssl` module have been

improved.

* The :mod:`hashlib` module received support for the BLAKE2, SHA-3 and SHAKE

hash algorithms and the :func:`~hashlib.scrypt` key derivation function.

Windows improvements:

* :ref:`PEP 528 <whatsnew36-pep528>` and :ref:`PEP 529 <whatsnew36-pep529>`,

Windows filesystem and console encoding changed to UTF-8.

* The ``py.exe`` launcher, when used interactively, no longer prefers

Python 2 over Python 3 when the user doesn't specify a version (via

command line arguments or a config file). Handling of shebang lines

remains unchanged - "python" refers to Python 2 in that case.

* ``python.exe`` and ``pythonw.exe`` have been marked as long-path aware,

which means that the 260 character path limit may no longer apply.

See :ref:`removing the MAX_PATH limitation <max-path>` for details.

* A ``._pth`` file can be added to force isolated mode and fully specify

all search paths to avoid registry and environment lookup. See

:ref:`the documentation <windows_finding_modules>` for more information.

* A ``python36.zip`` file now works as a landmark to infer

:envvar:`PYTHONHOME`. See :ref:`the documentation <windows_finding_modules>` for

more information.

.. _PyPy dict implementation: https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html

New Features

============

.. _whatsnew36-pep498:

PEP 498: Formatted string literals

----------------------------------

:pep:`498` introduces a new kind of string literals: *f-strings*, or

:ref:`formatted string literals <f-strings>`.

Formatted string literals are prefixed with ``'f'`` and are similar to

the format strings accepted by :meth:`str.format`. They contain replacement

fields surrounded by curly braces. The replacement fields are expressions,

which are evaluated at run time, and then formatted using the

:func:`format` protocol::

>>> name = "Fred"

>>> f"He said his name is {name}."

'He said his name is Fred.'

>>> width = 10

>>> precision = 4

>>> value = decimal.Decimal("12.34567")

>>> f"result: {value:{width}.{precision}}" # nested fields

'result: 12.35'

.. seealso::

:pep:`498` -- Literal String Interpolation.

PEP written and implemented by Eric V. Smith.

:ref:`Feature documentation <f-strings>`.

.. _whatsnew36-pep526:

PEP 526: Syntax for variable annotations

----------------------------------------

:pep:`484` introduced the standard for type annotations of function

parameters, a.k.a. type hints. This PEP adds syntax to Python for annotating

the types of variables including class variables and instance variables::

primes: List[int] = []

captain: str # Note: no initial value!

class Starship:

stats: Dict[str, int] = {}

Just as for function annotations, the Python interpreter does not attach any

particular meaning to variable annotations and only stores them in the

``__annotations__`` attribute of a class or module.

In contrast to variable declarations in statically typed languages,

the goal of annotation syntax is to provide an easy way to specify structured

type metadata for third party tools and libraries via the abstract syntax tree

and the ``__annotations__`` attribute.

.. seealso::

:pep:`526` -- Syntax for variable annotations.

PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach,

and Guido van Rossum. Implemented by Ivan Levkivskyi.

Tools that use or will use the new syntax:

`mypy <https://www.mypy-lang.org/>`_,

`pytype <https://github.com/google/pytype>`_, PyCharm, etc.

.. _whatsnew36-pep515:

PEP 515: Underscores in Numeric Literals

----------------------------------------

:pep:`515` adds the ability to use underscores in numeric literals for

improved readability. For example::

>>> 1_000_000_000_000_000

1000000000000000

>>> 0x_FF_FF_FF_FF

4294967295

Single underscores are allowed between digits and after any base

specifier. Leading, trailing, or multiple underscores in a row are not

allowed.

The :ref:`string formatting <formatspec>` language also now has support

for the ``'_'`` option to signal the use of an underscore for a thousands

separator for floating-point presentation types and for integer

presentation type ``'d'``. For integer presentation types ``'b'``,

``'o'``, ``'x'``, and ``'X'``, underscores will be inserted every 4

digits::

>>> '{:_}'.format(1000000)

'1_000_000'

>>> '{:_x}'.format(0xFFFFFFFF)

'ffff_ffff'

.. seealso::

:pep:`515` -- Underscores in Numeric Literals

PEP written by Georg Brandl and Serhiy Storchaka.

.. _whatsnew36-pep525:

PEP 525: Asynchronous Generators

--------------------------------

:pep:`492` introduced support for native coroutines and ``async`` / ``await``

syntax to Python 3.5. A notable limitation of the Python 3.5 implementation

is that it was not possible to use ``await`` and ``yield`` in the same

function body. In Python 3.6 this restriction has been lifted, making it

possible to define *asynchronous generators*::

async def ticker(delay, to):

"""Yield numbers from 0 to *to* every *delay* seconds."""

for i in range(to):

yield i

await asyncio.sleep(delay)

The new syntax allows for faster and more concise code.

.. seealso::

:pep:`525` -- Asynchronous Generators

PEP written and implemented by Yury Selivanov.

.. _whatsnew36-pep530:

PEP 530: Asynchronous Comprehensions

------------------------------------

:pep:`530` adds support for using ``async for`` in list, set, dict

comprehensions and generator expressions::

result = [i async for i in aiter() if i % 2]

Additionally, ``await`` expressions are supported in all kinds

of comprehensions::

result = [await fun() for fun in funcs if await condition()]

.. seealso::

:pep:`530` -- Asynchronous Comprehensions

PEP written and implemented by Yury Selivanov.

.. _whatsnew36-pep487:

PEP 487: Simpler customization of class creation

------------------------------------------------

It is now possible to customize subclass creation without using a metaclass.

The new ``__init_subclass__`` classmethod will be called on the base class

whenever a new subclass is created::

class PluginBase:

subclasses = []

def __init_subclass__(cls, **kwargs):

super().__init_subclass__(**kwargs)

cls.subclasses.append(cls)

class Plugin1(PluginBase):

pass

class Plugin2(PluginBase):

pass

In order to allow zero-argument :func:`super` calls to work correctly from

:meth:`~object.__init_subclass__` implementations, custom metaclasses must

ensure that the new ``__classcell__`` namespace entry is propagated to

``type.__new__`` (as described in :ref:`class-object-creation`).

.. seealso::

:pep:`487` -- Simpler customization of class creation

PEP written and implemented by Martin Teichmann.

:ref:`Feature documentation <class-customization>`

.. _whatsnew36-pep487-descriptors:

PEP 487: Descriptor Protocol Enhancements

-----------------------------------------

:pep:`487` extends the descriptor protocol to include the new optional

:meth:`~object.__set_name__` method. Whenever a new class is defined, the new

method will be called on all descriptors included in the definition, providing

them with a reference to the class being defined and the name given to the

descriptor within the class namespace. In other words, instances of

descriptors can now know the attribute name of the descriptor in the

owner class::

class IntField:

def __get__(self, instance, owner):

return instance.__dict__[self.name]

def __set__(self, instance, value):

if not isinstance(value, int):

raise ValueError(f'expecting integer in {self.name}')

instance.__dict__[self.name] = value

# this is the new initializer:

def __set_name__(self, owner, name):

self.name = name

class Model:

int_field = IntField()

.. seealso::

:pep:`487` -- Simpler customization of class creation

PEP written and implemented by Martin Teichmann.

:ref:`Feature documentation <descriptors>`

.. _whatsnew36-pep519:

PEP 519: Adding a file system path protocol

-------------------------------------------

File system paths have historically been represented as :class:`str`

or :class:`bytes` objects. This has led to people who write code which

operate on file system paths to assume that such objects are only one

of those two types (an :class:`int` representing a file descriptor

does not count as that is not a file path). Unfortunately that

assumption prevents alternative object representations of file system

paths like :mod:`pathlib` from working with pre-existing code,

including Python's standard library.

To fix this situation, a new interface represented by

:class:`os.PathLike` has been defined. By implementing the

:meth:`~os.PathLike.__fspath__` method, an object signals that it

represents a path. An object can then provide a low-level

representation of a file system path as a :class:`str` or

:class:`bytes` object. This means an object is considered

:term:`path-like <path-like object>` if it implements

:class:`os.PathLike` or is a :class:`str` or :class:`bytes` object

which represents a file system path. Code can use :func:`os.fspath`,

:func:`os.fsdecode`, or :func:`os.fsencode` to explicitly get a

:class:`str` and/or :class:`bytes` representation of a path-like

object.

The built-in :func:`open` function has been updated to accept

:class:`os.PathLike` objects, as have all relevant functions in the

:mod:`os` and :mod:`os.path` modules, and most other functions and

classes in the standard library. The :class:`os.DirEntry` class

and relevant classes in :mod:`pathlib` have also been updated to

implement :class:`os.PathLike`.

The hope is that updating the fundamental functions for operating

on file system paths will lead to third-party code to implicitly

support all :term:`path-like objects <path-like object>` without any

code changes, or at least very minimal ones (e.g. calling

:func:`os.fspath` at the beginning of code before operating on a

path-like object).

Here are some examples of how the new interface allows for

:class:`pathlib.Path` to be used more easily and transparently with

pre-existing code::

>>> import pathlib

>>> with open(pathlib.Path("README")) as f:

... contents = f.read()

...

>>> import os.path

>>> os.path.splitext(pathlib.Path("some_file.txt"))

('some_file', '.txt')

>>> os.path.join("/a/b", pathlib.Path("c"))

'/a/b/c'

>>> import os

>>> os.fspath(pathlib.Path("some_file.txt"))

'some_file.txt'

(Implemented by Brett Cannon, Ethan Furman, Dusty Phillips, and Jelle Zijlstra.)

.. seealso::

:pep:`519` -- Adding a file system path protocol

PEP written by Brett Cannon and Koos Zevenhoven.

.. _whatsnew36-pep495:

PEP 495: Local Time Disambiguation

----------------------------------

In most world locations, there have been and will be times when local clocks

are moved back. In those times, intervals are introduced in which local

clocks show the same time twice in the same day. In these situations, the

information displayed on a local clock (or stored in a Python datetime

instance) is insufficient to identify a particular moment in time.

:pep:`495` adds the new *fold* attribute to instances of

:class:`datetime.datetime` and :class:`datetime.time` classes to differentiate

between two moments in time for which local times are the same::

>>> u0 = datetime(2016, 11, 6, 4, tzinfo=timezone.utc)

>>> for i in range(4):

... u = u0 + i*HOUR

... t = u.astimezone(Eastern)

... print(u.time(), 'UTC =', t.time(), t.tzname(), t.fold)

...

04:00:00 UTC = 00:00:00 EDT 0

05:00:00 UTC = 01:00:00 EDT 0

06:00:00 UTC = 01:00:00 EST 1

07:00:00 UTC = 02:00:00 EST 0

The values of the :attr:`fold <datetime.datetime.fold>` attribute have the

value ``0`` for all instances except those that represent the second

(chronologically) moment in time in an ambiguous case.

.. seealso::

:pep:`495` -- Local Time Disambiguation

PEP written by Alexander Belopolsky and Tim Peters, implementation

by Alexander Belopolsky.

.. _whatsnew36-pep529:

PEP 529: Change Windows filesystem encoding to UTF-8

----------------------------------------------------

Representing filesystem paths is best performed with str (Unicode) rather than

bytes. However, there are some situations where using bytes is sufficient and

correct.

Prior to Python 3.6, data loss could result when using bytes paths on Windows.

With this change, using bytes to represent paths is now supported on Windows,

provided those bytes are encoded with the encoding returned by

:func:`sys.getfilesystemencoding`, which now defaults to ``'utf-8'``.

Applications that do not use str to represent paths should use

:func:`os.fsencode` and :func:`os.fsdecode` to ensure their bytes are

correctly encoded. To revert to the previous behaviour, set

:envvar:`PYTHONLEGACYWINDOWSFSENCODING` or call

:func:`sys._enablelegacywindowsfsencoding`.

See :pep:`529` for more information and discussion of code modifications that

may be required.

.. _whatsnew36-pep528:

PEP 528: Change Windows console encoding to UTF-8

-------------------------------------------------

The default console on Windows will now accept all Unicode characters and

provide correctly read str objects to Python code. ``sys.stdin``,

``sys.stdout`` and ``sys.stderr`` now default to utf-8 encoding.

This change only applies when using an interactive console, and not when

redirecting files or pipes. To revert to the previous behaviour for interactive

console use, set :envvar:`PYTHONLEGACYWINDOWSSTDIO`.

.. seealso::

:pep:`528` -- Change Windows console encoding to UTF-8

PEP written and implemented by Steve Dower.

.. _whatsnew36-pep520:

PEP 520: Preserving Class Attribute Definition Order

----------------------------------------------------

Attributes in a class definition body have a natural ordering: the same

order in which the names appear in the source. This order is now

preserved in the new class's :attr:`~type.__dict__` attribute.

Also, the effective default class *execution* namespace (returned from

:ref:`type.__prepare__() <prepare>`) is now an insertion-order-preserving

mapping.

.. seealso::

:pep:`520` -- Preserving Class Attribute Definition Order

PEP written and implemented by Eric Snow.

.. _whatsnew36-pep468:

PEP 468: Preserving Keyword Argument Order

------------------------------------------

``**kwargs`` in a function signature is now guaranteed to be an

insertion-order-preserving mapping.

.. seealso::

:pep:`468` -- Preserving Keyword Argument Order

PEP written and implemented by Eric Snow.

.. _whatsnew36-compactdict:

New :ref:`dict <typesmapping>` implementation

---------------------------------------------

The :ref:`dict <typesmapping>` type now uses a "compact" representation

based on `a proposal by Raymond Hettinger

<https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_

which was `first implemented by PyPy

<https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_.

The memory usage of the new :func:`dict` is between 20% and 25% smaller

compared to Python 3.5.

The order-preserving aspect of this new implementation is considered an

implementation detail and should not be relied upon (this may change in

the future, but it is desired to have this new dict implementation in

the language for a few releases before changing the language spec to mandate

order-preserving semantics for all current and future Python

implementations; this also helps preserve backwards-compatibility

with older versions of the language where random iteration order is

still in effect, e.g. Python 3.5).

(Contributed by INADA Naoki in :issue:`27350`. Idea

`originally suggested by Raymond Hettinger

<https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.)

.. _whatsnew36-pep523:

PEP 523: Adding a frame evaluation API to CPython

-------------------------------------------------

While Python provides extensive support to customize how code

executes, one place it has not done so is in the evaluation of frame

objects. If you wanted some way to intercept frame evaluation in

Python there really wasn't any way without directly manipulating

function pointers for defined functions.

:pep:`523` changes this by providing an API to make frame

evaluation pluggable at the C level. This will allow for tools such

as debuggers and JITs to intercept frame evaluation before the

execution of Python code begins. This enables the use of alternative

evaluation implementations for Python code, tracking frame

evaluation, etc.

This API is not part of the limited C API and is marked as private to

signal that usage of this API is expected to be limited and only

applicable to very select, low-level use-cases. Semantics of the

API will change with Python as necessary.

.. seealso::

:pep:`523` -- Adding a frame evaluation API to CPython

PEP written by Brett Cannon and Dino Viehland.

.. _whatsnew36-pythonmalloc:

PYTHONMALLOC environment variable

---------------------------------

The new :envvar:`PYTHONMALLOC` environment variable allows setting the Python

memory allocators and installing debug hooks.

It is now possible to install debug hooks on Python memory allocators on Python

compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks:

* Newly allocated memory is filled with the byte ``0xCB``

* Freed memory is filled with the byte ``0xDB``

* Detect violations of the Python memory allocator API. For example,

:c:func:`PyObject_Free` called on a memory block allocated by

:c:func:`PyMem_Malloc`.

* Detect writes before the start of a buffer (buffer underflows)

* Detect writes after the end of a buffer (buffer overflows)

* Check that the :term:`GIL <global interpreter lock>` is held when allocator

functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and

:c:macro:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called.

Checking if the GIL is held is also a new feature of Python 3.6.

See the :c:func:`PyMem_SetupDebugHooks` function for debug hooks on Python

memory allocators.

It is now also possible to force the usage of the :c:func:`malloc` allocator of

the C library for all Python memory allocations using ``PYTHONMALLOC=malloc``.

This is helpful when using external memory debuggers like Valgrind on

a Python compiled in release mode.

On error, the debug hooks on Python memory allocators now use the

:mod:`tracemalloc` module to get the traceback where a memory block was

allocated.

Example of fatal error on buffer overflow using

``python3.6 -X tracemalloc=5`` (store 5 frames in traces)::

Debug memory block at address p=0x7fbcd41666f8: API 'o'

4 bytes originally requested

The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.

The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb):

at tail+0: 0x02 *** OUCH

at tail+1: 0xfb

at tail+2: 0xfb

at tail+3: 0xfb

at tail+4: 0xfb

at tail+5: 0xfb

at tail+6: 0xfb

at tail+7: 0xfb

The block was made by call #1233329 to debug malloc/realloc.

Data at p: 1a 2b 30 00

Memory block allocated at (most recent call first):

File "test/test_bytes.py", line 323

File "unittest/case.py", line 600

File "unittest/case.py", line 648

File "unittest/suite.py", line 122

File "unittest/suite.py", line 84

Fatal Python error: bad trailing pad byte

Current thread 0x00007fbcdbd32700 (most recent call first):

File "test/test_bytes.py", line 323 in test_hex

File "unittest/case.py", line 600 in run

File "unittest/case.py", line 648 in __call__

File "unittest/suite.py", line 122 in run

File "unittest/suite.py", line 84 in __call__

File "unittest/suite.py", line 122 in run

File "unittest/suite.py", line 84 in __call__

...

(Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.)

.. _whatsnew36-tracing:

DTrace and SystemTap probing support

------------------------------------

Python can now be built ``--with-dtrace`` which enables static markers

for the following events in the interpreter:

* function call/return

* garbage collection started/finished

* line of code executed.

This can be used to instrument running interpreters in production,

without the need to recompile specific :ref:`debug builds <debug-build>` or

providing application-specific profiling/debugging code.

More details in :ref:`instrumentation`.

The current implementation is tested on Linux and macOS. Additional

markers may be added in the future.

(Contributed by Łukasz Langa in :issue:`21590`, based on patches by

Jesús Cea Avión, David Malcolm, and Nikhil Benesch.)

Other Language Changes

======================

Some smaller changes made to the core Python language are:

* A ``global`` or ``nonlocal`` statement must now textually appear

before the first use of the affected name in the same scope.

Previously this was a :exc:`SyntaxWarning`.

* It is now possible to set a :ref:`special method <specialnames>` to

``None`` to indicate that the corresponding operation is not available.

For example, if a class sets :meth:`~object.__iter__` to ``None``, the class

is not iterable.

(Contributed by Andrew Barnert and Ivan Levkivskyi in :issue:`25958`.)

* Long sequences of repeated traceback lines are now abbreviated as

``"[Previous line repeated {count} more times]"`` (see

:ref:`whatsnew36-traceback` for an example).

(Contributed by Emanuel Barry in :issue:`26823`.)

* Import now raises the new exception :exc:`ModuleNotFoundError`

(subclass of :exc:`ImportError`) when it cannot find a module. Code

that currently checks for ImportError (in try-except) will still work.

(Contributed by Eric Snow in :issue:`15767`.)

* Class methods relying on zero-argument ``super()`` will now work correctly

when called from metaclass methods during class creation.

(Contributed by Martin Teichmann in :issue:`23722`.)

New Modules

===========

.. _whatsnew36-pep506:

secrets

-------

The main purpose of the new :mod:`secrets` module is to provide an obvious way

to reliably generate cryptographically strong pseudo-random values suitable

for managing secrets, such as account authentication, tokens, and similar.

.. warning::

Note that the pseudo-random generators in the :mod:`random` module

should *NOT* be used for security purposes. Use :mod:`secrets`

on Python 3.6+ and :func:`os.urandom` on Python 3.5 and earlier.

.. seealso::

:pep:`506` -- Adding A Secrets Module To The Standard Library

PEP written and implemented by Steven D'Aprano.

Improved Modules

================

array

-----

Exhausted iterators of :class:`array.array` will now stay exhausted even

if the iterated array is extended. This is consistent with the behavior

of other mutable sequences.

Contributed by Serhiy Storchaka in :issue:`26492`.

ast

---

The new :class:`ast.Constant` AST node has been added. It can be used

by external AST optimizers for the purposes of constant folding.

Contributed by Victor Stinner in :issue:`26146`.

asyncio

-------

Starting with Python 3.6 the ``asyncio`` module is no longer provisional and its

API is considered stable.

Notable changes in the :mod:`asyncio` module since Python 3.5.0

(all backported to 3.5.x due to the provisional status):

* The :func:`~asyncio.get_event_loop` function has been changed to

always return the currently running loop when called from coroutines

and callbacks.

(Contributed by Yury Selivanov in :issue:`28613`.)

* The :func:`~asyncio.ensure_future` function and all functions that

use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`,

now accept all kinds of :term:`awaitable objects <awaitable>`.

(Contributed by Yury Selivanov.)

* New :func:`~asyncio.run_coroutine_threadsafe` function to submit

coroutines to event loops from other threads.

(Contributed by Vincent Michel.)

* New :meth:`Transport.is_closing() <asyncio.BaseTransport.is_closing>`

method to check if the transport is closing or closed.

(Contributed by Yury Selivanov.)

* The :meth:`loop.create_server() <asyncio.loop.create_server>`

method can now accept a list of hosts.

(Contributed by Yann Sionneau.)

* New :meth:`loop.create_future() <asyncio.loop.create_future>`

method to create Future objects. This allows alternative event

loop implementations, such as

`uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster

:class:`asyncio.Future` implementation.

(Contributed by Yury Selivanov in :issue:`27041`.)

* New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>`

method to get the current exception handler.

(Contributed by Yury Selivanov in :issue:`27040`.)

* New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>`

method to read data from the stream until a separator bytes

sequence appears.

(Contributed by Mark Korenberg.)

* The performance of :meth:`StreamReader.readexactly() <asyncio.StreamReader.readexactly>`

has been improved.

(Contributed by Mark Korenberg in :issue:`28370`.)

* The :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`

method is optimized to avoid calling the system ``getaddrinfo``

function if the address is already resolved.

(Contributed by A. Jesse Jiryu Davis.)

* The :meth:`loop.stop() <asyncio.loop.stop>`

method has been changed to stop the loop immediately after

the current iteration. Any new callbacks scheduled as a result

of the last iteration will be discarded.

(Contributed by Guido van Rossum in :issue:`25593`.)

* :meth:`Future.set_exception <asyncio.Future.set_exception>`

will now raise :exc:`TypeError` when passed an instance of

the :exc:`StopIteration` exception.

(Contributed by Chris Angelico in :issue:`26221`.)

* New :meth:`loop.connect_accepted_socket() <asyncio.loop.connect_accepted_socket>`

method to be used by servers that accept connections outside of asyncio,

but that use asyncio to handle them.

(Contributed by Jim Fulton in :issue:`27392`.)

* ``TCP_NODELAY`` flag is now set for all TCP transports by default.

(Contributed by Yury Selivanov in :issue:`27456`.)

* New :meth:`loop.shutdown_asyncgens() <asyncio.loop.shutdown_asyncgens>`

to properly close pending asynchronous generators before closing the

loop.

(Contributed by Yury Selivanov in :issue:`28003`.)

* :class:`Future <asyncio.Future>` and :class:`Task <asyncio.Task>`

classes now have an optimized C implementation which makes asyncio

code up to 30% faster.

(Contributed by Yury Selivanov and INADA Naoki in :issue:`26081`

and :issue:`28544`.)

binascii

--------

The :func:`~binascii.b2a_base64` function now accepts an optional *newline*

keyword argument to control whether the newline character is appended to the

return value.

(Contributed by Victor Stinner in :issue:`25357`.)

cmath

-----

The new :const:`cmath.tau` (*τ*) constant has been added.

(Contributed by Lisa Roach in :issue:`12345`, see :pep:`628` for details.)

New constants: :const:`cmath.inf` and :const:`cmath.nan` to

match :const:`math.inf` and :const:`math.nan`, and also :const:`cmath.infj`

and :const:`cmath.nanj` to match the format used by complex repr.

(Contributed by Mark Dickinson in :issue:`23229`.)

collections

-----------

The new :class:`~collections.abc.Collection` abstract base class has been

added to represent sized iterable container classes.

(Contributed by Ivan Levkivskyi, docs by Neil Girdhar in :issue:`27598`.)

The new :class:`~collections.abc.Reversible` abstract base class represents

iterable classes that also provide the :meth:`~object.__reversed__` method.

(Contributed by Ivan Levkivskyi in :issue:`25987`.)

The new :class:`~collections.abc.AsyncGenerator` abstract base class represents

asynchronous generators.

(Contributed by Yury Selivanov in :issue:`28720`.)

The :func:`~collections.namedtuple` function now accepts an optional

keyword argument *module*, which, when specified, is used for

the :attr:`~type.__module__` attribute of the returned named tuple class.

(Contributed by Raymond Hettinger in :issue:`17941`.)

The *verbose* and *rename* arguments for

:func:`~collections.namedtuple` are now keyword-only.

(Contributed by Raymond Hettinger in :issue:`25628`.)

Recursive :class:`collections.deque` instances can now be pickled.

(Contributed by Serhiy Storchaka in :issue:`26482`.)

concurrent.futures

------------------

The :class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>`

class constructor now accepts an optional *thread_name_prefix* argument

to make it possible to customize the names of the threads created by the

pool.

(Contributed by Gregory P. Smith in :issue:`27664`.)

contextlib

----------

The :class:`contextlib.AbstractContextManager` class has been added to

provide an abstract base class for context managers. It provides a

sensible default implementation for ``__enter__()`` which returns

``self`` and leaves ``__exit__()`` an abstract method. A matching

class has been added to the :mod:`typing` module as

:class:`typing.ContextManager`.

(Contributed by Brett Cannon in :issue:`25609`.)

datetime

--------

The :class:`~datetime.datetime` and :class:`~datetime.time` classes have

the new :attr:`~datetime.time.fold` attribute used to disambiguate local time

when necessary. Many functions in the :mod:`datetime` have been

updated to support local time disambiguation.

See :ref:`Local Time Disambiguation <whatsnew36-pep495>` section for more

information.

(Contributed by Alexander Belopolsky in :issue:`24773`.)

The :meth:`datetime.strftime() <datetime.datetime.strftime>` and

:meth:`date.strftime() <datetime.date.strftime>` methods now support

ISO 8601 date directives ``%G``, ``%u`` and ``%V``.

(Contributed by Ashley Anderson in :issue:`12006`.)

The :func:`datetime.isoformat() <datetime.datetime.isoformat>` function

now accepts an optional *timespec* argument that specifies the number

of additional components of the time value to include.

(Contributed by Alessandro Cucci and Alexander Belopolsky in :issue:`19475`.)

The :meth:`datetime.combine() <datetime.datetime.combine>` now

accepts an optional *tzinfo* argument.

(Contributed by Alexander Belopolsky in :issue:`27661`.)

decimal

-------

New :meth:`Decimal.as_integer_ratio() <decimal.Decimal.as_integer_ratio>`

method that returns a pair ``(n, d)`` of integers that represent the given