MagicStack / asyncpg

Due to executable file locking `pip install -U pip` doesn't work on
Windows, one has to use `python -m pip install -U pip` instead.

We didn't really retry the connection without SSL if the first SSL
connection fails under sslmode=prefer, that led to an issue when the
server has SSL support but explicitly denies SSL connection through
pg_hba.conf. This commit adds a retry in a new connection, which
makes it easy to implement the sslmode=allow retry.

Fixes #716

Python 3.5 on GitHub Action ran into some certificate issue that newer
version of pip doesn't have, so upgrading to the latest by default.

The README.md says that PostgreSQL versions supported are 9.5 to 13,
copied to the doc.

Co-authored-by: Fernando Ike <fernando.ike@maburix.com>

This fixes an issue in 0.22.0 where we passed WriteBuffer to writelines
by mistake, which leads to an error under SSL and uvloop - the
implementation that calls len() on each line of writelines().

Fixes: #700

Only add isolation to repr if it's not None

A new asyncpg release is here.

Notable additions include Python 3.9 support, support for recently added
PostgreSQL types like `jsonpath`, and last but not least, vastly
improved `executemany()` performance.  Importantly, `executemany()` is
also now _atomic_, which means that either all iterations succeed, or
none at all, whereas previously partial results would have remained in
place, unless `executemany()` was called in a transaction.

There is also the usual assortment of improvements and bugfixes, see the
details below.

This is the last release of asyncpg that supports Python 3.5, which has
reached EOL last September.

Improvements
------------

* Vastly speedup executemany by batching protocol messages (#295)
  (by @fantix in 690048d for #295)

* Allow using custom `Record` class
  (by @elprans in db4f1a6 for #577)

* Add Python 3.9 support (#610)
  (by @elprans in c05d726 for #610)

* Prefer SSL connections by default (#660)
  (by @elprans in 16183aa for #660)

* Add codecs for a bunch of new builtin types (#665)
  (by @elprans in b53f038 for #665)

* Expose Pool as `asyncpg.Pool` (#669)
  (by @rugleb in 0e0eb8d for #669)

* Avoid unnecessary overhead during connection reset (#648)
  (by @kitogo in ff5da5f for #648)

Fixes
-----

* Add a workaround for bpo-37658
  (by @elprans in 2bac166 for #21894)

* Fix wrong default transaction isolation level (#622)
  (by @fantix in 4a627d5 for #622)

* Fix `set_type_codec()` to accept standard SQL type names (#619)
  (by @elprans in 68b40cb for #619)

* Ignore custom data codec for internal introspection (#618)
  (by @fantix in e064f59 for #618)

* Fix null/NULL quoting in array text encoder (#627)
  (by @fantix in 92aa806 for #627)

* Fix link in connect docstring (#653)
  (by @samuelcolvin in 8b313bd for #653)

* Make asyncpg work with pyinstaller (#651)
  (by @Atem18 in 5ddabb1 for #651)

* Fix possible `AttributeError` exception in `ConnectionSettings` (#632)
  (by @petriborg in 0d23182 for #632)

* Prohibit custom codecs on domains
  (by @elprans in 50f964f for #457)

* Raise proper error on anonymous composite input (tuple arguments) (#664)
  (by @elprans in 7252dbe for #664)

* Fix incorrect application of custom codecs in some cases (#662)
  (by @elprans in 50f65fb for #662)

The porting of arm64 builds to Github Actions uncovered an
architecture-specific (or, perhaps, virtualization-specific) bug in
asyncpg, so, rather than blocking the release, drop aarch64 wheels for
now.  Also, `manylinux2014_aarch64` is not considered stable yet
(see pypa/manylinux#84) and so the compiled wheels might not even work
correctly on all aarch64 boxes.

There is a small typo in asyncpg/connection.py.

Should read `verification` rather than `verifiction`.

UNLISTEN is now available in Hot Standby mode in all supported
PostgreSQL versions, therefore there's no reason anymore to wrap
it in DO block. This should significantly speed up connection reset.

Use absolute URL to the performance chart so that it renders properly on
PyPI and elsewhere.

Remove the `copy_records_to_table()` note as `executemany()` is almost
as efficient now and rephrase the note about the new behavior with
respect to atomicity.

The current implementation has a bunch of `CheckExact` calls in front of
most `Record` methods, effectively making them unusable in subclasses.
Relax the check to include the subclasses.

While at it, add a check that `__init__` and `__new__` are not redefined
on the provided Record subclass.  Doing so is pointless, because
`Record` instance initialization effectively bypasses both, so raise an
`InterfaceError` to avoid needless confusion.

Fixes: #676

Asyncpg currently erroneously prefers binary I/O for underlying type of
arrays effectively ignoring a possible custom text codec that might have
been configured on a type.

Fix this by removing the explicit preference for binary I/O, so that the
codec selection preference is now in the following order:

- custom binary codec
- custom text codec
- builtin binary codec
- builtin text codec

Fixes: #590
Reported-by: @neumond

Currently asyncpg would crash with an arcane "could not resolve query
result and/or argument types in 6 attempts", which isn't helpful.  Do
the right thing by raising an `UnsupportedClientFeatureError`
explicitly instead.

Fixes #476.

Postgres always includes the base type OID in the RowDescription message
even if the query is technically returning domain values.  This makes
custom codecs on domains ineffective, and so prohibit them to avoid
confusion and bug reports.

See postgres/postgres@d9b679c and
https://postgr.es/m/27307.1047485980%40sss.pgh.pa.us for context.

Fixes: #457.

The last round of support for esoteric builtin types was quite a while
ago, so catch up.  Out of non-internal types this adds support for the
new `jsonpath` type.

Fixes: #635.

Switch the default SSL mode from 'disabled' to 'prefer'.  This matches
libpq's behavior and is a sensible thing to do.

Fixes: #654

This makes `Pool` accessible as `asyncpg.Pool` for ease of typing annotations and subclassing.

Fixes: #643

Closes: #598.

Accessing settings._settings raises an AttributeError exception because 
object.__getattr__ does not exist.

Fixes #649

Now `Bind` and `Execute` pairs are batched into 4 x 32KB buffers to take
advantage of `writelines()`. A single `Sync` is sent at last, so that
all args live in the same transaction.

pgbench results of inserting 1000 rows per query with executemany() on 
Python 3.6 of 2.2GHz 2015 MacBook Air (best out of 5 runs):

asyncpg 0.18.2:

    710 queries in 30.31 seconds
    Latency: min 341.88ms; max 636.29ms; mean 425.022ms; std: 39.782ms (9.36%)
    Latency distribution: 25% under 401.67ms; 50% under 414.26ms; 75% under 435.37ms; 90% under 478.39ms; 99% under 576.638ms; 99.99% under 636.299ms
    Queries/sec: 23.42
    Rows/sec: 23424.32

This patch:

    4125 queries in 30.02 seconds
    Latency: min 23.14ms; max 734.91ms; mean 72.723ms; std: 49.226ms (67.69%)
    Latency distribution: 25% under 59.958ms; 50% under 65.414ms; 75% under 71.538ms; 90% under 80.95ms; 99% under 175.375ms; 99.99% under 734.912ms
    Queries/sec: 137.39
    Rows/sec: 137389.64

This is a backwards incompatible change.  Here `executemany()` becomes
atomic, whereas previously any error in the middle of argument iteration
would retain the results of the preceding set of arguments unless an explicit
transaction block was used.

Closes: #289

When given a textual json codec that yields 'null', the array
encoder should generate b'["null"]' instead of b'["NULL"]'
for a JSON[] type.

Refs: #623

Fixes: #617

Currently, `Connection.set_type_codec()` only accepts type names as they
appear in `pg_catalog.pg_type` and would refuse to handle a standard SQL
spelling of a type like `character varying`.  This is an oversight, as
the internal type names aren't really supposed to be treated as public
Postgres API.  Additionally, for historical reasons, Postgres has a
single-byte `"char"` type, which is distinct from both `varchar` and
SQL `char`, which may lead to massive confusion if a user sets up a
custom codec on it expecting to handle the `char(n)` type instead.

Issue: #617

* Fix wrong default transaction isolation level

This fixes the issue when the default_transaction_isolation is
not "read committed", `transaction(isolation='read_committed')`
won't start a transaction in "read committed" isolation level.

`asyncio.wait_for()` currently has a bug where it raises a
`CancelledError` even when the wrapped awaitable has completed.
The upstream fix is in python/cpython#21894.  This adds a workaround
until the aforementioned PR is merged, backported and released.

Co-authored-by: Adam Liddell <git@aliddell.com>
Fixes: #467
Fixes: #547
Related: #468
Supersedes: #548

Python 3.9 moved a bunch of GC-related symbols around, including
`_PyObject_GC_IS_TRACKED` which is used in `recordobj.c`.

Fixes: #609

Add the new `record_class` parameter to the `create_pool()` and
`connect()` functions, as well as to the `cursor()`, `prepare()`,
`fetch()` and `fetchrow()` connection methods.

This not only allows adding custom functionality to the returned
objects, but also assists with typing (see #577 for discussion).

Fixes: #40.