Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cockroachdb unimplemented feature multiple active portals not supported #580

Open
nagylzs opened this issue May 31, 2020 · 5 comments
Open

Cockroachdb unimplemented feature multiple active portals not supported #580

nagylzs opened this issue May 31, 2020 · 5 comments

Comments

@nagylzs
Copy link

@nagylzs nagylzs commented May 31, 2020

  • asyncpg version: 0.20.1
  • Cockroachdb version: v20.1.0 (connected to a 5-node on premise cluster)
  • Python version: 3.8.0
  • Platform: Linux
  • Do you use pgbouncer?: No
  • Did you install asyncpg with pip?: Yes

Example code:

    async with pool.acquire() as conn:
        async with conn.transaction():
            if not await conn.table_exists("sys_app_config"):
                await conn.execute("""
                    CREATE TABLE sys_app_config(
                        key  text NOT NULL PRIMARY KEY,
                        value jsonb
                    );
                """)

The table_exists method looks like this:

class MyConnection(Connection):
    async def table_exists(self, table_name: str) -> bool:
        """Tell if a table exists."""
        return await self.fetchval(
            "select table_name from information_schema.tables where table_name=$1", table_name) is not None

This code is actually called from tornado.ioloop.IOLoop.current().run_sync so it is guaranteed that nothing else is running queries concurrently.

Here is the traceback that I'm getting:


 File "/home/user/my_project/db/initialize.py", line 19, in init_database
    if not await conn.table_exists("sys_app_config"):
  File "/home/user/.local/share/virtualenvs/backend-YgzYW8Ky/lib/python3.8/site-packages/asyncpg/transaction.py", line 91, in __aexit__
    await self.__commit()
  File "/home/user/.local/share/virtualenvs/backend-YgzYW8Ky/lib/python3.8/site-packages/asyncpg/transaction.py", line 179, in __commit
    await self._connection.execute(query)
  File "/home/user/.local/share/virtualenvs/backend-YgzYW8Ky/lib/python3.8/site-packages/asyncpg/connection.py", line 272, in execute
    return await self._protocol.query(query, timeout)
  File "asyncpg/protocol/protocol.pyx", line 316, in query
asyncpg.exceptions.FeatureNotSupportedError: unimplemented: multiple active portals not supported
HINT:  You have attempted to use a feature that is not yet implemented.
See: https://github.com/cockroachdb/cockroach/issues/40195

The referenced issue tells that it is not possible to keep multiple queries open and fetch from them interleaved.

However, I'm not doing anything that should result in opening multiple queries and fetching from them that way. As far as I see, my code should not open a new query before closing the previous one.

I suspect that asyncpg itself forgot to close something in the background.

@nagylzs
Copy link
Author

@nagylzs nagylzs commented May 31, 2020

I have uploaded a minimal working example here:

https://gist.github.com/nagylzs/acc716982f8a8213cbe5d6d6495e5387

@nagylzs
Copy link
Author

@nagylzs nagylzs commented May 31, 2020

Some more info: if I don't start a new transaction ( async with conn.transaction() ) then it works.

@vaperce
Copy link

@vaperce vaperce commented Jun 5, 2020

Coud it be related to this issue from Cockroach cockroachdb/cockroach#42912?

@nagylzs
Copy link
Author

@nagylzs nagylzs commented Jun 6, 2020

It could be. Then it would mean that fetchval or execute did not close the resultset before returning the result? (I'm guessing - these are the only two queries opened.)

@vaperce
Copy link

@vaperce vaperce commented Jun 8, 2020

From my analysis, Cockroach is unable to implicitly close a result set (named also portal) in a transaction which it should do if it was respecting completely PSQL spec.

As far as I understand, they are not willing to fix this anytime soon because it would require them to do a lot of refactoring and only a few clients are behaving like asyncpg.

In this example:

  • fetchval opens a portal because it has arguments
  • if the condition is False the next execute else the hidden "COMMIT;" when finishing transaction raises the error because Cockroach expects the portal to be closed before doing anything else

I'm working on a fix (for asyncpg)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.