Skip to content

bpo-35347: Fix test_socket.NonBlockingTCPTests #10791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 30, 2018
Merged

bpo-35347: Fix test_socket.NonBlockingTCPTests #10791

merged 1 commit into from
Nov 30, 2018

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Nov 29, 2018

testAccept() and testRecv() of test_socket.NonBlockingTCPTests have a
race condition: time.sleep() is used as a weak synchronization
primitive and the tests fail randomly on slow buildbots. Fix these
tests using a reliable threading.Event.

Changes:

  • Replace send() with sendall()
  • Expect specific BlockingIOError rather than generic OSError
  • Add a timeout to testAccept() and testRecv() select
  • Use addCleanup() and context manager to close sockets
  • Use assertLess() and assertRaises()

https://bugs.python.org/issue35347

@vstinner
Copy link
Member Author

cc @pablogsal FYI ;-)

@vstinner vstinner changed the title bpo-35347: Fix NonBlockingTCPTests.testRecv() bpo-35347: Fix test_socket.NonBlockingTCPTests Nov 29, 2018
@vstinner vstinner requested review from 1st1 and pablogsal November 29, 2018 10:40
testAccept() and testRecv() of test_socket.NonBlockingTCPTests have a
race condition: time.sleep() is used as a weak synchronization
primitive and the tests fail randomly on slow buildbots. Fix these
tests using a reliable threading.Event.

Changes:

* Replace send() with sendall()
* Expect specific BlockingIOError rather than generic OSError
* Add a timeout to testAccept() and testRecv() select
* Use addCleanup() and context manager to close sockets
* Use assertLess() and assertRaises()
@vstinner
Copy link
Member Author

@1st1, @pablogsal: I wrote a larger change than I expected to enhance and fix test_socket.NonBlockingTCPTests. Would you mind to review this change?


First, I wrote a much larger change to "cleanup" the whole test case. Since I would like to apply this fix to 2.7, 3.6 and 3.7, I rewrote the change to have a "minimum" fix for testAccept() and testRecv(). Once the fix will be merged, I will write a second "cleanup" PR specific to master.

@vstinner
Copy link
Member Author

@vstinner vstinner removed the needs backport to 2.7 label just now

This change should be backported to 2.7, but I will backport it manually since BlockingIOError must be written differently in Python 2.7.

@vstinner vstinner merged commit ebd5d6d into python:master Nov 30, 2018
@miss-islington
Copy link
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.6, 3.7.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@vstinner vstinner deleted the test_nonblocking_recv branch November 30, 2018 11:29
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 30, 2018
testAccept() and testRecv() of test_socket.NonBlockingTCPTests have a
race condition: time.sleep() is used as a weak synchronization
primitive and the tests fail randomly on slow buildbots.

Use a reliable threading.Event to fix these tests.

Other changes:

* Replace send() with sendall()
* Expect specific BlockingIOError rather than generic OSError
* Add a timeout to select() in testAccept() and testRecv()
* Use addCleanup() to close sockets
* Use assertRaises()
(cherry picked from commit ebd5d6d)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
@bedevere-bot
Copy link

GH-10815 is a backport of this pull request to the 3.7 branch.

@bedevere-bot
Copy link

GH-10816 is a backport of this pull request to the 3.6 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 30, 2018
testAccept() and testRecv() of test_socket.NonBlockingTCPTests have a
race condition: time.sleep() is used as a weak synchronization
primitive and the tests fail randomly on slow buildbots.

Use a reliable threading.Event to fix these tests.

Other changes:

* Replace send() with sendall()
* Expect specific BlockingIOError rather than generic OSError
* Add a timeout to select() in testAccept() and testRecv()
* Use addCleanup() to close sockets
* Use assertRaises()
(cherry picked from commit ebd5d6d)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
@vstinner
Copy link
Member Author

I'm sorry, I am impatient :-( I reviewed again my code and then I merged it my fix to make CIs more reliable.

miss-islington added a commit that referenced this pull request Nov 30, 2018
testAccept() and testRecv() of test_socket.NonBlockingTCPTests have a
race condition: time.sleep() is used as a weak synchronization
primitive and the tests fail randomly on slow buildbots.

Use a reliable threading.Event to fix these tests.

Other changes:

* Replace send() with sendall()
* Expect specific BlockingIOError rather than generic OSError
* Add a timeout to select() in testAccept() and testRecv()
* Use addCleanup() to close sockets
* Use assertRaises()
(cherry picked from commit ebd5d6d)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
miss-islington added a commit that referenced this pull request Nov 30, 2018
testAccept() and testRecv() of test_socket.NonBlockingTCPTests have a
race condition: time.sleep() is used as a weak synchronization
primitive and the tests fail randomly on slow buildbots.

Use a reliable threading.Event to fix these tests.

Other changes:

* Replace send() with sendall()
* Expect specific BlockingIOError rather than generic OSError
* Add a timeout to select() in testAccept() and testRecv()
* Use addCleanup() to close sockets
* Use assertRaises()
(cherry picked from commit ebd5d6d)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
vstinner added a commit that referenced this pull request Nov 30, 2018
testAccept() and testRecv() of test_socket.NonBlockingTCPTests have a
race condition: time.sleep() is used as a weak synchronization
primitive and the tests fail randomly on slow buildbots.

Use a reliable threading.Event to fix these tests.

Other changes:

* Replace send() with sendall()
* Add a timeout to select() in testAccept() and testRecv()
* Use addCleanup() to close sockets
* Use assertRaises()

(cherry picked from commit ebd5d6d)
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora 2.7 has failed when building commit dab59fa.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/57/builds/266) and take a look at the build logs.
  4. Check if the failure is related to this commit (dab59fa) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/57/builds/266

Click to see traceback logs
fatal: unable to access 'https://github.com/python/cpython.git/': transfer closed with outstanding read data remaining

make: *** No rule to make target `distclean'.  Stop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip news tests Tests in the Lib/test dir
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants