Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-37596: Update test_deterministic_sets to correctly handle different string hash algorithms #28147

Merged
merged 1 commit into from Sep 4, 2021
Merged

bpo-37596: Update test_deterministic_sets to correctly handle different string hash algorithms #28147

merged 1 commit into from Sep 4, 2021

Conversation

@brandtbucher
Copy link
Member

@brandtbucher brandtbucher commented Sep 3, 2021 •
edited by bedevere-bot 

$ for HASH_ALGORITHM in siphash24 fnv
> do
>     ./configure --with-hash-algorithm=$HASH_ALGORITHM &> /dev/null
>     make -j &> /dev/null
>     ./python -c 'import sys; print(f"{sys.hash_info.algorithm = }")'
>     ./python -m test test_marshal
> done
sys.hash_info.algorithm = 'siphash24'
0:00:00 load avg: 3.28 Run tests sequentially
0:00:00 load avg: 3.28 [1/1] test_marshal

== Tests result: SUCCESS ==

1 test OK.

Total duration: 316 ms
Tests result: SUCCESS

sys.hash_info.algorithm = 'fnv'
0:00:00 load avg: 15.70 Run tests sequentially
0:00:00 load avg: 15.70 [1/1] test_marshal

== Tests result: SUCCESS ==

1 test OK.

Total duration: 311 ms
Tests result: SUCCESS

https://bugs.python.org/issue37596
@brandtbucher brandtbucher requested a review from tiran as a code owner Sep 3, 2021
@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Sep 3, 2021

🤖 New build scheduled with the buildbot fleet by @brandtbucher for commit e72a25b 🤖

If you want to schedule another build, you need to add the "🔨 test-with-buildbots" label again.

gvanrossum
gvanrossum approved these changes Sep 3, 2021
View changes
Copy link
Member

@gvanrossum gvanrossum left a comment

So did you just try different strings until you found one that hashes the same under fnv and siphash24?? That's amazing. (I didn't even know we had all this machinery -- all I know is that we have "hash randomization" and that you can force the seed by setting an env var. :-)

@brandtbucher
Copy link
Member Author

@brandtbucher brandtbucher commented Sep 3, 2021

So did you just try different strings until you found one that hashes the same under fnv and siphash24?? That's amazing.

No, I just needed to find one that hashes in such a way that sets containing them have different orders under PYTHONHASHSEED=0 and PYTHONHASHSEED=1 for each algorithm! It's a much easier task. 🙃

(I didn't even know we had all this machinery -- all I know is that we have "hash randomization" and that you can force the seed by setting an env var. :-)

Same, I learned a lot about this stuff today!

@brandtbucher
Copy link
Member Author

@brandtbucher brandtbucher commented Sep 4, 2021

That buildbot's been failing for a while with that same error. I'll open an issue for it.

@brandtbucher brandtbucher merged commit 3beef86 into python:main Sep 4, 2021
73 of 74 checks passed
@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Sep 4, 2021

@brandtbucher: Please replace # with GH- in the commit message next time. Thanks!

@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Sep 4, 2021

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable LTO 3.x has failed when building commit 3beef86.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/271/builds/562) and take a look at the build logs.
  4. Check if the failure is related to this commit (3beef86) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/271/builds/562

Summary of the results of the build (if available):

== Tests result: ENV CHANGED ==

413 tests OK.

10 slowest tests:

  • test_peg_generator: 3 min 34 sec
  • test_concurrent_futures: 2 min 31 sec
  • test_gdb: 1 min 43 sec
  • test_nntplib: 1 min 20 sec
  • test_multiprocessing_spawn: 1 min 7 sec
  • test_multiprocessing_forkserver: 58.9 sec
  • test_asyncio: 56.2 sec
  • test_multiprocessing_fork: 53.0 sec
  • test_signal: 47.1 sec
  • test_io: 37.1 sec

1 test altered the execution environment:
test_ftplib

13 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_ossaudiodev
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

Total duration: 4 min 43 sec

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/threading.py", line 1009, in _bootstrap_inner
    self.run()
    ^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/test/test_ftplib.py", line 297, in run
    asyncore.loop(timeout=0.1, count=1)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/asyncore.py", line 214, in loop
    poll_fun(timeout, map)
    ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/asyncore.py", line 157, in poll
    read(obj)
    ^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/asyncore.py", line 94, in read
    obj.handle_error()
    ^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/test/test_ftplib.py", line 139, in handle_error
    raise Exception
    ^^^^^^^^^^^^^^^
Exception


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/asyncore.py", line 90, in read
    obj.handle_read_event()
    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/asyncore.py", line 427, in handle_read_event
    self.handle_read()
    ^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/asynchat.py", line 179, in handle_read
    self.found_terminator()
    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/test/test_ftplib.py", line 134, in found_terminator
    method(arg)
    ^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/test/test_ftplib.py", line 159, in cmd_pasv
    conn, addr = sock.accept()
                 ^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/socket.py", line 293, in accept
    fd, addr = self._accept()
               ^^^^^^^^^^^^^^
TimeoutError: timed out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gvanrossum gvanrossum

@tiran tiran

Assignees
No one assigned
Labels
CLA signed skip news type-tests
Projects
None yet
Milestone
No milestone
4 participants
@brandtbucher @bedevere-bot @gvanrossum @the-knights-who-say-ni