Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use select.poll instead of select.select in SocketServer.BaseServer.serve_forever #75791

Open
bboneva mannequin opened this issue Sep 27, 2017 · 4 comments
Open

Use select.poll instead of select.select in SocketServer.BaseServer.serve_forever #75791

bboneva mannequin opened this issue Sep 27, 2017 · 4 comments
Labels
3.13 new features, bugs and security fixes performance Performance or resource usage stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@bboneva
Copy link
Mannequin

bboneva mannequin commented Sep 27, 2017

BPO 31610
Nosy @vstinner, @giampaolo

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-09-27.11:24:44.981>
labels = ['library', 'performance']
title = 'Use select.poll instead of select.select in SocketServer.BaseServer.serve_forever'
updated_at = <Date 2017-09-28.15:05:48.447>
user = 'https://bugs.python.org/bboneva'

bugs.python.org fields:

activity = <Date 2017-09-28.15:05:48.447>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-09-27.11:24:44.981>
creator = 'bboneva'
dependencies = []
files = []
hgrepos = []
issue_num = 31610
keywords = []
message_count = 4.0
messages = ['303127', '303247', '303251', '303254']
nosy_count = 3.0
nosy_names = ['vstinner', 'giampaolo.rodola', 'bboneva']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue31610'
versions = []

@bboneva
Copy link
Mannequin Author

bboneva mannequin commented Sep 27, 2017

The select function does not work for file descriptors with number >= 1024. It has FD_SETSIZE limit (see https://stackoverflow.com/questions/7976388/increasing-limit-of-fd-setsize-and-select#7977082).
So, serve_forever won't work if more than 1024 file descriptors are opened in the process (ValueError: filedescriptor out of range in select() is raised).
Moreover, the select function is considered inefficient, hard to scale and old-fashioned nowadays. Refer to https://stackoverflow.com/questions/970979/what-are-the-differences-between-poll-and-select#3951845.
Better alternative will be to use poll or epoll even.

@bboneva bboneva mannequin added stdlib Python modules in the Lib dir performance Performance or resource usage labels Sep 27, 2017
@vstinner
Copy link
Member

Better alternative will be to use poll or epoll even.

epoll requires mulitple system calls, whereas the selector seems to only be used only once in most cases, no?

Anyway, maybe selectors.DefaultSelector can be used here?

About poll(): be aware of macOS issues with poll() :-/

@giampaolo
Copy link
Contributor

I recommend to use selectors.PollSelector and fallback on selectors.SelectSelector where not available (Windows) in order to use 1 syscall only. That's what we did already in socket.py, subprocess.py and others.

@vstinner
Copy link
Member

I recommend to use selectors.PollSelector and fallback on selectors.SelectSelector where not available (Windows) in order to use 1 syscall only. That's what we did already in socket.py, subprocess.py and others.

It would be nice to have an API in selectors to get a "lightweight" selector, for selector and poll.

This API would be appropriate to check on macOS if poll() works or not: bpo-28087.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
@iritkatriel iritkatriel added type-feature A feature request or enhancement 3.12 bugs and security fixes labels Sep 9, 2022
@erlend-aasland erlend-aasland added 3.13 new features, bugs and security fixes and removed 3.12 bugs and security fixes labels Jan 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.13 new features, bugs and security fixes performance Performance or resource usage stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

4 participants