The fact that Path.iterdir() only throws exceptions once you start iterating over it makes it very difficult to write correct code.

Let's look at an example: We'll iterate over all children of a directory and print their file size.

If we try to do it like this, the try...except has no effect whatsoever:

try:
    children = path_that_doesnt_exist.iterdir()
except FileNotFoundError:
    print("directory doesn't exist")
for path in children:
    print(path.stat().st_size)

If we explicitly check whether the path exists (and is a directory), we end up with a race condition:

if path_that_doesnt_exist.is_dir():
    print("directory doesn't exist")
else:
    for path in children:
        print(path.stat().st_size)

We can wrap the whole loop in a try...except, but then we might end up catching exceptions we didn't intend to catch. (For example, the exception that's thrown when we try to get the size of a broken symlink.)

try:
    for path in path_that_doesnt_exist.iterdir():
        print(path.stat().st_size)  # this can also throw FileNotFoundError
except FileNotFoundError:
    print("directory doesn't exist")

We can manually call next on the iterator inside of a try..except block, but that's awfully verbose and requires knowledge about iterators and the next function:

children = iter(path_that_doesnt_exist.iterdir())
while True:
    try:
        path = next(children)
    except FileNotFoundError:
        print("directory doesn't exist")
        break
    print(path.stat().st_size)

Or we can turn the iterator into a list inside of a try...except, which seems to be the best option, but completely defeats the point of having an iterator:

try:
    children = list(path_that_doesnt_exist.iterdir())
except FileNotFoundError:
    print("directory doesn't exist")
else:
    for path in children:
        print(path.stat().st_size)

As you can see, writing correct (and good) code with iterdir is more difficult than it has any right to be. Please change this behavior so that exceptions are thrown immediately when iterdir is called.

As an afterthought, I'd like to suggest an alternative solution: If changing the iterdir behavior is not possible or not desirable for some reason, please add a Path.listdir method that returns a list instead of an iterator. Lazy file system operations can often introduce unnecessary race conditions in the code, so I believe it would be very useful if every lazy method had an eager equivalent.

Made changes, pathlib.Path('.').iterdir() now throws a FileNotFoundError if the path is not valid.

Work-around: call list(path.iterdir()):

try:
    children = list(path_that_doesnt_exist.iterdir())
except FileNotFoundError:
    print("directory doesn't exist")
for path in children:
    print(path.stat().st_size)

I don't think we can change the iterdir() behaviour at this stage, nor does it justify the introduction of a similar method like listdir() to my mind.

Yeah, the fact that it returns a generator/lazy iterator seems pretty fundamental to iterdir's behavior, and matches the other current and past iter* functions/methods in Pythons, as well as pathlib.glob and similar methods (as well as builtins like range and others) which would make it suddenly behaving eagerly quite surprising to many/most users, as well as sacrificing performance/efficiency.

list(<lazy iterator>) to make something iterable is a common idiom in Python to handle this situation (and others), and adding a separate method to do the same thing eagerly seems thoroughly redundant.

I'm going to close this issue per CAM's rationale above. Use list(path.iterdir()) if you need to handle exceptions before you iterate. Thanks all.

I don't believe list(iterdir()) is a suitable workaround, as it requires the entire directory to be materialized in memory, increasing storage with little benefit. I'd argue that itertools (or similar) should provide a mechanism to filter or stop on exceptions, so consuming the iterable can continue to be lazy.

more_itertools.iter_except almost handles this case, but (a) requires the callable to be passed and (b) can't handle StopIteration as a stop condition.

Upon reflection I agree. I've put a PR up. Do you have a preference on whether it should be backported?

A comment from @AA-Turner:

I agree fixing iterdir at the source would be cleaner

Fixed in 3.13 / #107320 / bdc3c88

Thanks for the fix!

Upon reflection I agree. I've put a PR up. Do you have a preference on whether it should be backported?

This feels like a change in behavior / improvement, so probably not suitable for a backport.