This was removed in python#9616

…9683)

Once we were in a directory with __init__.py we only recursed into subdirectories if those subdirectories also contained an __init__.py. This is what #9616 should have been.

Co-authored-by: hauntsaninja <>

This is the successor to #9632. Things should basically be as discussed in that PR. Since #9616 is merged, this should now resolve #5759.

We leave the Bazel integration with `--package-root` almost entirely untouched, save for a) one change that's a bugfix / doesn't affect the core of what `--package-root` is doing, b) another drive by bugfix that's not related to this PR.
Change a) fixes the package root `__init__.py` hackery when passed absolute paths. Change b) fixes the validation logic for package roots above the current directory; it was broken if you passed `..` as a package root

Since we're leaving `--package-root` alone for now, I named the new flag `--explicit-package-base` to try and avoid confusion. Doing so also matches the language used by BuildSource a little better.

The new logic is summarised in the docstring of `SourceFinder.crawl_up`.

Some commentary:
- I change `find_sources_in_dir ` to call `crawl_up` directly to construct the BuildSource. This helps codify the fact that files discovered will use the same module names as if you passed them directly.
- Doing so keeps things DRY with the more complicated logic and means, for instance, that we now do more sensible things in some cases when we recursively explore directories that have invalid package names.
- Speaking of invalid package names, if we encounter a directory name with an invalid package name, we stop crawling. This is necessary because with namespace packages there's no guarantee that what we're crawling was meant to be a Python package. I add back in a check in the presence of `__init__.py` to preserve current unit tests where we raise InvalidSourceList.
- The changes to modulefinder are purely cosmetic and can be ignored (there's some similar logic between the two files and this just makes sure they mirror each other closely)
- One notable change is we now always use absolute paths to crawl. This makes the behaviour more predictable and addresses a common complaint: fixes #9677, fixes #8726 and others.
- I figured this could use more extensive testing than a couple slow cmdline tests. Hopefully this test setup also helps clarify the behaviour :-)

Co-authored-by: hauntsaninja <>

This should cover the current state on master, as implemented
across python#9742, python#9683, python#9632, python#9616, python#9614, etc.

This will need to be changed if we can make `--namespace-packages` the
default (python#9636).

This should cover the current state on master, as previously
discussed / implemented across python#9742, python#9683, python#9632, python#9616, python#9614, etc.

This will need to be changed if we can make `--namespace-packages` the
default (python#9636).

I haven't documented some of the finer points of the changes, since it
felt like an inappropriate level of detail (e.g. using absolute paths
when crawling, how directories with invalid package names affect
crawling, etc)

This should cover the current state on master, as previously
discussed / implemented across python#9742, python#9683, python#9632, python#9616, python#9614, etc.

This will need to be changed if we can make `--namespace-packages` the
default (python#9636).

I haven't documented some of the finer points of the changes, since it
felt like an inappropriate level of detail (e.g. using absolute paths
when crawling, how directories with invalid package names affect
crawling, etc)

This should cover the current state on master, as previously
discussed / implemented across #9742, #9683, #9632, #9616, #9614, etc.

This will need to be changed if we can make `--namespace-packages` the
default (#9636).

I haven't documented some of the finer points of the changes, since it
felt like an inappropriate level of detail (e.g. using absolute paths
when crawling, how directories with invalid package names affect
crawling, etc)

Co-authored-by: hauntsaninja <>

This should cover the current state on master, as previously
discussed / implemented across #9742, #9683, #9632, #9616, #9614, etc.

This will need to be changed if we can make `--namespace-packages` the
default (#9636).

I haven't documented some of the finer points of the changes, since it
felt like an inappropriate level of detail (e.g. using absolute paths
when crawling, how directories with invalid package names affect
crawling, etc)

Co-authored-by: hauntsaninja <>