Five Ways in Which The Python Module System is Broken

“Namespace” packages are not the default

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

If not every file contains this, you run into the problem that you cannot nest while being disjoint: if you work for a big company, say, FooCorp, you cannot have the simple convention: “All our Python libraries are FooCorp.Something. Every product is FooCorp.<Product name>.” Otherwise, if one group in FooCorp violates the convention, and does not put with the right magic, and they happen to be first on the PYTHONPATH, you’re SoL. Better learn how to hack your Python path.

Namespace packages behave weirdly in the face of sys.path changes

That, of course, is only a problem when you have to have your top-level namespace package in the thing that is recalculating the path. Argh.

Eggs, .pth and all that

Python has zipimports by default since 2.3. This means that the right way to package Python packages is “use zip”. The right way to “install” Python packages is “drop the zip file somewhere on your path”. No more. No less. If you need dependencies, some automated tool can help, sure. This fails if you have a PYD, of course, which is why PYD files should always be standalone and ideally, used only as a performance boost. For 95% of people, this should not be a concern anyway.

Default Export

Every symbol is exported. The overhead on every single global variable that is not meant for export beginning with an “_” is too much. If the default was “__all__ is treated as [] by default, oh and also the dot operator on a module checks in __all__ first”, people would have used __all__. When needing to fiddle directly with module content, people could still use the module’s __dict__ attribute (but then there was clear distinction between “using a module as intended” and “working around some issue in module”).

No standard naming convention

While at least the popular big projects usually just grab one node at the top and put their stuff under that, all the other poor schlebs tend to post their stuff all over their place. Especially for things which are often reimplemented, this is a mess — just look at all the modules wrapping postgres stuff. If we had a convention that the top-level is always “the organization originating this stuff”, then we could easily distinguish postgresql.wrapper from zope.pgwrapper from moshez.pgwrapper, without needing to remember a weird mapping.


One Response to Five Ways in Which The Python Module System is Broken

  1. At least your first complaint, namespaces being explicit, is changing in 3.3 with Implicit Namespace Packages.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: