(Somewhat confused) Thoughts on Languages, Eco-systems and Development

There are a few interesting new languages that would be fun to play with: Rust, Julia, LFE, Go and D all have interesting takes on some domain. But ultimately, a language is only as interesting as the things it can do. It used to be that “things it can do” referred to the standard library. I am old enough to remember when “batteries included” was one of the most interesting benefits Python had — the language had dared doing such avant-garde things in ’99 as having a *built-in* url fetching library. That behaved, pretty much, the same on all platform. Out of the box. (It’s hard to convey how amazing this was in ’99.)

This is no longer the case. Now what distinguishes Python as “can do lots of things” is its built-in package management. Python, pip and virtualenv together give the ability to work on multiple Python projects, that need different versions of libraries, without any interference. They are, to a first approximation, 100% reliable. With pip 7.0 supporting caching wheels, virtualenvs are even more disposable (I am starting to think pip uninstall is completely unnecessary). In fact, except for virtualenv itself, it is rare nowadays to install any Python module globally. There are some rusty corners, of course:

  • Uploading packages is…non-trivial. Any time the default tool is so bad that the official docs say “use this 3rd party library instead”, something is wrong.
  • The distinction between “PyPI name” and “Python package name”, while theoretically reasonable, is just annoying and hard to explain
  • (Related) There is no official PyPI-name-conflict-resolution procedure.

I’ll note npm, for example, clearly does better on these three (while doing worse on some things that Python does better). Of the languages mentioned above, it is nice to see that most have out-of-the-box built-in tools for ecosystem creation. Julia’s hilariously piggybacks on GitHub’s. Go’s…well…uses URLs as the equivalent PyPI-level-names. Rust has a pretty decent system in cargo and crates.io (the .toml file is kind of weird, but I’ve seen worse). While there is justifiable excitement about Rust’s approach to memory and thread-safety, it might be that it will win in the end based on having a better internal package management system than the low-level alternatives.

Note: OS package mgmt (yum, apt-get, brew and whatever Windows uses nowadays) is not a good fit for what developers need from a language-level package manager. Locality, quick package refreshes — these matter more to developers than to OS end-users.


2 Responses to (Somewhat confused) Thoughts on Languages, Eco-systems and Development

  1. glyph says:

    Why aren’t locality and quick refreshes interesting to users? Users want their applications to continue to work, which means they want one application to be able to get its updates without disturbing the rest of their system, don’t they? What Windows does is “you distribute all dependencies along with your application except those that we, Microsoft, promise will have a compatible API until the last star winks out in the heavens”.

    • moshez says:

      It’s a trade-off. I’m sure end-users would also like a pony and a lollipop, but I think for end-users, integration into the OS (e.g., automatically appearing in menus) and stability (not having weird integration bugs) matters more than how fast is it from the time a developer did “setup.py upload” to the time I update the version in my requirements.txt. OTOH, I care nothing for systems-integration for my Python dependencies, and if there’s a weird integration bug, I’m 95% likely to prefer fixing it and temporarily pinning it.

      Note that this is true even for *me* wearing different hates: as a Python application/library developer, I care more about locality and update-speed, and as a guy running “freeciv”, I care about good OS integration and stability. I would *like* all of these, of course — but that’s the point with trade-offs.

      Also see for reference Nick Coghlan’s e-mail here: https://lists.fedoraproject.org/pipermail/env-and-stacks/2015-June/000796.html (a somewhat different perspective than I outline, but fascinating even as he’s wrong about every detail…)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: