Today I did a (slightly) more formal release of a software project that I’ve been working on pretty consistently for the last two months. It’s an extension or elaboration on the buildcloth, and is the groundwork for some other projects I’ve been working on.
While there are some new fixes and improvements to the initial meta-build tool components of the project as I’d been working on 5 months ago, this one goes even further and includes a complete build automation tool.
The deal with this is that I’d been running a build system for months that had a bunch of very small tasks, and performance was awful for no really good reason. Well for one reason: process creation. Each task needed to create its own shell, run, and exit, which was awful. The solution to this problem was running each task (which was ultimately just a Python function) in a Python multi-processing pool. The new version of buildcloth is an attempt to build some common infrastructure around this practice.
It still needs some real world testing, and there are some missing features that I’d like to add, and always more documentation, but it’s good enough that I wanted to get it out there so that people could start using it and giving feedback.
I’ll post more later on the the experience and lessons learned here in a bit. While I work on that, see:
Onward and Upward!