For 2013 and some of 2012 and 2014, I mostly stopped knitting and spent the time teaching myself how to really make software. It wasn’t exactly that intentional in the moment, but time is finite and both knitting and coding require similar kinds of attention and thought. In retrospect this is what happened.

I’ve started knitting again, not as much or in the same ways that I used to, but there’s a bit more space in my brain for additional projects. I’ve thought often about writing about similarities between knitting and programming.

The two activities have much in common. Code and knitting are both highly structured activities, with a reasonably finite syntax of possible component operations. Most programming languages have a few dozen keywords, fundamental types, and built in operations that give rise to all programs (more or less,) and knitting similarly provides a small number of operations (knit, purl, increase, decrease, slip) that you can combine and alter to produce pretty much any kind of fabric.

Knitting patterns are procedural (like programs, generally) and have a standard syntax that shares a number of common elements with contemporary programming languages, including loops, conditionals, map (functional) operations, as well as some non-trivial concurrency.

While each knitting stitch depends on the stitches that happen before and after it, they are all reasonably atomic within the knitted object. At the same time, many knitting projects have lots other objects that happen at the same time, along different dimensions: shaping, stitch patterns, short rows, sleeves/gloves as well as parallel objects like socks. Knitters routinely track multiple things happening at the same time, with some success: a task that many programmers struggle with.

The connection between knitting and programming isn’t terribly surprising or novel: indeed there’s a long history between the development of textile production and technology more broadly, but hand knitting is sort of a technological oddity, so the connections don’t have the same history that you’d have in weaving (say.)

I’ve struggled to write this post for a while and have always had a hard time figuring out what the conclusion is or should be. I think I’ve felt like there should be some potential for synthesis, some way to take the things I’ve learned about software and use that to write programs to support knitting or design work, or vice versa. Increasingly, I think that this is the anti-conclusion.

Sure you can represent knitting patterns using formal languages (such as programming languages,) and it wouldn’t be that hard to write software that converted a pattern, or validated the math, but the math is really the easy part of knitting.

The hard part about knitting things is not figuring out how many stitches you need to knit an object that’s the right size, but rather figuring out what the right size is. It’s easy to know how many stitches you need, and hard to decide how big or small a thing should be. It’s also true that actually knitting things takes the largest amount of time; figuring out the math and making the design decisions are comparatively quick for most projects.

Computers wouldn’t be able to tell how long we should knit the cuffs of our socks or the sleeves of our sweaters, no matter how much code we write.

In any case, I’ll be over here, knitting away at my sweater.

There’s been a lot of change in my life in the last six months or a year, and I’ve mostly avoided reflecting about it in writing. I’ve tried writing about it and tried fictionalizing it: I’ve concluded that I’m just not comfortable with my own interpretation of either the last six months or my current state in a way that I could write from. Nevertheless, given the recent calendar roll over, there’s a lot of reflection in the air.

The longer I don’t write, the larger the burden of not writing becomes. Surprisingly, I’ve not been alarmed by the feeling that I am not trustworthy observer of my own life: I’m confident that I am rapidly reaching a place where the present and the future will be easy to write from, particularly on the smaller scale. There’s just this hole in the larger narrative of my life, that I feel obligated to feel.

But maybe it’s alright if I don’t. Or if I don’t just yet.

It also feels fine to start writing about little things that have changed, things that I’m enjoying in my life that are new and different. This will be the first post of this kind, but hopefully not the last.

Today I had four people at my house for brunch.

I made collard greens cooked with a whole onion, an entire head of garlic, and one beef sausage for flavor. I’d never made collards before (it was great). For next time, I definitely need a larger pan, and need more onion and garlic, perhaps adding mushrooms instead of the sausage, which was nice but not noteworthy. Will make again.

I also made oatmeal, of the steal cut variety cooked using the overnight method where you boil the oats in milk and let it simmer for two minutes, and then let it cool and sit over night, and then boil again in the morning and simmer for 10-12 minutes. My modification was to put a couple of tea spoons of olive oil in with the oats at night. I served with blackberry jam and greek yogurt.

And, of course, several pots of coffee. (Chemex, for the win.)

Simple, not a lot of work. Excessively delicious. Nice to have people in my living room, chatting, eating, and enjoying being with people, and a good way to spend time on a weekend.

I used to think of weekend mornings as an opportunity to write or work on my own projects that required a lot of attention and focus. I’m not sure that I really like this way of ritualizing creativity. I also now live in the same neighborhood as a large number of my friends with half a dozen brunch places (or more!) within a mile. Brunch became a comfortable baseline ritual, and something that I’m much more interested in carving out space for.

While I certainly enjoy going somewhere for brunch, there’s something great about having people over and feeling awesome and comfortable about having friends over.

I don’t think I’m a different person now than I was before. I don’t think its fair or correct to narrate the changes in my life as a progress narrative. Lives change, settings change, people change: things are different, and perhaps that’s enough.

For the last 9 years or so, I’ve kept a copy of a novel that I wrote during my junior year of high school. It’s so crappy: I think I have a Word 95 file of it somewhere, but it mainly exists in this binder, which turns out to be just the right height to use as a foot rest, to keep my feet planted on the floor while I sit up at my desk.

It’s so bad folks. So bad. A few years back I reread the prologue in order to rip off a few things for a different, only slightly less terrible story. The plotting isn’t particularly inventive, the characters are juvenile, and the settings are bland.

I realized the other day that it’s like all of those shell scripts that I wrote when I was learning to program that are hundreds of lines long, that intermingle argument parsing with actual logic, with horrible unabstracted data structures and access patterns, and implementations that are neither clear nor efficient. Those same scripts that I run a few dozen times a day because they just work, and if I don’t think about it too hard, it doesn’t matter much.

There’s also the two hundred thousand words of total crap that I wrote for my last job. And another couple of un-salvageable fiction projects that I didn’t even bother to print. Even with my current job, I sometimes come across awful paragraphs that I know that I must have written a few years ago, but completely fail to understand what I was thinking.

We all, probably have our own examples of the detritus of our past efforts. I spent a long time feeling like these concrete examples of my ineptitude were a burden, a body of work that I had to revise before I could start working on new projects.

Untrue.

I learned a lot from writing these crappy pieces of code, the failed stories, and terrible pieces of work product. Furthermore all of the work I’ve done in these areas since producing the terrible dark piles, has been better, and continues to get better.

Embedded in the idea that these dark piles are a burden is the notion that the production of text, of code is difficult, challenging, and the majority of the work in creation. But this isn’t true. The work is in figuring out the interfaces, the implementation, figuring out the structure and flow of ideas through a system of functions or paragraphs. Typing it all out is the easy and fun part. It is, of course, important to revise and polish work, but sometimes it’s enough to learn from your mistakes, to remind ourselves of our progress, and move on.

Every time I sit down to write, I put my feet on this binder and remember how far I’ve come.

More than anything, I am thankful.

I’ve not posted for a long time. A lot of things have changed since I last wrote, and this post is probably not the best place to recount all of them. Indeed many things haven’t changed, but the highlights..

I’m still living in New York, but I bought a coop in Brooklyn and moved, which has been great. I’m surprised at how quickly I have felt at home and rooted. The sequence of changes in my life that brought me there are simple, really, but I’ve struggled to make sense of things even so. The fact that I am aware of development is both a great comfort, but it has been hard to write about my life with confidence.

I replaced my tea habit with a coffee habit. I attend yoga classes regularly. I sing Sacred Harp (and sometimes other shaped notes). I Morris dance with the Bowery Boys, the Men’s team in New York City, and continue to dance with Braintrust Morris, an kind of butch Morris team with a spiritual center in the Midwest.

I’m still working on the same kinds of developer documentation, build systems, and software development projects for the same company as before. I still think open source software is important. I am really interested in helping people develop technological literacy and understanding. I want to work on improving infrastructure for developers and development. I use Linux extensively, write a lot of Python, tinker with Go and Common Lisp, and live and breathe in Emacs.

Life is good. Life is difficult. Life is.

I look forward to writing about it here.

So the blog looks different. I changed some things:

I switched from using ikiwiki to using Sphinx and an extension called ablog. I wanted to use a system that I was familiar with and could hack on (that’s Sphinx, which is the core of the tool chain I use at work.) Also, I wanted to use reStructuredText, which I prefer.

Ikiwiki is great software, and I quite enjoy it’s architecture and use. The problem is that it’s not in particularly active development, the code base is in Perl (which I don’t know, and except for ikiwiki, have no reason to learn,) and the project has probably peaked in terms of its adoption curve. To boot, my goals and user story aren’t totally inline it’s goals and user stories.

Sphinx isn’t totally right for a blogging engine either, but it’s solid. I actively develop and maintain tools around Sphinx. Indeed, I was able to make some small changes in ablog that shaved about a third of the time off of the total build time. Not bad.

It took a lot of time to convert all the posts to the new format, but now that everything is in order and the tools are usable it is time to start writing again.

It’s good to be home.

I haven’t really written very much in the past six months or more, and while I think I’ve done cool things and learned about cool things,

Let me take that back. I haven’t written anything in a sustained sort of way that wasn’t for work. Hell, even what I’ve been doing for work has been smaller and more tactical. Unfortunately you don’t write books, or book-like-objects as small tactical approaches.

So I’ve been trying to figure out how to get back to that. It’s about rhythm more than anything. If you approach time management for log-form rationally, there’s never enough time, so you have to make time, and trick yourself into writing and figure out how to make a little progress on a regular

As someone who definitely tends towards binge writing (and who doesn’t) remembering to write a little bit on a regular basis is hard and nearly counter intuitive. So as I’ve been attempting to restart the writing habit, I’ve been thinking about what I’ve done in the past that’s worked to keep up the momentum and work on projects. I’ve uncovered:

Reading

Even more important, I think, than reestablishing a habig of writing regularly, is restablishing the practice of reading regularly. I often get hung up on the fact that I don’t think I’m a very fast or very through reader. But reading is quite inspiring, and I often find that the more I read, the more I want to write.

So I’ve been reading the Vorkosigan novels, and it’s been great.

Tracking Word Counts

I like having some record of my progress, and I’ve taken different approaches to tracking progress, mostly using word counts, over time.

When you’re writing longer pieces, particularly in editors that don’t reflect word counts or page numbers in the interface, it’s easy to loose context for how much progress you’re making.

Way back when, I used to record the current page and word count for all of my projects in a note boot, and this developed into a rather incurable tick to mash a few keys down every few sentences to check the word count. Then, three years ago, I wrote a script to check the word count of all my projects on a regular interval.

I stopped really using it about a year ago, because the script handled different branches (in git) of the same project really poorly, and there wasn’t a good way to hack that in.

So I recently rewrote this program, and I rather like it. I’ll post more about it soon.

Physical Activity and Care

To do writing one definitely needs to spend a serious amount of time in front of a computer typing. Unless you really like notebooks and pens, there’s no other way to get things written.

But you can’t take this to the extreme: if all you do is sit in front of your computer and stare at cursor waiting for inspiration to strike. As people we need different kinds of focus, and different kinds of experiences, on the small scale, to keep the generative impulses functional.¹

Reading voraciously, as above, is a key part of being able to write effectively, but other kinds of activity are also useful. When I’m not writing, my first instinct is to try and find more time to write, but sometimes--often--what I need more is more time for things that aren’t writing: exercise, reading, and so forth.

I’ve started to do yoga, as I’ve written before, a few times a week. I need to also remember to set aside time for other non-writing activities: more exercise, walks, dinners with friends, dates with my email backlog, etc.

Onward and Upward!

I’ve been a big fan of fabric which is a Python tool for deployment orchestration: deploying a new release of an application, uploading files, deploying new configurations to a group of hosts in a cluster. Before fabric, the options were either to write fragile shell scripts that often didn’t do a good job of handling multiple hosts, or use more heavy weight configuration management tools, which had a lot of overhead and bother.

Fabric is great. Make a python module called “fabfile,” write idiomatic Python, stick anywhere, and then you can call functions in that module using the fab command. And it includes a lot of helpers for interacting with groups of hosts and dealing with remote and local resources that is designed for takes into consideration common contemporary application development.

For a while, I basically thought that fabric, or tools like it would completly obviate the need for shell scripting

I think there are two major problems with using fabric as a framework for general purpose scripting:

1. Global state can get messy.

   This isn’t only an issue with Fabric, but I think the fab solution isn’t great here. Fabric has this envy object that you can import and then use to track the settings or set custom “global” data values. It’s better than having actual global data, maybe, but you end up having to use it too much, particularly it for communication between different related operations and you end up using tasks and global state to run-time configuration.

2. There’s no real overriding abstraction to guide code organization.

   In a lot of ways fab scripts are just a way of writing a large number of scripts within a single code base and a command line interface for specifying different entry points. It would be nice to be able to express relationships between tasks, or centralize the registry of tasks in some of non-adhoc.

I actually don’t think that these are problems with fabric, itself, but I think it’s probably the case that we need a few different kinds of tools and patterns to help support the authorship of useful and maintainable shell scripts and build systems.

• a good, usable, non-procedural way manage argument parsing.

  In python, I like argprase and think it works really well, so I’m not saying we need some set of tools to actually parse the arguments that people input on the command line. But I think when you’re writing a script, argparse is too low level, and pretty soon there are a few hundred lines of declarative code, and you still have to connect this up with the actual code, which leads to maintenance headaches and all sorts of dumb errors.

• a good framework for expressing multi-stage tasks and their dependencies and relationships.

  This may basically be waf, but I think we can do something similar. I made an attempt at this with buildcloth but I think I ended up with something that’s too much of a (meta) build automation tool, and not enough of a general purpose paradigm for organizing code to perform related tasks in a clean, modular, and well organized sort of way.

These aren’t fabric specific problems, or Python-specific problems, of course. But they are particularly relevant to Python moderatly sized Python code bases in general and most fabric implementations that I’veb seen. Maybe Python and fabric make writing useful code easy enough that we start to see complexity problems earlier than we would otherwise. Not sure.

As you may have been able to guess at this point, I am somewhat burdened by a large and somewhat overgrown fabric-based project, that needs refactoring. Thankfully, matters are not critical, but I do want to figure out the right way to solve this kind of problem before I start hacking.

At work people often ask for more “best practices” guides. In some ways this is sign of success: they’re no longer begging for fundamental reference material and descriptions of basic use. Nevertheless I almost always wince:

1. “Best practices” carries an implicit sense of guarantee along the lines of “if you adhere to the best practices, then you won’t run into problems,” which is sort of difficult to assert with confidence, and is really a product design issue, not a documentation issue.

2. It is really hard to make one-size-fits-all recommendations.

3. Best practice-guides don’t address actual needs of users, or actual solutions to real problems, because they generalize problems and solutions beyond the point of re-usability.

This isn’t to say that it’s difficult or impossible to give recommendations, and indeed documentation is the best way that the purveyors of software have to to shape practices. However, the documentation should guide users towards better practices everywhere and isolating recommendations into certain types of documents is probably counter productive.

Onward and Upward!

As part of my project to learn common lisp, or at least write more common lisp as part of my day to day work and life, I’ve

This is a total rip off of this blog post, with a few minor changes:

• I hacked some makefile goodness so that it will automatically create binaries for all .lisp files, and means that you can drop a script in the directory and not have to edit the makefile to get the magic to happen.
• I switched to using buildapp to rather than cl-launch. Buildapp feels a bit more maintained, and I wanted practice using it. Otherwise, I don’t think it matters.

The makefile:

TARGETS := $(subst .lisp,,$(wildcard *.lisp))

all:$(TARGETS)

%:%.lisp
    @echo [build]: creating $@
    @buildapp --load $< --entry script:run --output $@
    @echo [build]: created $@

clean:
    @rm -f $(TARGETS)

Hello world:

(cl:defpackage #:script
(:use #:cl)
(:export #:run))

(cl:in-package :script)

(defun run (argv)
   (format t "hello world~%"))

As an aside, I also updated the buildapp aur package.

Pros:

• Common lisp scripts.
• The ability to integrate writing lisp into your existing Linux/Unix workflow and processes.
• Not having to think about packaging or build architecture for trivial operations.

Cons:

• ~40+ meg executable.
• Only one source file per script.