I’ve never been really good at the blogging+picture game, and while maybe once upon a time it was technical limitation--taking photos and getting them online was complicated--anymore it’s probably not. To this end, I’ve started a knitting specific Instagram account as a kind of photoblog for knitting things. It’s @gestaltknitting, if you’re interested.

While I took this picture a while ago, I must confess that my knitting basically looks the same now.

The same, not because I’ve made no progress, but because sleeves take a while and it’s just plain knitting, so unless you have a very discerning eye, you might miss the details.

Indeed, I really want my next project to also have a lot of plain knitting with black yarn: I expect the photographs will be captivating. Perhaps it will be enjoyable for people to be able to spot the different patterns of embedded cat hair in the sweaters.

I get that knitting is visual for a lot of people, and I do like a smart looking sweater as much as the next guy, but I’ve always felt somewhat resistant to this view: knitting is about the process and the act more than it is about the product, and so the things that are most exciting aren’t the visuals.

While it’s gotten much easier to take high quality pictures, my intention for this book that I’ve been writing is that it mostly would not be a book with a lot of picture, though we’ll see: If anything, I suspect that diagrams and cartoons may be more effective for this kind of application.

Having said that, it’s nice to see what other people are knitting, and I like the way that the ephemeral nature of instagram stories make it less daunting to post in-progress updates on projects. So I’ve definitely been enjoying that.

We’ll see!

It seems obligatory to mark the new year, and what a year it was, for me and everyone else: pandemic, quarantine, changing jobs, new hobbies, totally different routines. The backdrop of the pandemic makes looking back (and forward!) so weird, and I find myself asking: will the things that changed in my life still be true when there’s less quarantine? did the things that happen in 2020 happen because of the pandemic or would they have happened anyway? The certainty that most (but not all!) of 2021 will, in practical terms, look a lot like 2020 is intense and makes this whole “obligatory new years” thing a bit harder. Instead of doing something like “resolutions”/“goals”--which are always fraught--or some kind of lofty synthetic review of the last year, I think I’d like to muse on features of 2020 that I hope and expect to continue in 2021

• Knitting: I took a lot of time off of knitting, but I found that I’d been missing it, and I think there are ways that it fits well into my life quarantine or no. It’s also been quite fun to write about knitting and knitting projects, and become more engaged with other knitters, and I look forward to knitting things for friends and family. I have list in my head of some nifty ideas for sweaters and some other things to knit, so I expect this to stay.
• Blogging: I’ve been writing blog posts for years and while I’ve always found it rewarding, but everything else related to blogging has been hard. During the summer, while I was interviewing for jobs, I wrote blog posts most days, and sort of fell down on posting them, and still have 25 (or so) in the draft folder. Writing is the easy part, it’s editing (or letting go!) promotion, and remembering to move things out of draft. I’ve spruced up the blog and done some work to automate regular publishing, and it seems like I might be able to make this work! I definitely want to.
• Pickling: I’ve really enjoyed pickling things, and if anything I expect that I’ll enjoy doing this more after quarnainte when it’s easier to share these things. Right now I have some cranberries on the go, and a lot of sauerkraut in the fridge that I’m slowly eating. It’ll be fun to have more people to share it with! I’m excited to explore radishes as well as nappa cabbage.
• Coffee: I started drinking coffee in 2014 and have mostly had coffee made by other people: tech jobs in NYC have pretty great office coffee, and the process was something of a mystery to me at the beginning. While I had a Chemex and made coffee for myself sometimes, it was definitely a special occasion sort of thing and not part of my regular routine. Now making a pot of coffee is part of my morning routine, and I find it pretty satisfying, and while I definitely look forward to drinking coffee that other people make more often in the future, I really like getting up making a pot of coffee and sitting down to write nearly every morning.

I’ve had a blog¹ for more than 15 years, and I’ve found this experience to be generally quite rewarding. I’ve learned a lot about writing, and enjoyed the opportunity to explore a number of different topics² in great detail. While I haven’t blogged as much in recent years, I’ve been thinking in the past few weeks about getting back into writing more regularly, which has lead me to reflect on my writing in the past and my goals for this in the future.

First, the blog as a genre has changed fundamentally in the last 17 or 18 years. In 2000 or 2001, blogs were independent things that grew out of communities (e.g. MetaFilter, or web-diary/journaling) and were maintained by independent writers or small groups. Then the tooling got better, the community got better and eventually started to segment based on topic, and finally the press³ gained competence in the form.

Publishing a blog today, is a vastly different proposition today even in the recent past.

True to my form, this leaves me with a collection of divergent thoughts:

• maybe the “write everything in one blog even though the topics are not really of interest to any specific group” approach that I’ve always taken. More distinct blogs means more writing (maybe a good thing,)
• having a writing practice is good for focusing thoughts, but also for sharing and distributing understating, and I think that sharing understanding is a really important part of learning and growing, and I miss having a structure for these kinds of notes.
• perhaps, it would make sense to outsource/hire a freelancer to take care of some editing and marketing-adjacent work, which is more required if you want to engage with users more consistently but that I find distracting and outside of my ability to focus on properly. The problem then is figuring out how to fund that work in a longer-term/sustainable way.
• In the past RSS has been a (the?) leading way to distribute content to serious readers, but that isn’t true now and likly hasn’t been true for years. So while I feel able to write a lot of things, I don’t know what the best way to engage with regular readers is
• I used to think that I wanted to organize group blogs, and while I think that blog-discussions are fun, and I think there is merit to combined efforts, I’m less interested in doing the organizing myself.
• There was a period where I wasn’t blogging much because my day job was very writing focused, and I needed side projects that didn’t involve the English language, and I spent a long time focusing on learning programming, which took a lot of time. Now that work mostly involves programming, and only a little writing, and I’ve had some time to recover as a writer, it feels like I have some space.
• I’m not really sure how to host a blog in 2018. The old set up and server I have is more than functional, but there are a lot of services, tools, and patterns that I’m not familiar with and I have some learning to do, even though I probably mostly just want to write things.

In an effort to relaunch tychoish with a more contemporary theme and a publishing tool that (hopefully) will support a more regular posting schedule, I also wrote a nifty go library for dealing with reStructuredText, which may be useful and I think illustrates something about build systems.

In my (apparently still) usual style, there’s some narrative lead in that that takes a bit to get through.

Over the past couple of weeks, I redesigned and redeployed my blog. The system it replaced was somewhat cobbled together, was missing a number of features (e.g. archives, rss feeds, social features, etc) and to add insult to injury it was pretty publishing was pretty slow, and it was difficult to manage a pipeline of posts.

In short, I didn’t post much, though I’ve written things from time to time that I haven’t done a great job of actually posting them, and it was hard to actually get people to read them, which was further demotivating. I’ve been reading a lot of interesting things, and I’m not writing that much for work any more, and I’ve been doing enough things recently that I want to write about them. See this twitter strand I had a bit ago on the topic.

So I started playing around again. Powering this blog is hard, because I have a lot of content¹ and I very much want to use restructuredText. ² There’s this thing called hugo which seems to be pretty popular. I’ve been using static site generators for years, and prefer the approach. It’s also helpful that I worked with Steve (hugo’s original author) during its initial development, and either by coincidence, or as a result our conversations and a couple of very small early contributions a number of things I cared about were included in its design:

• support for multiple text markup features (including reStructuredText,) (I cobbled together rst support. )
• customizeable page metadata formats. (I think I pushed for support of alternate front-matter formats, specifically YAML, and might have made a few prototype commits on this project)
• the ability to schedule posts in the future, (I think we talked about this.)

I think I also winged a bunch in those days about performance. I’ve written about this here before, but one of the classic problems with static site generators is that no one expects sites with one or two thousand posts/content atoms, and so they’re developed against relatively small corpus' and then have performance that doesn’t really scale.

Hugo is fast, but mostly because go is fast, which I think is, in most cases, good enough, but not in my case, and particularly not with the rst implementation as it stood. After all this preamble, we’ve gotten to the interesting part: a tool I’m calling shimgo.

The initial support for rst in hugo is straight forward. Every time hugo encounters an rst file, it calls the shell rst2html utility that is installed when you install docutils, passing it the content of the file on standard input, and parsing from the output, the content we need. It’s not pretty, it'’s not smart, but it works.

Slowly: to publish all of tychoish it took about 3 minutes.

I attempted an rst-to-markdown translation of my exiting content and then ran that through the markdown parsers in hugo, just to get comparative timings: 3ish seconds.

reStructuredText is a bit slower to parse than markdown, on account of it’s comparative strictness and the fact that the toolchain is in python and not go, but this difference seemed absurd.

There’s a go-rst project to write a pure-go implementation of reStructuredText, but I’ve kept my eye on that project for a couple of years, and it’s a lot of work that is pretty far off. While I do want to do more to support this project, I wanted to get a new blog up and running in a few weeks, not years.

Based on the differences in timing, and some intuition from years of writing build systems, I made a wager with myself: while the python rst implementation is likely really slow, it’s not that slow, and I was loosing a lot of time to process creation, teardown, and context switching: processing a single file is pretty quick, but the overhead gets to be too much at scale.

I built a little prototype where I ran a very small HTTP service that took rst as a POST request and returned processed HTML. Now there was one process running, and instead of calling fork/exec a bunch, we just had a little but of (local) network overhead.

Faster: 20 second.

I decided I could deal with it.

What remains is making it production worthy or hugo. While it was good enough for me, I very much don’t want to get into the position of needing to maintain a single-feature fork of a software project in active development, and frankly the existing rst support has a difficult to express external dependency. Adding a HTTP service would be a hard sell.

This brings us to shimgo: the idea is to package everything needed to implement the above solution in an external go package, and package it behind a functional interface, so that hugo maintainers don’t need to know anything about its working.

Isn’t abstraction wonderful?

So here we are. I’m still working on getting this patch mainlined, and there is some polish for shimgo itself (mostly the README file and some documentation), but it works, and if you’re doing anything with reStructuredText in go, then you ought to give shimgo a try.

I’ve posed about some of these projects before, and I used to regularly post little overviews of the things that I’m working on. But I’ve not done a lot of this recently. Time to do some back fill.

I think it’s probably good to take a step back from time to time and inventory and evaluate the priorities of various projects. Not to mention the fact that I usually say “I’m not really doing much these days,” when this isn’t really true. Here goes:

Mango

This is a project that is private, at the moment, because its mostly useful as an experimental piece of testing infrastructure for work. The idea is to use the same underlying infrastructure to start, stop, and configure processes, but provide REST and command line interfaces for all of these operations.

We have a lot of distinct software that does this internally and it’s always fragile and limited. While grand discussions of code reuse are sort of silly, in this case, it’s a bit annoying that we’ve reinvented this wheel a dozen times… And have to make different changes to a dozen tools as configurations change.

This was also my first project written in Go, which means its been a great learning experience and the place where a number of other Go packages that have become useful in their own right.

Future work:

• Write all the tests.
• Make the REST interface feature compatible with one of the legacy tools it aims to supplant.
• Make a new REST interface that’s more sensible and might be easier to use in more circumstances.
• Figure out better ways to block for the appearance of synchronous operations, despite the fact that internally the operations are non-blocking.

Gimlet

Gimlet Blog Post

Gimlet Github

This is really just some convenience work to make it easy to build REST interfaces in Go, without needing to suffer through tools that are designed to support complete “full-stack” web applications. It’s built on the same Negroni/Gorilla Mux stack that I think everyone uses, and it’s very net/http compliant, but with an API that makes it easy (even fun,) to provide high quality JSON+HTTP interfaces.

It struck me, when working on part of Mango, that this chunk of the code didn’t have anything to do with the actual core application and was all about getting a REST-like application to happen. So I split that out, for everyone’s pleasure/suffering.

Future work:

• Documentation.
• More tests.
• Exposed API stabilization and versioning.
• Develop story for authentication, sessions and SSL termination.

Grip

Grip Blog Post

Grip Github

Grip is a logging package for Go that attempts to resolve my constant feelings of “I miss x feature of the Python logging package.” It’s not feature comparable with Python logging (but that’s ok,) and since I was working on writing a logging package, I got to add some nifty features.

Future Work:

• More documentation.
• Better examples, and potentially support for “print this message x% of the time.”
• Support for logging to conventional syslog sources in addition to systemd’s logging.

Archer/Dropkick

I’ve wanted to work on a tool to unify a number of personal operations and scripts in a single system and tool. The problem that I’m trying to solve is that I have a number of different computers that I use with some frequency, including laptops, desktops, and a number of servers, and test systems, and I want to be able to describe their configuration, and synchronize files and git data between machines with ease.

My first approach was getting a bunch of random system setup scripts out of a makefile and into a configuration file that a Go program knew how to read and process, and then to expand from there.

I haven’t gotten to the git repository management stuff, because I was working on the Gitgone project.

Future Work:

• add better support for creating and managing containers and images using systemd-nspawn and docker.
• support for setting up git repositories
• support for syncing automatically (i.e. dropox-like functional it -> dropkick).
• report status of repositories via a REST API
• triggering syncs on remote systems.

Gitgone

Gitgone Github

The idea here is to provide a consistent and full featured way to access and interact with git repositories from Go without needing to wrap the git command yourself (or worse, learn the ins and outs of the git2go). This is largely modeled off of a similar project I did as part of libgiza that does the same sort of thing for Python repositories.

The cool thing about this project is its build abstractly so that you can use one interface and switch between a libgit2 implementation and one that wraps the git command itself.

Future Work:

• complete implementation using libgit2
• write more extensive tests.
• add support for creating repository tags.
• provide access to the log.

Novel Project

I’ve been, sporadically, working on notes for writing a new novel. It’s not going anywhere fast, and I’m not even to the point. where I’m outling plot.

I’m trying to tell a story about urban development and how smaller local communities/groups participate in larger communities/groups. How does urban development in place a, impact nation building more globally, and what does this all look like to people as they get to work in the morning, and have to build neighborhood institutions like gyms and restaurants and grocery stores.

But there’s a lot of work to do, and while thinking about the project is fun, there’s a lot of work, and I feel like I’m not ready to commit to a writing project of this scope and, I’m not sure how publishable this project will be (and furthermore, even if its' publishable, will I be willing to do all of that work.)

Software projects are much harder to justify and prioritize than writing projects.

I’ve not posted for a long time. A lot of things have changed since I last wrote, and this post is probably not the best place to recount all of them. Indeed many things haven’t changed, but the highlights..

I’m still living in New York, but I bought a coop in Brooklyn and moved, which has been great. I’m surprised at how quickly I have felt at home and rooted. The sequence of changes in my life that brought me there are simple, really, but I’ve struggled to make sense of things even so. The fact that I am aware of development is both a great comfort, but it has been hard to write about my life with confidence.

I replaced my tea habit with a coffee habit. I attend yoga classes regularly. I sing Sacred Harp (and sometimes other shaped notes). I Morris dance with the Bowery Boys, the Men’s team in New York City, and continue to dance with Braintrust Morris, an kind of butch Morris team with a spiritual center in the Midwest.

I’m still working on the same kinds of developer documentation, build systems, and software development projects for the same company as before. I still think open source software is important. I am really interested in helping people develop technological literacy and understanding. I want to work on improving infrastructure for developers and development. I use Linux extensively, write a lot of Python, tinker with Go and Common Lisp, and live and breathe in Emacs.

Life is good. Life is difficult. Life is.

I look forward to writing about it here.

So the blog looks different. I changed some things:

I switched from using ikiwiki to using Sphinx and an extension called ablog. I wanted to use a system that I was familiar with and could hack on (that’s Sphinx, which is the core of the tool chain I use at work.) Also, I wanted to use reStructuredText, which I prefer.

Ikiwiki is great software, and I quite enjoy it’s architecture and use. The problem is that it’s not in particularly active development, the code base is in Perl (which I don’t know, and except for ikiwiki, have no reason to learn,) and the project has probably peaked in terms of its adoption curve. To boot, my goals and user story aren’t totally inline it’s goals and user stories.

Sphinx isn’t totally right for a blogging engine either, but it’s solid. I actively develop and maintain tools around Sphinx. Indeed, I was able to make some small changes in ablog that shaved about a third of the time off of the total build time. Not bad.

It took a lot of time to convert all the posts to the new format, but now that everything is in order and the tools are usable it is time to start writing again.

It’s good to be home.

For the most part, I’m quite happy with everything that I was able to accomplish last year. I’ve moved cities (for the second year in a row) and last year I changed jobs twice: in both cases, I think the current will stick for a while. And I’m working on other projects, with some impressive speed. Last year wasn’t been great for finishing things, but I guess there’s room for improvement this year.

After a fair amount of professional angst I’m finally doing pretty much exactly what I want to be doing: I’m writing a substantial/total revision of a software manual for a company developing an open source database system. I’ll leave you to figure out the details, but it’s great.

A couple years ago, I said to myself, that I wanted to be a “real technical writer,” which is to say, work with engineering teams, write documentation and tutorials for a single product or group of products, and operate on a regular release schedule. I’ve done a great deal of writing for technology companies: from project proposals and journalism, to tutorials and content for distributors, to white papers, marketing, and sales materials. Delightfully, I’ve managed to get there, and in retrospect it’s both somewhat amazing, and incredibly delightful.

A while back, I had dinner with a friend who’s been doing the same thing I do for a long time (we know each other through folk dance and singing,) and by comparing our experiences it was great to learn that my experience is quite typical, both in terms of the work I’m doing and the procedural engineering practice frustrations (e.g. “What do you mean you changed the interface without telling me?!?!")

At work we have this thing where we send in an account of what we did during the day so that other people know what we’re working on, and so that we can keep our team on the same page. After all, when you’re all looking at computer screens all day, and in a few different time zones, it’s easy to loose track of what people are working on.

At the bottom of these emails, we’re prompted to ask “what are your blockers and impediments.” Often I say something clever like “Compiler issue with Spacetime interface or Library.” Or something to that effect. It feels like a good description of the last year.

Onward and Upward!

I went through a few days ago to collect all of the updates and work that I’d done since the last time I did one of these posts. Sometimes just looking through an activity log is all you need to remember that you’re actually doing something. Here’s what I’ve been working on:

• isd" is a developing page of material for my ongoing story project. I’ve written it in my “Wiki Fiction” post. This particular page is basically sysadminning-punk, if I may be so bold. The story as a whole is largely a future history of human’s expansion into the solar system and beyond, using Kim Stanley Robinson’s idea of Accelerando. This piece in particular is all about the emergence of an asynchronous database network that eventually replaces the Internet as we know it, because interstellar distances render trans-planetary TCP (in any form) unworkable. I’ve got an outline and I’m slowly working to fill in the blanks. Editing, ideas, and any amount of text are all welcome.
• I added a page to the technical writing section on “Why the Documentation Sucks,” and I’ve done some additional editing and revising of this section of the wiki. The /technical-writing/documentation-sucks page has also served as the foundation of my post about documentation publishing systems.
• I’ve added a number of new posts that I hope won’t be forgotten too quickly. They are:
  • Is Android the Future of Linux
  • Advice for Blogging Successfully
  • :Issue Tracking and the Health of Open Source Software
  • Tablet Interfaces and Intuition
  • Org Mode and Mobile Writing
  • Publishing System Requirements
  • Android Tablets and the Workstations of the Future
• There has also been some discussions of new and old rhizomes on the discourse pages.
• A few weeks ago I posted a few new scripts in the code section. Tell me what you think:
  • ikiwiki tasklist
  • download mail
• My post, 9 Awesome SSH Tricks from March 2011, spent a number of hours on the front page of hacker new. You can read the comments over there, and probably on some other sites as well.

The coolest part about this is that some of you have helped to build a page of ssh tricks on the wiki that go above and beyond the little tricks that I use.

And finally,I ’d like to welcome Kevin Grande, who made a new folk page recently. I’m also very sorry that I haven’t been updating more frequently. I started a new job on September 26, and between that and my usual gallivanting around for singing and dancing my blogging habit has. One the other hand I’m writing about 1200 words a day, and life is pretty good so no complaints there.

Onward and Upward!