Values for Collaborative Codebases

After, my post on what I do for work I thought it'd be good to describe the kinds of things that make software easy to work on and collaborative. Here's the list:

  1. Minimal Documentation. As a former technical writer this is sort of painful, but most programning environments (e.g. languages) have idioms and patterns that you can follow for how to organize code, run tests and build artifacts. it's ok if your project has exceptional requirements that require you to break the rules in some way, but the conventions should be obvious and the justification for rule-breaking should be plain. If you adhere to convention, you don't need as much documentation. It's paradoxical, because better documentation is supposed to facilitate accessibility, but too much documentation is sometimes an indication that things are weird and hard.
  2. Make it Easy To Run. I'd argue that the most difficult (and frustrating) part of writing software is getting it to run everywhere that you might want it to run. Writing software that runs, even does what you want on your own comptuer is relatively, easy: making it work on someone else's computer is hard. One of the big advantages of developing software that runs as web apps means that you (mostly) get to control (and limit) where the software runs. Making it possible to easily run a piece of software on any computer it might reasonably run (e.g. developer's computers, user's computers and/or production environments.) Your software itself, should be responsible for managing this environment, to the greatest extent possible. This isn't to say that you need to use containers or some such, but having packaging and install scripts that use well understood idioms and tools (e.g. requirements.txt, virtualenv, makefiles, etc.) is good.
  3. Clear Dependencies. Software is all built upon other pieces of software, and the versions of those libraries are important to record, capture, and recreate. Now it's generally a good idea to update dependencies regularly so you can take advantage of improvements from upstream providers, but unless you regularly test against multiple versions of your dependencies (you don't), and can control all of your developer and production environments totally (you can't), then you should provide a single, centralized way of describing your dependencies. Typically strategies involve: vendoring dependencies, using lockfiles (requirements.txt and similar) or build system integration tends to help organize this aspect of a project.
  4. Automated Tests. Software requires some kind of testing to ensure that it has the intended behavior, and tests that can run quickly and automatically without requiring users to exercise the software manually is absolutely essential. Tests should run quickly, and it should be possible to run a small group of tests very quickly to support iterative development on a specific area of the code. Indeed, most software development can and should be oriented toward the experience of writing tests and exercising new features with tests above pretty much everything else. The best test suites exercise the code at many levels, ranging from very small unit tests to ensure the correct behavior of the functions and methods, to higher level tests that test the functionality of higher-order functions and methods, and full integration tests that test the entire system.
  5. Continuous Integration. Continuous integration system's are tools that support development and ensure that changes to a code pass a more extensive range of tests than are readily available to developers. CI systems are also useful for automating other aspects of a project (releases, performance testing, etc.) A well maintained CI environment provide the basis for commonality for larger projects with a larger number for projects larger groups of developers, and is all but required to ensure a well supported automated test system and well managed dependency.
  6. Files and Directories Make Sense. Code is just text in files, and software is just a lot of code. Different languages and frameworks have different expectations about how code is organized, but you should be able to have a basic understanding of the software and be able to browse the files, and be able to mostly understand what components are and how they relate to each other based on directory names, and should be able to (roughly) understand what's in a file and how the files relate to eachother based on their names. In this respect, shorter files, when possible are nice, are directory structures that have limited depth (wide and shallow,) though there are expections for some programming language.
  7. Isolate Components and Provide Internal APIs. Often when we talk about APIs we talk about the interface that users access our software, but larger systems have the same need for abstractions and interfaces internally that we expose for (programmatic) users. These APIs have different forms from public ones (sometimes,) but in general:
    • Safe APIs. The APIs should be easy to use and difficult to use incorrectly. This isn't just "make your API thread safe if your users are multithreaded," but also, reduce the possibility for unintended side effects, and avoid calling conventions that are easy to mistake: effects of ordering, positional arguments, larger numbers of arguments, and complicated state management.
    • Good API Ergonomics. The ergonomics of an API is somewhat ethereal, but it's clear when an API has good ergonomics: writing code that uses the API feels "native," it's easy to look at calling code and understand what's going on, and errors that make sense and are easy to handle. It's not simply enough for an API to be safe to use, but it should be straightforward and clear.

6 Awesome Arch Linux Tricks

A couple of years ago I wrote "Why Arch Linux Rocks" and "Getting the most from Arch Linux." I've made a number of attempts to get more involved in the Arch project and community, but mostly I've been too busy working and using Arch to do actual work. Then a few weeks ago when I needed to do something minor with my system--I forget what--and I found myself thinking "this Arch thing is pretty swell, really."

This post is a collection of the clever little things that make Arch great.

abs

I'm using abs as a macro for all of the things about the package build system that I enjoy.

Arch packages are easy to build for users: you download a few files read a bash script in the PKGBUILD file and run the makepkg command. Done. Arch packages are also easy to specify for developers: just specify a "build()" function and some variables int eh PKGBUILD file.

Arch may not have as many packages as Debian, but I think it's clear that you don't need comprehensive package coverage when making packages is trivially easy.

If you use Arch and you don't frequent that AUR, or if you ever find yourself doing "./configure; make; make install" then you're wasting your time or jeopardizing the stability of your server.

yaourt

The default package management tool for Arch Linux, pacman, is a completely sufficient utility. This puts pacman ahead of a number of other similar tools, but to be honest I'm not terribly wild about it. Having said that, I think that yaourt is a great thing. It provides a wrapper around all of pacman's functionality and adds support for AUR/ABS packages in a completely idiomatic manner. The reduction in cost of installing this software is quite welcome.

It's not "official" or supported, because it's theoretically possible to really screw up your system with yaourt but if you're cautious, you should be good.

yaourt -G

The main yaourt functions that I use regularly are the "-Ss" which provides a search of the AUR, and the -G option. -G just downloads the tarball with the package specification (e.g. the PKGBUILD and associated files) from the AUR and untars the archive into the current directory.

With that accomplished, it's trivial to build and install the package, but you get to keep a record of the build files for future reference and possible tweaking. So basically, you this is the way to take away the tedium of getting packages from the AUR, while giving you more control and oversight of package installation.

rc.conf

If you've installed Arch, then you're already familiar with the rc.conf file. In case you didn't catch how it works, rc.conf is bash script that defines certain global configuration values, which in turn controls certain aspects of the boot process and process initialization.

I like that it's centralized, that you can do all kinds of wild network configuration in the script, and I like that everything is in one place.

netcfg

In point of fact, one of primary reasons I switched to Arch Linux full time, was because of the network configuration tool, netcfg. Like the rc.conf setup, netcfg works by having a network configuration files which define a number of variables which are sourced by netcfg when imitating a network connection.

It's all in bash, of course, and it works incredibly well. I like having network management easy to configure, and setup in a way that doesn't require a management daemon.

Init System

Previous points have touched on this, but the "BSD-style" init system is perfect. It works quickly, and boot ups are stunningly fast: even without an SSD I got to a prompt in less than a minute, and probably not much more than 30 seconds. With an SSD: it's even better great. The points that you should know:

  • Daemon control scripts, (i.e. init scripts) are located in /etc/rc.d. There's a pretty useful "library" of shell functions in /etc/rc.d/function and a good template file in``/etc/rc.d/skel` for use when building your own control scripts. The convention is to have clear and useful output and easy to understand scripts, and with the provided material this is pretty easy.

  • In /etc/rc.conf there's a DAEMON variable that holds an array. Place names, corresponding to the /etc/rc.d file name, of daemons in this array to start them at boot time. Daemons are started synchronously by default (i.e. order of items in this array matters and the control script must exit before running the next script.) However, if a daemon's name is prefixed by an @ sign, the process is started in the background and the init process moves to the next item in the array without waiting.

    Start-up dependency issues are yours to address, but using order and background start-up this is trivial to implement. Background start ups lead to fast boot times.

Little Goals and Big Projects

Given all of the recent changes in my life, I found myself building and rebuilding a list of things that I'd like to "in the future," sometime. The list is more than a set of mid-year "New Years Resolutions," and it's not quite as impressive as a "5 Year Plan." As a blogger with firmly ingrained habits, I feel compelled to share this list with you:

  1. Properly resume my knitting and spinning habit.
  2. Read more veraciously. A book a month, plus the Asimov's subscription with which I've done a bad job of keeping current. And that's probably just a starting point. I have a backlog of things I want to read, there's new stuff that I want to read coming out all the time, and I'm not going to get that done without attending to it.
  3. I'd like to refresh/redo the cyborg institute site to reflect the reality of what I want to accomplish there, what I'm actually doing, and make it be updatable in a meaningful sort of way. I also want to get a number of project that I've been sitting on for a while together and published there. Those are:
  • The ikiwiki templates and configuration for tychoish.com.
  • A general purpose emacs "starter configuration," based on my configuration. It would be fairly simple to generate, but I want to write a build script so that I can keep it updated as I tweak my own configuration. I would also need to write some documentation.
  • Similarly, I would like to produce a basic configuration for StumpWM that would make it easier to use Stump.
  • A couple of other specifications along the lines of the Sygn.
  1. I would like to start dancing regularly with a Morris Dance team.
  2. I would like to reestablish a thrice-weekly non-dance exercise regime that includes some weight training.
  3. I want to apply to go to Clarion. But not before 2013, and preferably before 2016.
  4. I'd like to try learning to play melodeon for Morris, and maybe other tune good playing times.
  5. I'd like to go to Camp FaSoLa in 2012 or 2013. I'm also a little interested in maybe learning how to key sacred harp. Maybe I've been singing too much tenor recently and found myself as one of the not-new basses more than a few times that I'm getting a bit more confident.
  6. I'd like to finish this damn novel sometime soon, revise the first bit into a passable short story, and begin shopping the story around.
  7. I'd like to try writing short stories for a little while. But I also want to revise Knowing Mars, finish Station Keeping in some form, and potentially figure out a way to make the other open/lingering story on my plate become Knowing Mars, Part 2.
  8. Write more regularly here. I think about a lot of things that I think are interesting, and writing ideas and asking questions is really the only way to make progress on scholarly, intellectual, and/or creative projects. I'm aware of a lot of potential in this work on tychoish.com, and I'd like to be able to make significant progress.

five advancements

1. For many months, I've been using my Blackberry as a music player for the purpose of listening to podcasts while I walk/drive. I've recently switched to using an iPod Touch (first gen; obtained as part of a promotion) that I've been syncing (for podcasts) to my work computer (which is a mac). As nice as "convergence was," and as much as this little iPod is "overkill" for what I'm doing with it, I think this is a step in the right direction. So, dedicated media players here I come.

2. In college I'd drink tea pretty much all day--even into the evening--and it didn't really affect my ability to fall asleep when tired. Since graduating I'd pretty much scaled back on the caff, to the point where I'd avoid drinking tea after about 6pm. In the last couple of weeks, I've started streaching this downward, and making a big pot of tea in the evening and drinking it pretty much till it runs out, even if that means have a pipping hot cup of caff at 10pm. (mmmm.) The effect seems to be the same no matter what: I fall asleep at 11:30 or midnight-ish; but until then I'm a little more productive if I'm working on a cup of tea. Often without tea, I have the ability to read about a page in a book before falling asleep; with tea, I can read something of consequence before passing out. Again, I'm not actually staying up later, I'm just more productive/alert with the time I have. Also, I tend to have my first cup of tea at the office, and I don't really feel like I need to to wake up as much as I need it to stay alert. So, there you go.

3. I've had technology angst for a while. I miss having a single, portable machine that I can use for 98% of all my computing tasks. I have this awesome little laptop, but the screen's resolution is 1024x786 and at my (old-normal) font size that was simply too small to really do anything except work in full screen mode. Divide up the screen at all and suddenly you have to scroll horizontally for everything. It's a shame, because I love the portability of this computer, and it's built like a rock, and except for this trivial little thing, it works. After much dithering about on the subject (and spec-ing out the ideal laptop, which I do hope to own some day,) I had the brilliant thought (mid-post draft,) of setting to font size in emacs (my editor) to... I think the technical term is "insanely fucking small," and we'll see how that works. It's a bit too small for comfortable/close reading, but it's fine for moment to moment work, so I've set up some key-bindings in emacs (if you must know: "C-c f s" and "C-c f b") to switch between font sizes, along with some conditional statements in my init file to load big fonts on big screens and small fonts on small screens, and we'll see how it goes. I'll post the code soon. The sad thing is aside from the screen issue and the fact that my desk doesn't work well for laptops, I've basically already made the switch.

4. I've started to read Jonathan Straham's "The New Space Opera: 2" Anthology, and I have to say that it's already better than the first one. I liked the first one, a lot, enough to buy the second, but it was reflexive and subtle and understated in a way that this one is big and exciting and energetic. If these are even acceptable terms for the description of a collection of short stories. Books like this have left me convinced of the fact that there's a future in short fiction, and that the magazine isn't really the ideal format for the consumption of short fiction. Where I think there are some logistical problems with the publication of web-zines, in the "fiction periodical" model, doing one or two-off anthologies in the Internet-age has potential I think. Another project for Critical Futures, I suspect. (Also, space opera rocks.)

5. (Messages from the past:) I'm going to New York City (first time since I was a kid!) this weekend with some friends to do an all-day-shape-note singing. This is going to rock. I like small regular singings a lot, don't get me wrong, but I often feel like it takes me a few hours to get to the point where I can really feel the harmonies and chords, and most local signings are almost over by that point. This should be fun. (Summary: It was awesome, and I'll do it justice later, but I think getting into this singing thing counts as an advancement without question.)

Five By Five

Five Things I Could do to Make My Cats Happier

1. Have fewer hobbies that involve non-feline-objects on my lap (knitting, reading, writing/laptop).

  1. Be at home Sleep More.
  2. Grow More Appendages.
  3. Get a roommate/spouse.

Five Books I Would like to Read in the Next Month

  1. Iain M. Banks' Consider Phebas
  2. The New Space Opera 2 Anthology
  3. Tad Williams' Otherland
  4. Giles Deleuze's Difference and Repetition (ha!)
  5. Michael Chabon's The Yiddish Police Man's Union

Five Elements of My Daily Ritual That I'd like to Improve

  1. Wake Time
  2. Lunch Preparation (the night before?)
  3. Reading RSS feeds.
  4. Making more time for knitting
  5. Join a Gym to better Exercise at night.

Five Improvements I'd Like to Make to This Blog

  1. Add a wiki
  2. Build more useful archives.

3. Streamline Build Process. (Requires serious hacking. Getting #2 might help this.)

  1. Increase increase discussion quotient.

5. Provide more useful resources, and include more citations and links to the cool things that are happening on the Internet

Five Jobs I'd Like to Have Before I Retire

  1. Editor (of a fiction or non-fiction publication)
  2. Workshop leader
  3. Researcher/Professor
  4. Community Manager
  5. Cultural Critic/Industry Analyst

return of five things

Ways I've Injured Myself Recently

  1. The tip of my right index finger caring a server.

2. My left knee (recurrent, minor) dancing, probably jumping. Design flaw, I'm convinced.

3. I have some sort of strain/dislocation of the first knuckle of my right ring finger, and its oddly sore.

  1. My right shoulder, because I sleep on it funny.

5. My left wrist is occasionally stiff from typing and what not, but I think it's sort of interesting that these 3 of the five hand-related.

Things I would Change about my Cell Phone if I could.

1. I would be able to SSH into my cell phone, for the purpose of sending libnotify-esque notifications to the cell phone

2. It would have a plethora of hardware keys, potentially some sort of keyboard (on a slider) and only require touch-screen interfaces when intuitive.

  1. I would want it to run emacs, at least to be able to check on things.
  2. I would like some sort of native terminal client on the cellphone.
  3. It would be unlocked.

Objects that I would like to Combine

  1. eBook reader/music player/tablet-or-pocket computer.
  2. A tea kettle and yogurt maker.
  3. My network router and my computer.
  4. A keyboard and desk chair.
  5. The mouse (and my computer's dependence on it) and /dev/null

Things I wish I spent more time doing

  1. Knitting
  2. Reading
  3. Editing things I've written
  4. Spinning
  5. Being social

Abilities of which I am Jealous in Others

  1. Musical talent, mostly playing the violin/viola and melodian.

2. Signing and leading songs effectively (including remembering lyrics completely).

  1. The facility to function on much less sleep than I seem to require.
  2. The appetite to drink coffee without wanting to wretch.
  3. The ability to write computer programs with skill and grace.