Values for Collaborative Codebases

After, my post on what I do for work I thought it'd be good to describe the kinds of things that make software easy to work on and collaborative. Here's the list:

  1. Minimal Documentation. As a former technical writer this is sort of painful, but most programning environments (e.g. languages) have idioms and patterns that you can follow for how to organize code, run tests and build artifacts. it's ok if your project has exceptional requirements that require you to break the rules in some way, but the conventions should be obvious and the justification for rule-breaking should be plain. If you adhere to convention, you don't need as much documentation. It's paradoxical, because better documentation is supposed to facilitate accessibility, but too much documentation is sometimes an indication that things are weird and hard.
  2. Make it Easy To Run. I'd argue that the most difficult (and frustrating) part of writing software is getting it to run everywhere that you might want it to run. Writing software that runs, even does what you want on your own comptuer is relatively, easy: making it work on someone else's computer is hard. One of the big advantages of developing software that runs as web apps means that you (mostly) get to control (and limit) where the software runs. Making it possible to easily run a piece of software on any computer it might reasonably run (e.g. developer's computers, user's computers and/or production environments.) Your software itself, should be responsible for managing this environment, to the greatest extent possible. This isn't to say that you need to use containers or some such, but having packaging and install scripts that use well understood idioms and tools (e.g. requirements.txt, virtualenv, makefiles, etc.) is good.
  3. Clear Dependencies. Software is all built upon other pieces of software, and the versions of those libraries are important to record, capture, and recreate. Now it's generally a good idea to update dependencies regularly so you can take advantage of improvements from upstream providers, but unless you regularly test against multiple versions of your dependencies (you don't), and can control all of your developer and production environments totally (you can't), then you should provide a single, centralized way of describing your dependencies. Typically strategies involve: vendoring dependencies, using lockfiles (requirements.txt and similar) or build system integration tends to help organize this aspect of a project.
  4. Automated Tests. Software requires some kind of testing to ensure that it has the intended behavior, and tests that can run quickly and automatically without requiring users to exercise the software manually is absolutely essential. Tests should run quickly, and it should be possible to run a small group of tests very quickly to support iterative development on a specific area of the code. Indeed, most software development can and should be oriented toward the experience of writing tests and exercising new features with tests above pretty much everything else. The best test suites exercise the code at many levels, ranging from very small unit tests to ensure the correct behavior of the functions and methods, to higher level tests that test the functionality of higher-order functions and methods, and full integration tests that test the entire system.
  5. Continuous Integration. Continuous integration system's are tools that support development and ensure that changes to a code pass a more extensive range of tests than are readily available to developers. CI systems are also useful for automating other aspects of a project (releases, performance testing, etc.) A well maintained CI environment provide the basis for commonality for larger projects with a larger number for projects larger groups of developers, and is all but required to ensure a well supported automated test system and well managed dependency.
  6. Files and Directories Make Sense. Code is just text in files, and software is just a lot of code. Different languages and frameworks have different expectations about how code is organized, but you should be able to have a basic understanding of the software and be able to browse the files, and be able to mostly understand what components are and how they relate to each other based on directory names, and should be able to (roughly) understand what's in a file and how the files relate to eachother based on their names. In this respect, shorter files, when possible are nice, are directory structures that have limited depth (wide and shallow,) though there are expections for some programming language.
  7. Isolate Components and Provide Internal APIs. Often when we talk about APIs we talk about the interface that users access our software, but larger systems have the same need for abstractions and interfaces internally that we expose for (programmatic) users. These APIs have different forms from public ones (sometimes,) but in general:
    • Safe APIs. The APIs should be easy to use and difficult to use incorrectly. This isn't just "make your API thread safe if your users are multithreaded," but also, reduce the possibility for unintended side effects, and avoid calling conventions that are easy to mistake: effects of ordering, positional arguments, larger numbers of arguments, and complicated state management.
    • Good API Ergonomics. The ergonomics of an API is somewhat ethereal, but it's clear when an API has good ergonomics: writing code that uses the API feels "native," it's easy to look at calling code and understand what's going on, and errors that make sense and are easy to handle. It's not simply enough for an API to be safe to use, but it should be straightforward and clear.

The One Page Brainstorm

For a few years, rather than really double down on some kind of electronic note taking and task management tool, I've mostly done that kind of work long hand, using pen a paper. I'm not great at using a pen to write, but it can be fun, and I quite enjoy the opportunity to take a step back, slow down, and focus on some brain work. I don't really have a system, as such, though I did read the bullet journal website and I guess I probably stole some of those ideas.

I definitely keep a pad on my desk that's some kind of loose idea of "things I want to get done today or in the next couple of days." Its less a "todo list," in the sense that it's not really an exhaustive list of everything that I need to get done, and I don't use so that I can write things down and then forget them, but more so that when I sit down at my desk, I can more quickly find something useful to do, and that every few days when look over the previous list, I can get a sense of what I've done (or not done.)

I also, and this is perhaps more interesting, have a notebook that's just for brainstorming. I've sort of intentionally selected "half-size" (e.g. 6.5" x 8.25") notebooks for this purpose. My routine is pretty simple: make a point of sitting down with a pen and a blank sheet of paper periodically and just brainstorming or writing about a topic. It's really easy to fill a page and often easy enough to fill the facing page with a bunch of ideas about whatever topic it is, usually something I'm writing but also sometimes code or another kind of project.

I think I developed this practice when writing fiction, which I don't seem to do much of, any more, (though I've been making notes for a couple of years so I suspect it'll happen,) but the idea is less to be systematic about the notes, or to collect them in ways that will be textually useful in the future, but more as a way of focusing and putting all or most of your attention on a problem for a few minutes. Often, though, the notes are useful, both because they force you to answer questions about what you're working on and can function as a creative jumpstart, both when doing work as a thing to review and also because sometimes once you start doing the mental work moving into actual work (typically for me, typing) becomes compelling. I suppose there is almost a meditative quality to the activity.

When thinking about writing fiction, the "write a half sized page full of notes" about an anecdote from a character, or a bit of worldbuilding, or a description of a plot progression (at any scale,) makes a lot of sense and use a useful exercise.


In some ways, I suppose, this is a little bit if a follow up to my Get More Done post, as one of the things I do help begin to make progress or make sure that it's easy to focus and have a clear idea of what to do when you're ready to write more formally.

How to Become a Self-Taught Programmer

i am a self taught programmer. i don't know that i'd recommend it to anyone else there are so many different curricula and training programs that are well tested and very efficacious. for lots of historical reasons, many programmers end up being all or mostly self taught: in the early days because programming was vocational and people learned on the job, then because people learned programming on their own before entering cs programs, and more recently because the demand for programmers (and rate of change) for the kinds of programming work that are in the most demand these days. and knowing that it's possible (and potentially cheaper) to teach yourself, it seems like a tempting option.

this post, then, is a collection of suggestions, guidelines, and pointers for anyone attempting to teach themselves to program:

  • focus on learning one thing (programming language and problem domain) at a time. there are so many different things you could learn, and people who know how to program seem to have an endless knowledge of different things. knowing one set of tools and one area (e.g. "web development in javascript," or "system administration in python,") gives you the framework to expand later, and the truth is that you'll be able to learn additional things more easily once you have a framework to build upon.

  • when learning something in programming, always start with a goal. have some piece of data that you want to explore or visualize, have a set of files that you want to organize, or something that you want to accomplish. learning how to program without a goal, means that you don't end up asking the kinds of questions that you need to form the right kinds of associations.

  • programming is all about learning different things: if you end up programming for long enough you'll end up learning different languages, and being able to pick up new things is the real skill.

  • being able to clearly capture what you were thinking when you write code is basically a programming super power.

  • programming is about understanding problems [1] abstractly and building abstractions around various kinds of problems. being able break apart these problems into smaller core issues, and thinking abstractly about the problem so that you can solve both the problem in front of you and also solve it in the future are crucial skills.

  • collect questions or curiosities as you encounter them, but don't feel like you have to understand everything, and use this to guide your background reading, but don't feel like you have to hunt down the answer to every term you hear or see that you don't already know immediately. if you're pretty rigorous about going back and looking things up, you'll get a pretty good base knowledge over time.

  • always think about the users of your software as you build, at every level. even if you're building software for your own use, think about the future version of yourself that will use that software, imagine that other people might use the interfaces and functions that you write and think about what assumptions they might bring to the table. think about the output that your program, script, or function produces, and how someone would interact with that output.

  • think about the function as the fundamental building block of your software. lower level forms (i.e. statements) are required, but functions are the unit where meaning is created in the context of a program. functions, or methods depending, take input (arguments, ususally, but sometimes also an object in the case of methods) and produce some output, sometimes with some kind of side-effect. the best functions:

    • clearly indicate side-effects when possible.
    • have a mechanism for reporting on error conditions (exceptions, return values,)
    • avoid dependencies on external state, beyond what is passed as arguments.
    • are as short as possible.
    • use names that clearly describe the behavior and operations of the function.

    if programming were human language (english,) you'd strive to construct functions that were simple sentences and not paragraph's, but also more than a couple of words/phrases, and you would want these sentences to be clear to understand with limited context. if you have good functions, interfaces are more clear and easier to use, code becomes easier to read and debug, and easier to test.

  • avoid being too weird. many programmers are total nerds, and you may be too, and that's ok, but it's easier to learn how to do something if there's prior art that you can learn from and copy. on a day-to-day basis, a lot of programming work is just doing something until you get stuck and then you google for the answer. If you're doing something weird--using a programming language that's less widely used, or in a problem space that is a bit out of mainstream, it can be harder to find answers to your problems.

Notes

[1]I use the term "problem" to cover both things like "connecting two components internally" and also more broadly "satisfying a requirement for users," and programming often includes both of these kinds of work.

Quarantine Knitting Television

I've always enjoyed watching television while knitting, knitting itself is easy and doesn't take much thought most of the time, and I've never needed to watch it for every stitch, so it makes sense. I've never really felt as engaged with audio-only media while knitting for long stretches, though I do try sometimes.

My tastes, in this context, tend toward procedurals: not particularly because I love crime and/or medical shows, but because the cadence of the story telling is a good fit for my attentional needs while knitting, and the overall predictability means my attention can drift in and out as need be. It's also helpful that a lot of procedurals have had long runs which means there's a lot of material to keep me busy. Things that I've watched recently:

  • Bones has been quite fun to watch because it sort of fits the bill correctly, there are a lot of episodes. It's also very interesting because I remember watching some of the early seasons more or less when they aired, and it totally feels dated now.
  • Stargate Universe, which couldn't stick with, despite a lot of affection for earlier Stargate shows, I couldn't stick it out. I think that some combination of the show being very dark and isolating, with a few of the characters being deeply unlikeable.
  • The last few years of Star Trek TNG and Voyager. Which were nice, but it's super weird to watch television that's this episodic, and while I've watched a lot of Star Trek here and there, I must confess that I didn't really watch them systematically before.
  • Eureka is totally absurd, but was fun to watch, and was the right kind of "delightful, but mostly lighthearted."
  • White Collar has been fun. I'm particularly partial to shows set in New York City, and it's fun to be able to pick out the settings from familiar places. Sometimes the characters are great, and sometimes some of the "character growth" stories are a little bit ham-fisted. I also wonder what'd be like if there was a buddy-cop show where there characters were actually gay and not just incredibly homoerotic all the time.

I'm not super sure where to go after this, and would gladly take suggestions if people have favorites. I haven't tended to enjoy things like CSI very much, for whatever that's worth.

Knitting Gadgets

While I took a bunch of time off knitting, I didn't quite take enough time off to completely divest myself of all of my knitting things, which means when I decided rather abruptly that I wanted to start knitting again, I just had to run and pull a couple of boxes down off a shelf and I was off to the races. At the same time, after many years of small apartment living I didn't have a very large collection of gadgets or yarn.

While I don't really have a great interest in building a collection of knitting things, or yarn outside of material I have proximal use for, in the nearly 10 years, since I was a regular knitter, the state of the craft has evolved or at least changed a bit. While my collection of things hasn't changed much, the following objects

  • knitting needles, have always been complicated. I tend to work at fairly small sizes which makes needle flex/bend an issue, and my skin tends to react with nickle which rules out a lot of options.
    • at my mother's recommendation I got a few Dyak Craft knitting needles. The small-sized interchangeable needles are great, and the US 0s (which has been my primary needle) don't bend or flex at all, and have good cable/needle joins.
    • I have a small collection of 5 inch carbon fiber, double pointed needles in some small sizes and I think they're just perfect. I'm a loose knitter, so the little bit of grip that they have is great. Somehow, metal needles and six inch needles end up hurting my hands, and I've never used a pair of wooden needles that haven't broken tragically.
  • After a couple of projects where I was just breaking the yarn by hand, or using kitchen scissors, I gave in and bought a 4 inch pair of very plain Gingher embroidery scissors (for 20 bucks,) and they're brilliant for trimming threads and cutting open steeks. As a left handed human, scissors have always been something of a sore spot, and these are quite good.
  • I bought a couple of boxes of those lightbulb-shaped coil-less safety pins. I think the going rate is 6 bucks for a pack of 120, and they come in a few different colors. These are great both as stitch markers for the needle, and also to mark rows while knitting.
  • When I was going through my knitting things, I very quickly found a little cone of 8/2 mercerized cotton that I've had for years in lime green--a color I've never even gotten close to knitting with--that's really perfect for setting stitches aside or for provisional cast-ons. It's nice to use smooth, non-wool yarns for this purpose, and you don't use very much of it, but it's great to have around. While I've had this cone for 16 years or more, and it's conceivable that I may never finish it, it's great.
  • I've also given in a started storing in-progress knitting projects in draw-string canvas bags purpose built for knitting/crafting, as opposed to the vinyl bags that bed-linens come in, which had previously been my default. It's pretty essential that I be able to keep things safe from cats or the other things in my backpack (not that I leave the house much these days,) so containment is necessary, and avoiding velcro and zippers is ideal.

And that's about it!

Test Multi-Execution

Editoral Note: this is a follow up to my earlier Principles of Test Oriented Software Development post.

In software development, we write tests to make sure the code we write does what we want it to do. Great this is pretty easy to get behind.

Tests sometimes fail.

The goal, is that, most of the time when tests fail, it's because the code is broken: you fix the code and the test passes. Sometimes when test fail there's a bug in the test, it makes an assertion that can't or shouldn't be true: these are bad because they mean the test is broken, but all code has bugs, and test code can be broken so that's fine.

Ideally either pass or fail, and if a test fails it fails repeatedly, with the same error. Unfortunately, this is of course not always true, and tests can fail intermittently if they test something that can change, or the outcome of the test is impacted by some external factor like "the test passes if the processor is very fast, and the system does not have IO contention, but fails sometimes as the system slows down." Sometimes tests include (intentionally or not) some notion of "randomnesses," and fail intermittently because of this.

A test suite with intermittent failures is basically the worst. A suite that never fails isn't super valuable, because it probably builds false confidence, a test suite that always fails isn't useful because developers will ignore the results or disable the tests, but a test that fails intermittently, particularly one that fails 10 or 20 percent of the time, means that developers always will always look at the test, or just rerun the test until it passes.

There are a couple of things you can do to fix your tests:

  • write better tests: find sources of non-determinism in your test and rewrite tests to avoid these kinds of "flaky" outcomes. Sometimes this means restructuring your tests in a more "pyramid-like" structure, with more unit tests and fewer integration tests (which are likely to be less deterministic.)
  • run tests more reliably: find ways of running your test suite that produce more consistent results. This means running tests in more isolated environments, changing the amount of test parallelism, ensure that tests clean up their environment before they run, and can be as logically isolated as possible.

But it's hard to find these tests and you can end up playing wack-a-mole with dodgy tests for a long time, and the urge to just run the tests a second (or third) time to get them to pass so you can merge your change and move on with your work is tempting. This leaves:

  • run tests multiple times: so that a test doesn't pass until it passes multiple times. Many test runner's have some kind of repeated execution mode, and if you can combine with some kind of "stop executing after the first fail," then this can be reasonably efficient. Use multiple execution to force the tests to produce more reliable results rather than cover-up or exacerbates the flakiness.
  • run fewer tests: it's great to have a regression suite, but if you have unreliable tests, and you can't use the multi-execution hack to smoke out your bad tests, then running a really full matrix of tests is just going to produce more failures, which means you'll spend more of your time looking at tests, in non-systematic ways, which are unlikely to actually improve code.

Pattern Fragment 4

This is the follow up to Pattern Fragment 0, Pattern Fragment 1, Pattern Fragment 2, and Pattern Fragment 3.

Cut open the arm hole steeks, and joining new yarn at the top of the opening (after the shoulder seam), pick up stitches around the arm hole, at a rate of 2 stitches for every three rows. I picked up two extra stitches in the "corners" at the bottom of the sleeve and did not pick up a stitch from the shoulder seam.

Also, place markers where the shoulder decreases start at the bottom of the opening.

Knit 9 stitches, move yarn to front, slip the next stitch, move the yarn to the back, change directions, [1] slip the first stitch (again,) and work back to the "beginning of the row," at the shoulder seam. Work the next nine stitches, slip the next stitch, move the yarn to the front, change directions, slip the first stitch (again), move the yarn to the back, and knit across.

When you get to a "wrapped stitch" at the end of this "short row," pick up the wrap and knit it together with the stitch it wrapped, and then wrap the next stitch.

Continue in this manner until you get to the markers where the arm hole shaping ends, before switching to knitting in the round for the main body of the sleeve.

[1]You could turn the work, but I do this part just by knitting back backwards.

A Selection of Foutain Pen Recommendations

In addition to all of my other weird hobbies and practices, I have a minor fascination with and enjoyment of fountain pens, and have for a long time. After many years of not really writing very much long hand, I decided to get back to it a couple of years ago, and have amassed some pens, but perhaps more relevantly here, some opinions, which I thought I'd reflect on here.

A few notes and pieces of context first:

  • I am left handed, which I think makes me more intolerant of slow drying inks, and pens that have unexceptional writing experiences.
  • All of my long-hand writing is for myself. Aside from occasionally writing my name in the cover of a tune books, or something similar.
  • I'm color blind, and have stayed away from ink colors in the red/purple spectrum, opting for mostly blue/black, blues, and blue-green, colors (with one exception.)
  • None of these recommendations are sponsored. If you enjoy them, let me know!

So here we go.

  • I mostly enjoy using Clairefontaine notebooks, mostly wirebound, and mostly as it turns out in the A5 size. I keep a steno pad on my desk for todolists and notes, and a side-bound notebook around for longer form notes.
  • At least Somewhat to my own surprise, my favorite pens appear to be piston-filled German made pens with medium nibs. European pens tend to run more broad than average, and that's definitely true of my favorites:
    • Lamy 2000 is very modern looking and feeling, and it feels great to write with. The nib is very wet, so I've made a habit of using the quickest drying ink I have, and it's worked out well. If there's a complain, it's that the cap-seal is less than perfect if you are (as I am) an infrequent writer.
    • Pelikan m405 looks like a fountain pen, and while I suppose that's classic, and it is a very smart pen, I wasn't expecting this to be such a perfect pen. It's a joy to write with, the gold nib felt worth it, and the small size is actually an asset (I also have the larger m805, and find myself going for the smaller pen more often.)
  • Also compelling are Namiki Vanishing Point which write really well. and are just really damn cool. The VP pens are great for regular use and short-bursts of writing. As a Japanese pen, the nibs run fine, and the gold nib versions are worth it. The Decimo version is great (smaller and thinner,) but the carbon-fiber ones are pretty compelling. I use the bladder converters, and I like them, but it's a bit weird. Namiki cartrages are great as well, particularly the blue-black ones and they're particularly easy to eye-dropper, if that's your thing. Additionally, I have a Lamy Studio with a fine (steel) nib that's way more compelling.
  • Surprisingly compelling also is the TWSBI Diamond Mini and TWSBI Eco. I think I have a slight preference for the Mini over its larger version, and the Eco as well, but they're all great pens, particularly for the price, and as piston fillers, they're great if you're new to fountain pens or you want to have a color of ink available to you but not in regular rotation. The nibs are steel and you can tell, but it's not a huge detriment.
  • The most compelling inks are, to my mind: Noodler's Q-Eternity, which is a nice blue-black color and dries really fast. Having said that, it's probably still second in my mind to Diamine Twilight, which also dries quite quickly, is a very similar color, and has really compelling shading. Thankfully there's no need to pick favorites.
  • I have a short list of inks that I think are really good and worth being aware of if you're not. First Pilot's Standard Blue Black is a great ink, it dries pretty quick, the bottles are huge, and is somehow not super wet but also wet enough. I have a steel-nibbed Vanishing point pen, and this ink turned it from "meh" to compelling. I wasn't expecting to really like Diamine Oxblood which is a red-black ink, I suppose, but it's very nice. Finally, the Pilot Iroshizuku inks are very nice, and the colors are cool. I'm particularly fond of Tuski-Yo and Shin-Kai, but I suspect it's hard to go wrong.

That's it! I probably won't blog much about this, but would definitely be down to chat more about it, or pull together some posts if it's not too for you all.