Wikileaks, and the drama that has surrounded it for the past few months, brings forth images of the Internet as a very lawless and juvenile place, exactly the kind of thing that the cyberpunks of the 1980s were predicting. This isn’t always far from the truth, but the story of spies and international espionage, and digital attacks and counter attacks may distract us from thinking about other issues. Obviously Wikileaks causes us think about censorship, but issue of publishing, journalism, audience and community participation, transparency, and globalism in the digital context are also at play. Lets take the highlights:

Censorship

In the print world, I tend to think of post-facto censorship is incredibly difficult. Once print copies of something exist they’re there and it’s hard to get every copy and you can’t get people to “unread” what they’ve already seen. In the digital world, it’s really difficult to get content taken down, and once there are copies in people’s hands, the cost of making additional copies is low enough that censorship stops working. Right?

I suppose the appearance of multiple mirrors and copies of Wikileaks post-takedown proves this, but it also proves that there are aspects of the world wide web that are not decentralized and it’s possible to pull a domain and site off the Internet at a single point. That’s a very scary proposition. While information survives, I think many people thought that “you couldn’t effectively censor the Internet,” and Wikileaks says “yes you can, and here’s how.” (cite)

In response I think people have started to think about the shape and structure of the network itself. The Internet is designed to be resilient to this kind of thing, and this is a very startling example that it’s not.

Publishing, Journalism and Wikis

I suppose the thing that I get most offended by “Wikileaks” for is the appropriation of the term “Wiki,” because in the current (or last) form, Wikileaks wasn’t a wiki. Content was not edited or supervised by a community, the Wiki process didn’t allow the site to provide consumers with a more diverse set of opinions, it didn’t increase transparency. In this respect, Wikileaks mostly resembles a traditional “old media,” publication.

Once wikileaks stops being this radical new journalistic departure, and this community-driven site, what remains may be pretty difficult to reconcile. Is it useful for journalists to publish raw source material without analysis? What audience and purpose does that serve? The censorship of Wikileaks is problematic and requires some reflection, but there are problems with wikileaks itself that also require some serious thinking.

And just because it’s a sore point, particularly since Wikileaks is/was a traditional publication and is not a platform for independent speech, we have to think about this as “freedom of the press” issue rather than a “freedom of speech” issue.

Community Involvement

The involvement of a community in Wikileaks, is over-shadowed by the groundswell of “community” activity by Anonymous and other groups. Look to Gabriella Coleman (twitter) for more thorough analysis. I don’t have any answers or conclusions but: the role and effectiveness of (distributed) denial of service attacks in this instance is really quite important.

Usually DoSes are quick, messy and easily dealt with affairs: DoS someone, get noticed, target and attacking addresses get taken off the air and people loose interest and things return to normal. This back and forth, seems a bit unique (but not unheard of) the fact that the 4chan/Anon gang picked Wikileaks jives with their ethos, but it is impressive that they were able to get organized to support Wikileaks. I find myself curious and more surprised that someone was able to, at least for a while, throw something in the neighborhood of 10 gigabits (an unverified number, that originates with Wikileaks itself, so potentially inflated) at the original Wikileaks site. That’s huge, and I think largely unexplored.

Transparent Operations

In July, Quinn Norton wrote about transparency and wikileaks, basically, that exposing information doesn’t solve the problem that governments don’t operate in a transparent manner, and that access to documents and transparency are the result of a more open way of doing business/government. Particularly in light of this, the fact that wikileaks focuses on data dumps rather than more curated collections of information or actual analysis, is all the more problematic.

Similarly, “wiki” as practiced in the original model (as opposed to wikileaks,) is about editing and document creation in an open and transparent manner. Thus the issue with wikileaks is not that they have/had a professional staff, but that they didn’t disclose their process.

Onward and Upward!

I’m not a programmer. I find myself saying this with great frequency. Which is weird, because I end up writing a fair amount of content that looks like (and probably is) code. Almost always this takes the forms of little shell scripts that do the things that I would other wise have to do by hand.

Very early on in my development as a programmer and Linux user, bear told me that the greatest thing about the shell was that once you figured something out, you could save it and reuse it whenever you needed it, and you never had to figure out that thing again. That’s sort of the basis of how I live my life with computers these days.

And it works great, and it means that I’m pretty comfortable with a lot of really powerful shell tools. I’m personally amazed by everything that I can make sed do. The problem is that there are limitations to what one can do with shell programing (nothing for the web, and no way to edge yourself into object orientation) or other more advanced programming techniques. Though certainly there are a lot of things that I could do using the shell, I sort of know enough to know the bounds of what’s possible and what’s absurd.

So I’ve been trying to teach myself Python. Because it seems like a good option. Though I’m not a coder by any stretch, I read a bunch of code, and know enough about all of the various languages to appreciated Python and it’s possibilities. Seems like a good neutral option and I’m pretty sure it can do everything I want or need it to for a while, and the worst case scenario is that the skills will be transferable if I need to change later.

My first “project,” I think is to rewrite build novel to Python. I get what needs to happen, and it mostly just mangles some text files into a number of other files to keep source and publication versions of Knowing Mars in step with each other. The hope being that the script will be a bit more resilient, and also be more easily generalized to publishing other projects as I have them to publish.

I’ve gotten the hang of mangling variables and reading settings from configuration (YAML rocks, and is killer easy to use.) Next up, learning more about creating, writing, and copying files, with a likely side course in regular expressions for Python. I’ll keep you posted.

The Price of Free

Eventually, the Facebooks and Twitters of the world may be dethroned and replaced by provider-agnostic protocols, in much the same way AOL is no longer synonymous with email. However, no one is going to build protocols that threaten their bottom line. So long as “free” is paid for by surveillance, the Internet will represent a Faustian bargain for radical social movements.

-- The Price of Free

I’m doing my commute somewhat differently this morning. Same wake time, same destination, and many of the same trains, but a special event necessitated a different pattern. It’s amazing how much this shift has thrown me out of sync with my day. Not in a bad way, but in the way that I was sort of jittery about missing my train in a way that my normal patter has really ceased to evoke.

I told my fellow commuter that, while I could probably live without the time expenditure of the commute I’d grown fond of the rhythm and structure of the commute. It’s nice to have my body in programmed to get up at a specific time, it’s nice to have a fixed departure time to encourage me to get work done early and effectively, it’s even nice to have some firewalled time on the train to get things done like emails and blogging. Obviously if it was possible to change some things, I would, but all in all its not as bad as it could be or as it may sound.

The main take away lesson from this is: that habits and patterns make it easier for us to cope with the world. And not just in a “habits make stressful situations, like commutes and compulsions easier to live,” but having a routine established make it easier to spend your available time and brain cycles doing things you really care about rather than on keeping your head above water.

Also, I think as a secondary point, being able to take a block of available time and accomplish something concrete is an incredibly valuable skill. One of my biggest frustrations is with “dead time” where I’m waiting for something to happen, and I can’t do anything with that time. Some technology helps with this: long battery life and systems that suspend/resume quickly and preserve “state” are great. Having a list of things that you can do that are easy to pick up and for a number of different situations, with different amounts of free time. Ideally, you don’t want to have to spend time that you could be doing something, on thinking about what you ought to be doing.

Which I suppose is just another way of approaching habits.

There’s something about public transit, particularly train systems, that seeps into the riders. Little facts that make getting around easier. “You’re waiting for the R3? It’s always late,” they’ll say, or they’ll be able to tell you which stop is next by looking out the window in what looks like total blackness. Routines do that to you, I guess.

I was on a train last week, not my usual evening train--which is a short turned local train that ends one stop past mine--that only has two intermediate stops before my stop, and as we were getting of the train, a group of regular shad a conversation that resembled the following. I suppose I should also preface this by saying that we were all sitting near a person who discussed their health insurance coverage woes at great depth and volume with their seatmates.

“It’s nice to take these express trains, I usually just get on whatever’s next.”

“If this counts as express.”

“I mean, it’s better than the ones that stop at--”

[unison] “--North Broad.”

See, there’s this stop in northern Philadelphia that a lot of trans pass, but very few stop at, and hardly anyone gets on or off at. It’s near a subway stop too, which decreases it’s usage. It’s main purpose, I think, is to irritate those of us who *just want to get where we’re going.

It’s also, apparently, good for allowing irritated and tired commuters to commiserate over the unreclaimable minutes of their lives that they loose on a regular basis. It’s these kinds of interactions and phenomena that make urban living so attractive.

git is a version control system that does a number of things rather “differently.” By now, I suspect most people are familiar with git so I will refrain from focusing too much time here on explaining what git is or how it works. Git does things differently, and to make a long story short, git has no concept of a file rename. To be honest git doesn’t have a lot of sense of individual files at all, but that’s another story. As a result git looks at the contents of the files, and if a new file is similar enough to an old file that was deleted git recognizes this as a “rename,” or a moved filed. It can process file copies in the same manner.

You’d think this would be a problem, but it turns out that it isn’t. In fact it’s a great thing. You can store a bunch of data on a file system, use conventional tools to manipulate, move, and rename those files in the normal manner, and then do a little magic, and git automatically knows everything that it needs to know about what files were renamed and when.

This post is about that magic.

What you need to do is find out the names of the files that existed in the last commit but have been removed from the file system. If you run “git rm” on these files and then add everything that remains, git will be able to pick up the rename operations implicitly, and the next commit will reflect the current state of the file system.

I’ve thrown this little snippet in the code section of the site: `git-readd </code/git-readd>`_.

I’ve used this process as part of scripts in the past, and it’s great for managing Maildirs with git, but I found myself needing this operation from time to time as I’m manging a git repository. Enter, “git aliases.”

Aliases let you create git sub commands from your own scripts. Add the following lines to the ~/.gitconfig file on your system, and create it if necessary:

[alias]
readd = "~/scripts/git-readd"

Now you can run this command in any git repository on your system with the following command:

git readd

Magic!

See “Why The World Is Ready For Dexy” and “Dexy and Literate Documentation” as well as the technical writing section section of the tychoish wiki for some background to this post.

The brief synopsis: dexy is a new tool for handling the process of the documentation work flow between writing and publication. It takes snippets of code, and bits of documentation and passes these atomic chunks through filters to generate technical articles, manuals, and other high quality resources. It’s exciting because it may provide a way to write and manage the documentation process in a manner that is more effective than many other options and has the potential to produce better quality technical texts.

The reason, I think, is that dexy treats documentation like code. This is different, fundamentally, from systems that expect that developers write documentation. The former has interesting implications about the way technical writers work, and the later is nearly always a foolhardy proposition doomed to end in failure.

Documentation has a lot in common with code: documentation is written and revised in response to software versions, so the process of iterations has a lot in common. Documentation should typically be compiled, and the build process should produce a static product, between iterations. Documentation, like code, must also be maintained and fixed in response to new bugs and use-cases as they are found/developed.

If we accept this analogy, Dexy begins to look more like a tool like make which is used to manage compilation of code. make figures out what source files have changed, and what needs to be rebuilt in order to produce some sort of binary code. That doesn’t sound like a very important tool, but it is. make makes it easy to automate tasks with dependencies, without writing a bunch of novel code to check to see what’s been done and what has yet to be done, particularly when build processes need to be done in parallel. Furthermore, make is one of these typical “UNIX-like” utilities that does only one thing (but does it very well) and ties together functionality from a number of different kinds of programs (compilers, configuration tools, etc.)

Dexy is kind of like that. It manages a build process. It ties together a group of existing tools, thereby saving developer time and building something that can be more flexible and consistent.

This is, however, imperfect analogy: I think Dexy isn’t “make for documentation,” because it would be possible to use make to manage the build process for documentation as well as code.¹ Dexy manages text processing, make can work one level up--if needed--to build coherent texts from Dexy-processed documentation units. Dexy and make are glue that turns documentation and code into useful resources.

There are obviously some situations where this developer-like workflow may be overly complicated. For starters, Dexy, like make, really provides a framework for building documents. A portion of creating every project and text in this manner would necessarily go to developing build-related infrastructure. It’s not a huge burden, but it’s the kind of thing that requires a little bit of thought, and maybe some good “default” or base configuration for new projects and texts. Dexy is a great first step into a new way of thinking about and working with documentation, but there is much work yet to be done.

Onward and Upward!

When Oracle acquired Sun Microsystems, there was a lot of (warranted) concern for what would happen to the Open Source projects that Sun started, supported, and championed. Though the acquisition began years ago, now, I think we’re still in the process of figuring out how all of this filters down. The concern, largely grows from the fact that while Sun licensed a lot of work under free and permissive terms, and was an active contributor to communities, Sun (and now Oracle) remains the ultimate owner of that work moving forward. Thus, Free Software/Open Source projects are somewhat stranded because the leadership of these projects were supported by Sun, are supported by no one.

In response, there’s been a lot of movement among these projects to formally establish “rule-by-community” systems, that give community contributors authority and voice in the management of these projects.

Nevertheless, it feels a lot like writing policy to save oneself from the mistakes of the past. It also seems like an action of democracy fetish rather than bottom up democratic organization. I’m always struck by how not democratic open source projects are, in practice, and how little this matters. Python and Perl are iconic examples of this, but most other projects have one or two people who are in charge of everything, and while they know that their “rule” is subject to the confidence of the community, their leadership isn’t terribly democratic. I think this recent event in the org-mode community is a particularly useful example.

Maybe Sun was inflating an Open Source bubble of sorts, maybe corporate-sponsored open source, and community/consultancy-sponsored open source are different things that need different sorts of approaches.