Topic: Programming

So I was hoping to write you this morning from a brand new blogging system powered by jekyll that I did a little bit of tweeking on. Unfortunatly, all the pieces aren’t all in place, yet. So tychoish is still a wordpress event still on wordpress. The highlights:

• It seems to have not worked, because it’s notion of categories isn’t a bit more firm than my own notion of categories (ie. coda and tychoish), and there isn’t any clever (or not so clever) way to get around this. It’s way more geeky than normal, but basically there’s no way (that I can figure) without categories to get an array/object of some segment of the posts without using jeykll’s categories.

  The categories that jekyll has are really powerful, and my problem isn’t so much how they work, as it is that I have 1300-ish posts written “in the old way” that I need to figure something out for. I’ll do that, but it’ll take a bit longer, and it’ll mean that there’ll be a more substantive update to the site.

• I did something that I think improves jekyll, by adding an aditional permalink format to my fork. I’m not even passingly familar with Ruby (and I may have a slight professed disdain for it) and while it wasn’t a big change nor did it require a bunch of logic/smarts, it works and I think I did it “right” for whatever that means.

  I also, in the course of this, got sed to work right, which was mostly just a case of me reading the manual wrong for years and then avoiding it. But personal victories are victories all the same.

I can explain my jekyll issue/need in more detail if anyone wants to help me hack on this. Cheers!

So I was sorting through my org-mode files and I found a little snippet of code that I had lying around. I’ve had a problem with shell scripting for a while. I use bash a lot, and I’ve written a few functions, and procedural scripts that I’ve found useful in my travels, but anything that requires “logic,” often escapes me in tragic and epic ways.

Here’s the problem. I had a directory of notes that I had compiled for presentation in an ikiwiki blog for notes. As I wanted these notes to be in my org-mode system, I needed to take a directory of 60 or so files, and turn them into a single org file. As an additional bonus, I needed the files to be organized by date rather than by the alphabet. Also, as ikiwiki generated page titles from the filename, I needed to capture that information in the org-mode file. Here’s what I came up with

for i in `ls -t`;  do
   echo -e "$i" >> ~/org/legacy_notes.org &&
   cat "$i" >> ~/org/legacy_notes.org ;
done

It works as a one liner, but it works. I think previous attempts have left out crucial semi-colons or the do and done statements. No clue. Baby steps folks.

“blah blah tycho says he’s not a programmer, blah blah.” I say stuff like this (more or less) quite a bit, right? But I know a lot about how computers work, a lot about how programs are constructed (at least in the abstract,) and while I’ve not (really) written code of any note I do read a bunch of code, and can almost always figure out how things work. So I suppose I should confess that I’m… working on learning how to program (in my copious free time).

I, of course, will keep you posted about this as it progresses, but for now I’ll settle to just explain the origin of this development: I started using emacs a few months ago, and I found the lisp-dialect that serves as emacs' extension language to be quite understandable. Maybe it’s something about how emacs programs are written to be hacked on, and even poorly documented emacs-lisp is rather well documented. I read the beginning of an introduction to haskell, and I think it has something to do with the functional aspects of the language. In any case, something clicked, and so here I am.

I recently read the anti-web manifesto, which I found refreshing. If you haven’t read it, go do so. If your too lazy to read it, the gist is that we’re trying to get the web to do too much (ie. run applications, pixel-perfect layouts) and that quality browsers can’t exist, because what we use the web for these days is beyond the scope of what the web was intended to do. The document is also refreshingly snarky, in the long tradition of both hacker writing and the genre of manifestos in general, but don’t let that offend.

I’ve been known to say, “I hate the web,” which is an ironic thing to say given my line of work, but I think it’s mostly true. To be fair, I don’t hate the web, I just hate what it’s become: the only way to access what happens on the Internet. It’s great for publishing and accessing content, but for applications? Somewhat less great.

The Manifesto centers on the notion that the perfect web-browser is impossible to implement: Browsers have to implement inefficient scripting languages, and multiple implementations of the various web standards (because you have to implements both “how it should be done,” and “how the old, broken implementations that everyone wrote pages to, did it,” with the end result being that browsers themselves suck. And it’s not a case of just writing the perfect browser because, current expectations of the technology is flawed.

The course of action (theses?) are to:

• Eliminate CSS; use a little basic HTML formatting instead. Let the text stay in its natural format.
• Only basic font faces ([sans]serif, monospace), relative sizes to be supported.
• Eliminate scripting.
• Separate information from empty multimedia content: use Flash for the latter.

I’m not sure that I agree with this solution. I think HTML 5 will take care of the multimedia content, and I think flash should be avoided. I think scripting should be the first causality of the post-web Internet. I don’t see CSS as a problem, (the author sees it as a symptom of design orientation in website creation), though I’d concede that it’s used improperly most of the time.

Given this, I think four bullet points from tycho regarding “The ‘Post-Web’ Web” are in order:

• Eliminate JavaScript and all scripting in web-environments. JavaScript is the table of the ‘aughts and ‘teens.
• Develop/concentrate efforts on alternate (ie. non HTTP) protocols to facilitate the movement of dynamic information across the Internet, including well implemented clients.
• Develop robust/lightweight cross platform frameworks for developing applications on the desktop. Where’s GTK-on-Rails?
• Write a HTTP server that provides navigational meta-data automatically with pages, and a browser with the ability to construct site navigation based on this information. This way the architecture of the site depends on the file layout and a configured file, but is generated locally. Basically gopher, except designed in the casual manner of the ‘aughts.

Any takers?

A few weeks ago, a friend told me about this IDE (integrated development environment) called PIDA, and while I’m nothing more than a casual programmer, my interest was immediately piqued.

We’ll call it part of my interest in how programmers work. The truth is, as a text-editor junkie, my impulse is to say “meh, IDEs are passe,” but I think that’s probably unfair, and in my writing IDE post I think I recognized--by analogy--their worth. In any-case there’s something sort of nifty about PIDA: rather than recreate tools, it just makes use of what’s already there: the editor is emacs (or vim), the version control system is whatever you want (and already use). PIDA just brings all these things together in a nifty little package.

On the one hand, it’s not a really big deal. Cobbling together working software from a bunch of different existing tools isn’t particularly new. This is sort of the basis of unix-like computing, and further more it tracks the ways most people/geeks actually use computers: by finding the best tools for all of the jobs they have to do and then using them. This way of interacting seems to hold true for command-line and graphical users, I think. So rather than recreate the wheel, PIDA just uses all the existing wheels. The saddest part is that we don’t see more things like this in the graphical application world.

The end result of this mode of application development is that we’re given/build powerful tools that function in familiar ways and that are more powerful as a result of their integrations.

And that’s about it.

I initially thought that this was going to be a really long and really blathering post about integrated tools, and the power of open source/free software to allow tools to be combined rather than be forced to compete. While these are indeed important issues, they’re pretty self explanatory, and IDEs like PIDA provide a great example of how this can be the case, so much so that I find myself saying “why aren’t there more programs like this?”

Why not indeed?

There’s this idea in linguistic/cognitive anthropology that the limitations of linguistic possibility limit what the bounds of what we’re able to think about. If we lack words for a given thing or a concept, it’s really hard to even conceive what it is. I’ll get to the strengths and limits of the hypothesis in a bit, but lets just say the reception of these ideas (i.e. “linguistic relativism”) is somewhat mixed. Nevertheless it’s had a great impact on me and the kinds of ideas I deal in.

For instance, though I’m not an active programmer, I talk to programmers a bunch I tweak code from time to time, and I’ve tried to learn programming enough times that I sort of get the basics of enough stuff to know what’s going on, and if there’s one theme to my interests in open source and software development, it’s looking at the software and tools that developers us, in part for issues related to linguistic relativism. Basically, if the people who develop programming languages, and software itself don’t provide for possibilities, developers and users won’t be able to think about things downstream. Or at least that’s the theory.*

The problem with linguistic relativism in general, is that it’s really hard to test, and we get into causality issues. “Describe this thing that you don’t know about!” is a bad interview tactic and we run into the questions like: Is it really language that limits knowability or is some other combination of typical experiences that limits both knowability and language? I’ve read far too many a number of papers from a couple of different scholars, and I almost always end up in the “relativist camp,” but that might be a personality feature.

In computer science, I suppose it is also not quite so cut and dry. Questions like “Does the advancement of things like hardware limit technical possibility more than programing languages and tools?” come up, but I think for the most part it is more cut and dry: Erlang’s concurrency model makes thins possible, and makes programmers think in ways that they’re not prone to thinking about them otherwise. Git’s method promoting collaboration requires people to think differently about authorship and collaboration. Maybe. I mean it makes sense to me.

These low-level tools shape what’s possible on the higher level not simply in that a programing language implements features that are then used to build higher level applications, but if you teach someone to program in emacs-lisp (say) they’ll think about building software in a very different way from the folks who learn how to program Java. Or Perl. Or PHP.

And those differences work down the software food chain: what programmers are able to code, limits the software that people like you and me use on a day to day basis. That’s terribly important.

I think the impulse when talking about open source and free software is to talk about subjects that are (more) digestible to non-technical users, and provide examples of software projects that are more easily understood (e.g. firefox and open office rather than gcc and the Linux kernel, say.) This strikes me as the wrong impulse, when we could focus on talking about more technical projects and then use abstractions and metaphors to get to a more general audience if needed. I’m not saying I’ve mastered this, but I’m trying, and I think we’ll ultimately learn a lot more this way. Or so I hope. There is always much to learn.

Onward and Upward!

* In a previous era/maturity state I would have been somewhat guarded about that subject which I think is going to be a major theme in oh the rest of my life. But, this is open source and knowledge wants to be free and all that. Actually, less altruistically, I’m much more worried that it’s a lackluster research question than I am that someone is going to “snipe it from me,” and suggestions and challenges I think would be really productive.

My post about information buckets got me thinking, thanks to some other things that have filtered through my awareness in the last few weeks, about integrated development environments.

See, a couple years ago, I used everything buckets like mad, they were the application of the future, the software that seemed to just scratch every itch. I don’t know that I used them as general databases for everything but I did all of my writing and research-related tasks in them, and it was great.

The articles I read where there, along with the notes that I made, along with citation information, along with the final paper. Along with all of my previous papers and articles. I never had to go look for things again, it was all there.

This is similar to an IDE, or integrated development environment, which is a class of applications that have some measure of popularity amongst computer programmers. IDE’s generally (and I’m not clear on this, as I’ve never used one) provide tools for managing large projects and integrate with code-compilers, debugging tools, version control tools, multi-file project managers, and code browsers which allow you to find related parts of code across an entire project without scrolling through a lot of files. I think there’s probably some debate on weather these are good things for programmers (objections would include that they make for lazy/poor practices, and that IDE components do lots of things poorly) but generally I suspect they are good things. And as it turns out most of our beloved text-editors have a lot of IDE-like functionality anyway.

In a way, the information bucket provides an IDE-like environment for the information worker or writer. And that’s sort of a cool way of looking at this. Indeed, I think writers need better integrated environments, there are a lot of tools that I think we could benefit from. While I’ve been working on these posts, I’ve also been working on posts about the emacs org-mode, and while they’re ultimately not directly connected, I think something like org-mode is a great boon to writers and information workers. Integrated systems that manage citations, references, and notes are essential, as well as other helpful features that might include managing output, structure, and version control.

I hack most of these features together in emacs with some help from LaTeX, git, and BibTeX, but I recognize that this isn’t ideal or for everyone. There are other applications that aren’t “information buckets,” but provide writing environments that aren’t hacked from programing tools (emacs) or desktop publishing tools (MS Word). For example: Scrivener that provides a clever way to write longer form documents with structure, and Celtex that provides all sorts of screenplay writing tools (outlines, storyboards, character databases) and script management tools. Celtex even calls itself an “Integrated Media Pre-production” tool.

I’m not sure that these are the ideal tools for this task. Any of them. I’m partial to my solution, but I end up having to do a lot of informal organization to make it all stick together. So I wonder, what kind of software do other writers to keep all their ducks in a row?

More later. Of course.