When I started doing this website thing on the eve of the millennium, the burgening buzzword of the time was “web standards.” All of us in the know were working on learning and then writing to web standards like HTML 4.0 and eventually XHTML 1.0 along with CSS 1 and 2. And we were all hankering for browsers that implemented these standards in a consistent way.

Really all we wanted was for our web pages to look the same no matter who was viewing the page.

This pretty much never happened. Web browsers are pretty good these days, or they at least--in many ways--don’t suck as much as they used to, but they’re all a bit quirky and they all render things a bit differently from each other. And on top of that they’ve got poor architectures, so as programs they’re really bloated, and prone to crashing and the like. I’ve written before about being “against” websites, webapps, and the like and I think my disdain for the “web” grows out of the plan and simple fact that:

the web browser is broken, beyond repair.

So where does this put the cause of web standards in web design? Thoughts and questions:

Do we write to standards which aren’t going to get adopted usefully? Is ad hearing to standards a productive use of time?

Do we write to clients (specific browser implementations) that are broken, but at least assure that content looks “right?”

When the previous goals two goals aren’t compatible which wins?

Will HTML 5 and CSS 4 fix these problems, or is it another moving target that browsers won’t adopt for another 10 years, and even then only haphazardly?

Are there other methods of networked content delivery that bypass the browser that might succeed while the browser space (and the content delivered therein) continues to flounder? I’m thinking object/document databases with structured bidirectional, and limited hierarchy (in the system, objects might have internal hierarchy)?

Is the goal/standard of pixel-perfect layout rendering something which the browser is incapable of providing? Might it be the case that CSS is simply too capable of addressing problems which are outside of the ideal scope for defining a consistent style for a page: Let me run with this idea for a moment:

Maybe the problem with XHTML and CSS isn’t that it’s implemented poorly, but rather that we’re trying to use CSS classes and IDs and div tags in an attempt to make pixel-perfect renderings of pages, which is really beyond CSS’s “mission.” What would web standards and the state of the browser look like, if you dropped CSS IDs (eg. #id-name{ }) and made single instance classes (eg. .class-name{}) verboten? Aside from crashing and burning and completely killing off browser-based applications?

I look forward to hearing your thoughts on the subject.

I think I’ve touched on this question before but with the last technology as infrastructure post it seems like another opportunity to talk about the intersections between this topic--thinking about technology as infrastructure--and about the sort of small scale/cooperative economics that I was writing a lot about a couple of months back.

The question on my mind at the moment is, “What do the business models of technology firms look like, in a software-freedom-loving, non-corporate/cooperative-business way?”

And I’m not sure what the answers to this question are. Not really. I’ve been thinking about business models for the producers of software/technology services during earlier moments.

We have the example of the 70s and 80s when the prevailing technology companies were ATT and IBM. ATT made their money selling phone service, and licensing UNIX. IBM made their money selling mainframes. In the eighties and nineties we had the prevailing Microsoft lead “proprietary software licensing” business models, where consumers paid for the legal write to run code on their computers.

In the nineties and early naughties the successful business models were either from people buying hardware (ie. Sun Microsystems and IBM) or people buying support for operating systems (ie. RedHat). We’ve also seen some more stable business models centered around subscriptions-for-services (this seems to be what all the successful startups are doing), and more of the time honored selling hardware, and there are some support-services based companies that remain successful (eg. RedHat), the support market consolidated a lot recently. And it’s not like the Microsoft-consumer model doesn’t still exist.

So when we look at “infrastructural technology” it sure looks like there are some kinds of businesses that will continue to flourish:

• The people making mainframes/servers and the high level computing systems that provide the infrastructure.
• The people who provide the tools that make low level tools successful and useful to users. (eg. What UbuntuOne provides on top of SSH and rsync; What gmail provides on top of IMAP; What MobileMe provides ontop of WebDAV/CalDAV/IMAP).

These strike me as rather conventional business models, given the history. Does infrastructural computing also:

• further the development of subscription-based businesses?
• create a new kind of challenge in customizing solutions for organizations and groups that translate raw resources into “finished output?” Is this too much like IMAP --> Gmail?
• [other possibilities created by you, here]

I’m trying to approach this by asking myself “what creates value in this market,” rather than “where’s the money.” It strikes me that value exists in making systems “work” in a way that’s customized to the task at hand. It strikes me that value is created when individuals and organizations are able to take ownership of their own data and computing. Gmail is valuable, but running my own IMAP server is more valuable. Running my own IMAP server without the fuss of needing to personally manage the hardware and software of the server is even more valuable.

What else does the hive mind have for us?

Continued from, Technology as Infrastructure, Act Two.

Act Three

All my discussions of “technology as infrastructure” thus far have been fairly high level. Discussions of particular business strategies of major players (eg. google and amazon), discussions approaches to “the cloud,” and so forth. As is my way, however, I’ve noticed that the obvious missing piece of this puzzle is how users--like you and me--are going to use the cloud. How thinking about technology as infrastructure changes the way we interact with our technology, and other related issues.

One of my introductory interludes was a new use-case that I’ve developed for myself: I run my chat clients on a server, and then using GNU screen which is an incredibly powerful, clever, and impossible to describe application. I’ve written about it before, but lets just describe it’s functionality as such:

Screen allows users to begin a persistent (terminal/shell/console) session on one computer, and then “detatch” and continue that session on another machine where the session runs--virtually--indistinguishable from “native sessions.”

So my chat programs are running on a server “inside of” a screen session and when I want to talk to someone, I trigger something on my local machine that connects to that screen session, and a second later, the program is up and running just as I left it.

Screen can of course, be used locally (and I do use it in this mode every waking moment of my day) but there’s something fundamentally different about how this specific use case affects the way I think about my connection.

This is just one, and one very geeky, example of what infrastructural computing--the cloud--is all about. We (I) can talk till we’re (I’m) blue in the face, but I think the interesting questions arise not from thinking about how the infrastructure and the software will develop, but rather from thinking about what this means to people on the ground.

At a(n apparently) crucial moment in the development of “the cloud” my personal technological consumption went from “quirky but popular and mainstream” to fiercely independent, hackerish, and free-software-based. As a result, my examples in this area may not be concretely helpful in figuring out the path of things to come.

I guess the best I can do, at the moment is to pose a series of questions, and we’ll discuss the answers, if they seem apparent in comments:

• Does “the cloud” provide more--on any meaningful way--than a backup service? It seems like the key functionality that cloud services provide is hosting for things like email and documents, that is more reliable than saving and managing backups for the ordinary consumer>
• Is there functionality in standards and conventions that are underutilized in desktop computing that infrastructural approaches could take advantage without building proprietary layers on-top of java-script and HTTP?
• Is it more effective to teach casual user advanced computing techniques (ie. using SSH) or to develop solutions that make advanced infrastructural computing easier for casual users (ie. front ends for git, more effective remote-desktop services).
• Is it more effective for connections to “the cloud” to be baked into current applications (more or less the current approach) or to bake connections to the cloud into the operating system (eg. mounting infrastructural resources as file systems)
• Is the browser indeed the prevailing modality, or simply the most convenient tool for network interaction.
• Do we have enough conceptual experience with using technology to collaborate (eg. wikis, source control systems like git, email) to be able to leverage the potential of the cloud, in ways that reduce total workloads rather than increase said workloads?
• Does infrastructural computing grow out of the problem of limited computing power (we might call this “vertical complexity”) or a management problem of computing resources in multiple contexts (eg. work, home, laptop, desktop, cellphone; we might call this “horizontal complexity”) And does this affect the kind of solutions that we are able to think about and use?

Perhaps the last question isn’t quite user-centric, but I think it leads to a lot of interesting solutions about possible technologies. In a lot of ways the most useful “cloud” tool that I use, is Google’s Blackberry sync tool which keeps my calendar and address book synced (perfectly! so much that I don’t even notice) between my computer, the phone, and the web. Git, for me solves the horizontal problem. I’m not sure that there are many “vertical problems,” other than search and data filtering, but it’s going to be interesting to think about.

In any case, I look forward to discussing the answers and implications of these issues with you all, so if you’re feeling shy, don’t, and leave a comment.

Cheers!

Continued from, Technology as Infrastructure, Act One.

Act Two

Cnet’s Matt Assay covering this post by RedMonk’s Stephen O’Grady suggests that an “open source cloud” is unlikely because superstructure (hardware/concrete power) matters more than infrastructure (software)--though in IT “infrastructure” means something different, so go read Stephen’s article.

It’s my understanding that, in a manner of speaking, open source has already “won” this game. Though google’s code is proprietary, it runs on a Linux/java-script/python platform. Amazon’s “cloud” (EC2) runs on Xen (the open source virtualization platform) and nearly all of the operating system choices are linux based. (Solaris and Windows, are options).

I guess the question of “what cloud” would seem trite at this point, but I think clarifying “which cloud” is crucial at this point, particularly with regards to openness. There seem to be several:

• Cloud infrastructure. Web servers, hosting, email servers. Traditionally these are things an institution ran their own servers for, these days that same institution might run their servers on some sort of virtualized hardware for which there are many providers.

  How open? Open. There are certainly proprietary virtualization tools (VMware, windows-whatever, etc.), and you can vitalize windows, and I suppose HP-UX and AIX are getting virtualized as well. But Linux-based operating systems are likely virtualized at astonishing rates compared to non-open source OSes. And much of the server infrastructure (sendmail, postfix/exim, Apache, etc.) is open source at some point.

  In point of fact, this cloud is more or less the way it’s always been and is, I’d argue, open-source’s “home turf.”

• Cloud applications: consumer. This would be stuff like Gmail, flickr, wikipedia, twitter, facebok, ubuntuONE, googe docs, google wave, and other “application services” targeted at non-commercial/enterprise consumers and very small groups of people. This cloud consists of entirely software, provided as services and is largely dominated by google, and other big players (Microsoft, yahoo, etc.)

  How open? Not very. This space looks very much like the desktop computing world looked in the mid-90s. Very proprietary, very closed, the alternatives are pretty primitive, and have a hard time doing anything but throwing rocks at the feet of the giant (google.)

• Cloud applications: enterprise. This would be things like SalesForce (a software-as-a-service CRM tool.) and other SaaS application. I suppose google-apps-for-domains falls under this category, as does pretty much anything that uses the term SaaS.

  How open? Not very. SaaS is basically Proprietary Software: The Next Generation as the business model is based on the exclusivity of rights over the source code. At the same time, in most sectors there are viable open source projects that are competing with the proprietary options: SugarCRM, Horde, Squirrel Mail, etc.

• Cloud services: enterprise. This is what act one covered or eluded to, but generally this covers things like PBX systems, all the stuff that runs corporate intranets, groupware applications (some of which are open source), collaboration tools, internal issue tracking systems, shared storage systems.

  How open? Reasonably open. Certainly there’s a lot of variance here, but for the most part, but Asterisk for PBX-stuff, there are a number of open source groupware applications. Jira/perforce/bitkeeper aren’t open source, but Trac/SVN/git are. The samba project kills in this area and is a drop in replacement for Microsoft’s file-sharing systems.

The relationship, between open source and “the cloud,” thus, depends a lot on what you’re talking about. I guess this means there needs to be an “act three,” to cover specific user strategies. Because, regardless of which cloud you use, your freedom has more to do with practice than it does with some inherent capability of the software stack.

Act One

This post is inspired by three converging observations:

1. Matt posted a comment to a previous post: that read:

“Cloud” computing. Seriously. Do we really want to give up that much control over our computing? In the dystopian future celebrated by many tech bloggers, computers will be locked down appliances, and we will rely on big companies to deliver services to us.

2. A number of podcasts that I listened to while I drove to New Jersey produced/hosted/etc. by Michael Cote for RedMonk that discussed current events and trends in “Enterprise-grade Information Technology,” which is a world, that I’m only beginning to scratch the surface of.

3. Because my Internet connection at home is somewhat spotty, and because it makes sense have an always on (and mobile) connection to IRC for work, I’ve started running my chat clients (mcabber and irssi) inside of a gnu screen session on my server.

My specific responses:

1. Matt’s right, from a certain perspective. There’s a lot of buzz-word-heavy, venture capital driven, consumer targeted “cloud computing tools” which seem to be all about getting people to use web-based “applications,” and give up autonomy in exchange for data that may be more available to us because it’s stored on someones network.

Really, however, I think this isn’t so much a problem with “networked computing,” as it is with both existing business models for information technology, and an example of the worst kind of cloud computing. And I’m using Matt’s statement as a bit of a straw man, as a lot of the things that I’m including under the general heading of “cloud computing,” aren’t really what Matt’s talking about above.

At the same time I think there is the cloud that Matt refers to: the Google/Microsoft/Startup/Ubuntu One/etc. cloud, and then there’s all the rest of distributed/networked/infrastructural computing which isn’t new or sexy, but I think is really the same as the rest of the cloud.

2. The “enterprise” world thinks about computers in a much different way than I ever do. Sometimes this is frustrating: the tendrils of proprietary software are strongest here, and enterprise folks care way too much about Java. In other aspects it’s really fascinating, because technology becomes an infrastructural resource, rather than a concrete tool which accomplishes a specific task.

Enterprise hardware and software exists to provide large corporate institutions the tools to manage large amounts of data/projects/data/communications/etc.

This is, I think on some level, the real cloud. This “technology-as-infrastructure” thing.

3. In an elaboration of the above, I outsourced a chunk of my computing to “the cloud.” I could run those applications locally, and I haven’t given up that possibility, but one needs a network connection to use a chat client, so the realm of possibilities where I would want to connect to a chat server, but wouldn’t be able to connect to my server, is next to impossible (particularly because some of the chat servers run on my hardware.).

I guess the point I’m driving at is: maybe this “cloud thing” isn’t about functionality, or websites, or software, or business models, but rather about the evolution of our computing needs from providing a set of tools and localized resources to providing infrastructure.

And that the shift isn’t so much about the technology: in point of fact running a terminal application in a screen session over SSH isn’t a cutting edge technology by any means, but rather about how we use the technology to support what it is we do.

Or am I totally off my rocker here?

In light of my otherwise fried state of mind, I would like to present a list of things that I dislike. Because I’m snarky like that.

• HTML emails. I’ve yet to send or receive an email that really requires rich text formatting provided by HTML emails. While multi-part emails (which send multiple copies of the same email in rich and plain text) are a good thing, it’s a huge pain in the ass to get an email (particularly a long email) three times, just for the pleasure.
• Sites that recreate twitter without adding any useful features or discussion. It’s as if the dimwitted internet people said “holy shit, if we give people 140 characters to say banal things on our site maybe we’ll get traffic like twitter,” except this isn’t how the internet has ever worked (or worked well.)

Facebook is coming out with “usernames,” I’ve gotten an invitation to microblog on niche-social networking site, and everyone seems hard set on reimplementing this whole “status” thing a la twitter in the beginning, without any thought of interpretation (a la laconica) or doing something cool like jaiku-style threads, let alone the next big thing.

• Malformed emails. Dudes. Sending a plain text email is really simple, there’s no excuse for it to look like your cat took a nap on the tab key. I’m not chiding anyone for neglecting to test every email “blast” they send (because I’d be that lazy) but I am chiding folks for not testing it once. Writing a text file and sending it isn’t that hard.
• Reimplementation of email. I really hate getting facebook messages, and direct messages on [microblogging service], and each and every other fucking social networking site. Just send me email. Real email. It works, I have a good process for dealing with it, and I don’t have to screw around with anything. Thanks.
• The Twitter Fail Whale. Dudes. There was a while about a year ago, when a bunch of geeks were sitting around and thinking, “you know this twitter technology is going to be really cool, and there are a lot of possibilities here,” and there were, and I suppose there still are, but the truth is that I see the fail whale several times every day, and most of the cool things that I wanted to see in twitter two years ago and then a year ago (real xmpp support, track, federation, custom filtered feeds (a la LJ-style friends' filters),) still haven’t materialized. I think the addition of OAuth is a great thing, but it’s a baby step.
• The continued prevalence of IRC. Dudes discover jabber/xmpp. Thanks. A while back, I had a lot of nostalgia for IRC, and its true that IRC has a lot of history and is a standard to be reckoned with, but jabber is so much more elegant, secure, and provides features (persistence, logging, independence, etc) without having net-splits and complicated ad hoc registration schemes.

That’s all for now. What do you hate about the internet?

A post about the distributed version control system “git” in two parts.

Part One: Git Puns

My identi.ca buddy madalu and frequent commenter here posted a few weeks ago the following notice:

#ubuntu-one… No thanks! I’ll stick with my home-brewed git + server + usb drive solution. My git repos breed like rabbits!

Which basically sums up my opinion on ubuntuone. But I thought that the “my git repos breed like rabbits” was both accurate (git repositories are designed to be replicated in their entirety), and a sort of funny way to put it. And being the kind of person that I am, I decided to see what other (potentially dirty) puns I could make about git. Here’s what I came up with:

what did one git repo say to another git repo? pull my diff

what did mama git say when she found her remote in his room making new branches? octopus merge this instant!

what did one git remote say to entice another remote to branch? it’s ok we can just tell them we were cherry picking later.

what did dr. git say when a repo complained of bloating? git gc

I should point out that these four puns all demonstrate a factual feature of git, though the “pull my diff” isn’t exactly what happens.

“Octopus Merge” is the method that git uses when there are a lot of divergent branches (more than three) that need to be merged together. Similarly “cherry picking” is a way to manually select what changes get merged together if you’re not ready to do full merges, and git gc is the cleanup script that goes through and re-compresses and prunes the database so that your repo works faster and with less disk space.

Anyway, I’m out of puns, you all are welcome to join in.

Part Two: Atypical uses of Git.

I’m sure I’ve written a bunch here about how I’m not really a programmer, and while this is true I do use git a lot. In part I think this is because git is really mostly an ad-hoc file system and also given how I write, the kind of writing I do isn’t that different from programming.

So aside from storing my writing projects, and my orgmode, I do things like store all of my mail directories in git. Which you might think is kind of weird, but the truth is that it makes keeping lots of computers in sync a rather simple proposition, and its damn fast.

I also have a directory I call “garen” (but used to call “main”) that is basically my home directory. It has all my emacs lisp files, most of my non-mail related scripts, various configuration files. and so forth. It started out as a backup and workspace for smaller projects, but it’s since morphed into “that one thing I need to have of my computer in order to actually work.” When I was setting up the server it took a thousand things that might have been huge headaches and made them non-issues. Here’s what this repo looks like:

• emacs/ This is where my emacs-lisp files all live. I have a ‘init.el’ file which is basically the standard .emacs file, and a ‘gui-init.el’ file for code that I only want to run if I’m running desktop where I’ll be running non-console emacs frames. As a result on my machines my .emacs file looks like this: :

  (load "~/garen/emacs/gui-init.el")
  (load "~/garen/emacs/init.el")
  

  With the first line commented out if needed. End result, emacs loads the same everywhere, no thinking.

• scripts/ I add this to my path, so that any little bit of bash script that I want to be able to use is accessable and the same on all my machines.

• configs/ Generally my format is to have config_file.machine_name, for example: bashrc.leibniz. In the case of the bashrc, I have a “.common” file that has everything that all my machines need, while the machine specific files have everything that’s… well specific, and a source statement for the common file. So my “real” .bashrc looks like this: :

  source /home/tychoish/garen/configs/bashrc.leibniz
  

  And everything stays in sync between the machines. How cool is that.

That’s sort of the most important thing. The great thing is that this makes setting up a new user account on a server, or a box itself a piece of cake.

Food for thought!

I’m beginning to write this in the evening after a long day of system administration work. For birthday (though, really, it’s been on my todo list for a long time), I ordered and set up a server from these fine folks to serve as the technological hub of my activities. While I’ve been quite fond of those fine folks for quite a long time, there was a growing list of things that I always wished worked better with dreamhost, and it was finally starting to limit the kinds of projects I could undertake in. So I bit the bullet and went ahead and ordered the server and spent some time getting everything straightened out.

For some background: the server is just an instance running inside of a Xen hypervisor, which runs other servers together on the same hardware: this is good, I couldn’t really use a server that powerful all by my lonesome (and I wouldn’t want to have to pay for it all either). It’s also way more powerful that what I had before, and the vitalization allows me to act with impunity, because it’s as if I’m running my own server, really. I’ve been doing computer administration and web development for a long time, but I’ve never had to do anything like this so it’s been an experience to learn how DNS records really work, how all the different kinds of server applications work, how really cool the Apache web server really is. It’s a great learning experience, and I think it would be prudent (and potentially helpful for you) to reflect on the experience. So here are my notes on the adventure:

• I’ve been running Ubuntu on my personal/desktop machines since the great Linux switch, and I’ve been pretty pleased with it. Not totally wowed by it: it works, but my tendencies are toward the more minimal/lightweight systems. But more than anything, I’m really drawn to systems that just work more than I am to systems that work perfectly, and I’m pretty good at keeping systems working. In 5 years of OS X usage I installed an OS twice, and since I got things stable and running, the only installations I’ve done have been to put ubuntu on new machines.

  In any case, this server was a chance for me really explore debian stable (lenny), which I hadn’t ever done before. It’s so cool. It’s not sexy or daring or anything but in a server you don’t want that, and it just works. I think it probably also helps matters somewhat lenny was released only a few months ago, rather than nearly two years ago, but in any case I’m quite enamored of how well it works.

• Email is much more complicated than I think any of us really give it credit for. There’s all sorts of complicated mess with regards to how DNS servers identify mail servers to help fight spam, and then there’s all the filtering and sorting business, and even the “modern” email servers are a bit long in the tooth. I hope that it’s a “set it and forget about it” sort of thing, though to be truthful I just got it all running and set up initially, but there’s a lot of further setup to do, before I move it all around.

• I’m pretty proud of the fact that as i was going thought the set up process, I go to the point where it said “ok, now set up the FTP server, and I said ‘meh’ and moved around.” Turns out that I can do everything I need to do in terms of getting files onto the server with git/scp/ssh/rsync and FTP is just lame and old. Welcome to the new world, where file transfers are shorter, versioned, and more secure.

  This isn’t particularly new, I couldn’t tell you the last time I used FTP, but I think this represents both the utility in moving to a “real server,” and a larger shift in the way we think about webservers. FTP assumes that the main purpose of the webserver is to store and serve files. the ssh/rsync/git model assumes that your webserver exists to be “your computer in the sky.” Which it is. We interact with the computers on our desks in really complex ways; there’s no reason to interact with our computers in the sky by just copying files to and from it.

• I’m convinced that systems-administration work will increasingly be the “hard currency” (basis for exchange) for the networked computing age. It’s sort of onerous work, there are skills and knowledge that most people who need network service don’t have and don’t need to have, there are actual costs, the need is ongoing, and success is quantifiable.

  There’s definitely space here for people (like me, and others) to provide these kinds of services to people. Sort of “boutique” style. Clearly I have more to learn and more thinking to do on the subject, but it’s a start.

• Ejabberd is peculiar, and the Debian package is… less than ideal. I knew going in that there was a “web administration” interface which sounds cool until you realize that, it’s… not an administration panel as much as it is a sort of “web dashboard.” You still have to tweak the configuration file which is written in Erlang, and wow. That’s pain right there.

Having said that, it seems to work just fine, without much fussing, and I’m want the jabber-server to do a very limited set of things: host my own IM account and transports; host muc-chats (created by me); and that’s about it. I’m a bit worried that it might be a bit too heavy for this.

That’s about all. More to come, I’m sure.