Perhaps part of the reason the iPad doesn’t make sense for me is that I’ve never really used an iPhone, and at the core iPad is basically iPhone++. I opted to get a Blackberry because I wanted a device that did email right and messaging, I had/have an iPod Touch that I don’t really use and the little isolated applications that don’t interact well with my existing work flow and lock data in didn’t really appeal to me.

People are always talking about “the apps,” as if there’s something revolutionary happening here, and I’ve always been a bit non plussed. At least for my personal use, more “applications,” or places where “things can be done,” doesn’t always equate into more productivity.

Running in parallel to this, of course is the whole Web 2.0/web-based application development thing. (Is Web2.0 coasting to a halt yet? In any case, it’s probably still a current phenomena worth noting.) Although iPhone applications and Web applications are constructed in very ways that are quite technically distinct, there’s an underlying commonality between the way that both function. These applications are very much not general purpose. The best iPhone (or web) applications take a specific kind of data or information (or a collection of such), and provide a way for you to interact with that information in a way that’s (hopefully) intuitive.

In contrast the old familiar applications typically are designed to be either: platforms that enable you to accomplish a whole range of tasks within a certain sphere (“Office tools,” email applications, emacs), or some sort of filter or pipe that allows you to access data in one format in an interface that’s easy to use (RSS readers, Content management systems, pdf readers, web brwosers, etc.) There are certainly web applications that are basically pipes, and I’m sure Google would love it if we all used their application stack.

So, then, what describes this new “widget,” class of applications for mobile devices and the web? On the one hand it’s a case of developers “making do with what there is:” the iPhone imposes a number of serious constraints: small screen, limited on board computing resources, and a potentially inconsistent data connection. Small structured apps that are all about allowing you to interact with a small set of data work best in this environment. Additional I suspect that technology has advanced to a point where making these kinds of applications is finally possible and worth the it takes to build them: MVC frameworks have finally matured perhaps.

The enduring questions in my mind:

• Is the data that we’re throwing into databases by way of iPhone apps and web applications going to continue to be useful in the future? This gets at the “open data” question as much if not more than the “open source” aspect of applications. What’s the realistic utility of our data in the long form?
• Does this shift mark a move away from a certain kind of “paper-based logic,” to a more digital way of organizing or data and digital experiences?
• Is this just the process of bringing the kinds of “business applications,” that enterprises have developed and used for decades into the consumer sphere?
• How do the concepts of software freedom and data ownership transfer to this mode of using technology. While it’s easy to say “data ought to be free,” and “source code still matter” (both of which are true,) I think practically it’s a bit more difficult.
• What do these applications look like on the Desktop? Is the future JavaFX, Flex, Silverlight, and Apple Desktop Widgets? Is the desktop going to be stuck in the browser? Are there other options?

And while I think we’re definitely using and creating this kind of software with abandon, I don’t know if we’ve reached good answers to these (and other related) questions. I look forward to exploring these ideas, and hearing your thoughts!

A few years ago, highly citational, link/youtube video blogs came back into style again. This time rather than calling them blogs, we called them “tumblelogs.” I never really got into it, though I tried, and even my original inspiration for starting tychoish.com was to do a more “tumblelog-esque” blog.It never quite worked out. Then I read this post by Michael Coté which inspired a few things:

First, it got the following title to stick in my head and refuse to get out. Second, it left me with the idea that, although successful, sites like tumblr and to a different extent posterous basically ruined the tumblelogging revival.

Here’s the thing about tumblelogs: they worked and worked so well because they were efficient, because the people creating tumblogs were doing something unique and had unique voices, because you could keep your pulse on most of a single discourse by watching only a few sites/rss feeds. And then it became, very suddenly, trivial to make a tumble log. And so everyone had a tumblelog, and it was like blogging was new again, except things “meme’d out” at an epic pace and it became difficult to track what anyone was saying. It was like a distributed denial of service attack on our attention spans.

And as the dust settled, tumblelogs, at least as far as I could see, became less about a sort of delightful amalgamation of interesting content and more about templates, about piping in a fire hose of content from delicious/twitter/etc. So not only were there too many tumblelogs, but the style had devolved somewhat into this weird unedited, awkwardly template-ed mass of “crap” that is (in my opinion) quite hard to read or derive value from.

What Made Tumblelogs Work Originally

• The systems that powered them were kludgy but they made it very possible to post content easily. That’s a good thing.
• They used a unique sort of web design where design elements (tables/grids/CSS magic,) reflected and accented the content type.
• They were largely editorial functions. People followed tumblelogs because their authors were able to filter though content with exceptional speed and grace, and in the process of filtering provide real value.
• They were multimedia, and incorporated many different kinds of content. Not just links, not just embedded youtube videos, but snippets of IM and IRC conversations, song lyrics, pictures from flickr, and so forth.
• projectionist one of the first and best, was a group effort: when group blogs work, they really work. The tumblelog, seems like an ideal platform for group blogging.

How We Can Make Tumblelogs Work Again

• We use publishing systems and tools that are unique and that stretch and bend the form. A tumblelog theme for Wordpress, will probably always reek like wordpress. Same with other popular content management systems. Tumblelogs work because they’re not just blogs, they need to distinguish themselves both visually, and in terms of how their authors write the content.
• We undertake tumblogs as a collaborative effort. Group projects complicate things, of course, but they also create great possibilities.
• Vary content, intentionally, post quotes, chat excerpts, links, videos, lyrics, etc. Make sure that there’s a great deal of diversity of content. This is perhaps a problem to be solved in software, at least in port.
• Emphasize and cultivate editorial voice, and create an interface that forces authors and editors to touch the data.

Thoughts? Suggestions?

ETA: I’ve started to work on this wiki page outlining a “tumble manager” tool. I also did a bit of textual refactoring on February 27, 2010

By day I write documentation for systems administrators, and as a result I spend a lot (perhaps too much?) time thinking about how we organize computer systems so that they can be both useful and easy to manage in the long run. “Right, so…” you say? Well indeed. Recently it’s become clear to me that there are some generalizable lessons to be learned from sys-admining that might be helpful to those of us who are less organized than they’d like to be.

Which is pretty much everyone, right?

Right. In brief:

• Automate everything that can be automated.
• Closely followed by don’t automate something that doesn’t need automation.
• Prefer simplicity over complexity, and prefer systems that require you to remember fewer things.
• Design systems to make it possible for others to easily understand what you’ve done.

To elaborate:

Automation

Computers are really good at doing what you tell them to do, and although we often like to finddle with them to make them work better, ideally the more we let systems take care of themselves. Also tasks that are automated, if the automation is designed and tested properly don’t make silly mistakes. If you’ve written systems to automate your tasks, you can understand and predict how your system is going to handle the kind of data that you throw at it.

The admonishment to “not automate” until you need something, is basically a variant on age old recommendation to “avoid premature optimization.” While automation is a good thing indeed, and if the thing you’re automating is really something that can be delegated to the machine without intervention on your part, then that may be worth your while to automate that task. By the same token, it’s easy to think “we’re going to need to do this thing a lot, I might as well automate it before hand.” Which is a reasonable thought to hand, but this puts the cart before the horse, and leads to two undesirable and possible outcomes: first the task doesn’t need to be automated because it isn’t needed very often; you misunderstand what needs to be done and automate the wrong part of the task, or your automation doesn’t cover the edge cases and will need to be rewritten later.

Conventionally, automation tends to cover “coding” or scripting of some sort of task. Outside of programming and systems development, “automating” a task could be as simple as creating some sort of editor macro, or developing some new structure in your data store (database, files, etc.) to hold or manage a particular kind of data.

Simplicity and Complexity

The basic reasoning here is that while complex solutions are often elegant and attractive, and make a lot of sense when you’re setting something up, they always make you scratch your head six months or a year later when you need to go back and find something that you did back then or make a change to the system. Be wary of solutions to any problem that require too much consistency on the part of the user. If a system only works if you must remember to follow more than a few steps in a precise order, chances are things are too complex, and you’ll end up screwing yourself over later.

Ergo: Err on the side of simplicity, you’ll thank yourself later.

The more components and connections there are in a website application or deployment server the more potential for breakage is. The more complexity there is the better chance that FurtureYou or someone working in your footsteps will be totally confused by what you have set up. The same thing holds for whatever your trying to organize and manage.

Generalizable Organizational Methods

Chances are you’re the only one who will be taking notes/organizing your work/storing information in your system. Nevertheless, I think it always helps to assume that other people are going to need to be able to make sense of your system. Be it your notes, and research or in your web-servers. Other people are sometimes our future selves.

I tend to use the word system, in a way that most people would use the word “method.” I hope that’s not too confusing or distracting. I think I’ll probably elaborate on these topics a bit more before in a later post. In a lot of ways this is part of the core of Cyborg Institute, and if you feel interested or inspired by this kind of stuff, I’d love to hear more from you. Be in touch!

The Context

• git writing on wikish.
• haikus in git
• writing like a programer
• git thoughts

git is this version control system that’s designed to be used in a distributed manner, and supports a very divers and non-linear workflow. While it’s designed to support the work of software developers--particularly in large projects like the linux kernel--at the core, git is just a file system layer that has an awareness of time and iteration. It also does its magic on any kind of text files… code or writing. I use git to manage a lot of my writing--indeed, most of my digital life, which is a bit weird admittedly; and as a result people on the Internet, not to mention my coworkers,come to me with git questions from time. This post is a response to a more recent change.

How I Work

I have two kinds of repositories: general repositories which store a bunch of different kinds of files that I need to work: the general repositories that store files that I always need to get things done, and specific project-only repositories that only have the text (and possibly notes) for a very specific project. I also have a “writing” repository where I do drafting for the blog, and start writing projects that I’d like to version, but are too small yet for their own project repositories. The brief overview:

• garen is like my home directory within my home directory, and it has config files, scripts. and other daily essentials.
• org stores my org-mode files.
• fiction projects: I have five repositories in ~/ that store fiction projects, that I’m theoretically working on in some capacity, though I haven’t touched most of them regularly.
• writing holds blog drafting, and a couple of not-exactly-fiction, projects that I’m not quite ready to admit exist.
• website content: wikish, tychoish.com, cyborginstitute.com, the cyborg institute wiki and a few other website projects that I’m involved with have repositories to store their content.

The lesson here, about repository organization, is that git wants you to have distinct repositories for different projects. Its possible to merge repositories together (really!) and also to separate the histories of specific directories into their own repositories if you’re so inclined.

I write in emacs almost exclusively, I sometimes use magit, which is a delightful interface to git that works within emacs in a very emacs-centric way. If you use dired, magit will be familiar. Having said that, I mostly just add files, make commits and push repositories. Although I’ve been very interested in flashbake for some time, I’ve never really used it: it seems designed for people who aren’t used to version control or git, and the fact that I am means that it feels cumbersome to me. I suppose I should take this as a challenge, and attempt to hack it into something more usable from my perspective, but I’ve not felt the urge yet.

I use gitosis (but it’s in the debian repositories) on foucault (my server) to manage the publication of my git repositories. I push regularly, both to make sure that all of my machines are up to date, and also as a way of keeping my systems backed up. While I don’t take snapshots of my systems, I’ve been able to set up systems and been up and running inside of ninety minutes after reimaging a laptop without loosing a single bit. Although unorthodox, git is my backup strategy, and the restores work fine. I strongly recommend having your own git hosting set up. It’s not difficult, and while I think git hub is awesome on it’s own terms, independence and self sufficiency is really important here.

I don’t really take advantage of any branching and merging in git, though I’ve played with it enough to know how it works. I do have a branch in the repository for the novel I’m writing for an editor to be able to edit the novel as I write on it without needing to see their changes and comments until I get to that point.

And that’s sort of it. I use jekyll (or an old personal fork) and soon to be cyblog) as well as ikiwiki to publish content, but other than that, I just write stuff.

In any case, if you have thoughts on the subject I’d love to see your input on the wikish git writing page.

Write on!

Ok, as a self-respecting geek, I think I have to say something about this whole “iPad” thing.

I’m not as much of an Apple geek as I used to be, that’s for sure. It’s not that I don’t think that Apple’s doing something quasi-innovative and useful in the world of technology and the business around consumer technology. I think bringing UNIX to the hands of “mass market” desktop users was a great move. Although the iTunes Store is not without it’s issues and concerns, the fact that Apple was able to create a viable environment and market that allowed people to exchange money for software and content is probably a good thing. And Apple has brought to a general public a number of hardware configuration (in recent years) that I think make a lot of sense: the mini-tablet (iPhone/iPod Touch), laptops with usable battery lives, the pocket jukebox (iPod Classic,) and so forth.

Deep breath. Having said that… I’m not terribly impressed with the iPad, or moved by it’s potential at the moment. I know Apple often takes a few generations to make something really work, and so I think it’s important to not say “this implementation sucks, and so the whole notion sucks, and is with out merit.” Of all the things that I’ve heard (or said,) in the past few weeks of the iPad the following two threads have stood out:

• I’m really quite interested to see what other makers are going to be doing in this space. What’s the Lenovo tablet going to look like? HP? One of the leading complaints about the iPad (and iPhone) is Apple’s total-lock-down over the platform, and I think an Android tablet, or a super Nokia N900 is likely to be much more open and killer awesome.
• I’m interested to see what the iPad platform looks like in a revision or two. Add an SD slot? Multi-tasking? Additional input methods? It could look really awesome, and while I have misgivings (see below) I don’t want to write it off entirely.

At the same time, I don’t really feel like there’s an in-between device that I don’t currently have that I’d like to have. In a lot of ways, even I feel like I have too many devices, too many inboxes (of one sort or another), and too much technology to manage. I’m not complaining. The truth of the matter is that my laptop gets great battery life, isn’t very big at all, and does everything I need of a computer, and almost everything I need of technology in general. iPods are better for playing music if I’m moving around or in the car, and the Kindle is great for what it is, and I do sort of have a Blackberry habit.. but…

My technological challenge at the moment is that I don’t have enough time to get done that which I would like to get done, not that I have a situation where I could be more productive if I only I had a device that would do something more." That’s not something that seems to cross my mind very much. It might be nice to have a slightly more accessible emacs instance that I could use to enter snippets of text and work on things in the kinds of moment. I’m thinking a Nokia N900 might fit that bill pretty well, but I’m not sure.

If you’re thinking about getting an iPad, what’s the niche that you see it filing? Do you have a niche that seems like it might be iPad sized?

I’m thinking of canceling my home data connectivity and going with a 3G/4G wireless data connection from Sprint.

Here’s the argument for it:

• I’m not home very much. I work a lot (and there is plenty of internet there), and I spend about two thirds of my weekends away from home. This is something that I expect will become more--rather than less--intense as time goes on. It doesn’t make sense to pay for a full Internet connection here that I barely use.
• My bandwidth utilization is, I think, relatively low. I’ve turned on some monitoring tools, so I’ll know a bit more later, but in general, most of my actual use of the data connection is in keeping an SSH connection with my server alive. I download email, refresh a few websites more obsessively than I’d like (but I’m getting better with that), and that’s sort of it. I’ve also started running a reverse proxy because that makes some measure of sense.
• I find it difficult to use the data package on my cellphone. The fact that I get notified of all important emails on my phone, has disincentivized me from actually attending to my email in a useful way, and other than the occasional use of googlemaps (and I really should get an actual GPS to replace that…) If I get the right Wireless modem, however, it would be quasi-feasible to pipe my phone through the wireless Internet connection, so this might be a useful clarification.

The arguments against it are typical:

• The technology isn’t terribly mature, or particularly well deployed.
• Metered bandwidth is undesirable.
• Sprint sucks, or has in my experience, and the other providers are worse.

The questions that remain in my mind are:

• How well do these services work in moving vehicles? Cars? Trains?
• How much bandwidth do I actually use?
• Is this practical?

Feedback is, as always, very much welcomed here. I’m not in a huge rush to act, but I think it makes sense to feel things out. It also, I think posses an interesting question about how I (and we) use the Internet. Is the minimalist thing I do more idealistic than actual? I know that we have a pretty hard time conceptualizing how big a gigabyte of data actually is in practical usage. Further research is, clearly, indicated.

Edit: This plan would have to rely on the fact that I might be spending a large amount of time in a city with unmetered 4G access with sprint. I’ve used a gig and a half of transfer to my laptop’s wireless interface in 5 days. I think that would coincide with when I would be doing the heaviest traffic anyway. I wonder how unlimited the unlimited is…

One oft made complaint about emacs is that it takes forever to start up, particularly if you’ve got a lot of packages to load it can take a few seconds for everything to start up. In a lot of respects this is an old problem, that isn’t as relevant given contemporary hardware. Between improvements to emacs, and the fact that computers these days are incredibly powerful, it’s just not a major issue.

Having said that, until recently an emacs instance took as much as 7 seconds to start up. I’ve beaten it down to under two seconds, and using emacsclient and starting emacs with “emacs --daemon” makes the start up time much more manageable.

Step One: Manage your Display Yourself

I’ve written about this before, but really even a 2 second start time feels absurd, if I had to start a new emacs session each time I needed to look into a file. “emacs --daemon” and emacsclient mean that each time you “run” emacs rather than start a new emacs instance, it just opens a new frame on the existing instance. Quicker start up times. It means you can open a bunch of buffers in one frame, settle into work on one file, and then open a second buffer and edit one of the previous files you opened. Good stuff. The quirk is that if you’ve set up your emacs files to load the configuration for your window displays late in game, the windows won’t look right. I have a file in my emacs files called gui-init.el, and it looks sort of like this:

(provide 'gui-init)

(defun tychoish-font-small () (interactive) (setq default-frame-alist ‘((font-backend . “xft”)(font . “Inconsolata-08”) (vertical-scroll-bars . 0) (menu-bar-lines . 0) (tool-bar-lines . 0) (left-fringe . 1) (right-fringe . 1) (alpha 86 84))) (tool-bar-mode -1) (scroll-bar-mode -1) )

(if (string-match “laptop” system-name) (tychoish-font-big))

Modifying, of course, the system name, and the settings to match your tastes and circumstances. The (if) statement allows you to have a couple of these -font- functions defined and then toggle between them based on which machine you load emacs on. Then in your init file (e.g. .emacs), make sure the first two lines are:

(setq load-path (cons "~/confs/emacs" load-path))
(require 'gui-init)

Establish the load path first so that emacs knows where to look for your required files, and then use the (require) sexep to load in the file. Bingo.

Package Things Yourself

We saw this above, but as much as possible avoid using the load function. When you use load emacs has to (I’m pretty sure) do a fairly expensive file system operation and then load the file and then compile and load the file. This takes time. Using the require function is not without it’s own cost, but it does save some time compared to load because it lets you take advantage of the work emacs does with the library loading. At least in my experience.

In your various .el files, insert the following statement:

(provide 'package)

And then in your .emacs, use the following statement

(require 'package)

To load it in. You’re probably already familiar with using these to configure packages that you download. Better yet, don’t require at all, but use the auto-load function. This just creates a little arrow inside of emacs that says “when this function is called, load this file, and hopefully the ‘real’ function by this name will be in there.” This lets you avoid loading packages that you don’t use frequently until you actually need them. The following example provides an auto-load for the identica-mode:

(autoload 'identica-mode "identica-mode.el" "Mode for Updating Identi.ca Microblog" t)

Byte Compile files as much as you can.

Contrary to whatever you’ve been told, emacs isn’t a text editor, as much as it is a virtual machine with a good deal of low level functions established for interacting with text and textual environments and some editing-based interfaces. But really at the core, it’s just virtual machine that interprets a quirky Lisp dialect.

The execution model is pretty simple and straightforward, particularly to people who are used to Java and Python: you load source files, emacs imports them and compiles them half way, they’re not the kind of thing that you could read on your own or would want to write, but it’s not quite machine code either. Byte-compiled files are easier for the machine to read, and quicker to process, but they’re not human intelligible. Then when you need to do something with the function that it’s byte-compiled, emacs compiles it the rest of the way into machine code and executes it. Usually this all happens too fast that we don’t really notice it.

One tried and true means of speeding up emacs load times is to byte-compile files manually so that emacs doesn’t have to do it itself when it loads. The emacs-lisp libraries are byte compiled when emacs installs itself, but your files probably aren’t. Now generally, only byte-compile files that you’re not going to be editing yourself regularly. Byte compiled files have an .elc extension, and as soon as there’s a .el file and a .elc of the same name in a directory, emacs will ignore the .el file even if there have been changes made. To byte compile an emacs-lisp file, simply type M-x to get the execute-extended-command prompt, and then run the function byte-compile (i.e. “M-x byte-compile”). Viola!

I hope these all help you all and lead to a slightly more efficient emacs experience.

So much of the time, when we talk about network services, technological/software freedom, and this idea of “Cloud” computing, there’s a bunch of debate: “is it a good idea?” “are we giving up too much freedom?” “how does this work out economically?” “what about privacy in the cloud?” While these are important questions, without doubt, I fear that they’re too ethereal, and we end up tussling with a bunch of questions about the future and present of computing that might not be entirely worth debating (at least for the moment.)

Lets take 2 assertions, to start:

1. There are some applications--things we do with technology--that work best when these applications are running on high performance servers that have consistent connections to the Internet, that we can access regardless of where we are in the world.

2. The only way to have control over your data and computing experience is to be responsible for the administration and maintenance of these services yourself.

Huh?

I mean to say, that if we care about our autonomy, and our freedom as we use computers in the contemporary age (i.e. in the era of cloud computing), the only thing to be done is to run our own services. If the fact that Google has all of your data scares you: run your own mail server. If the fact that all of your microblogging output is on twitter, run your own status.net instance. And so forth.

If we really care about having power over our technological experiences, we must take responsibility for services on the Internet. We can say “wouldn’t it be nice if service providers weren’t such dicks with our data,” or “wouldn’t it be nice if software developers wrote networked software that respected our freedom.” And while it would be nice, these convinces don’t in and of themselves

Control over technology and autonomy in the networked context ultimately means that we as users have to:

• Administer networked servers that provide us with the services that we want and need to do whatever it is that we do.
• Participate in some exchange for networked services (i.e. pay for service, either in cash or by way of access to data.)

That’s hard! Computers should get easier to use not harder, right?

Leading question there, but…

Yes. One of the leading arguments for consumer-“Cloud Computing” is that by accessing computer services (software) in the browser, developers can provide a more structured and “safe” user experience. At least that’s how I understand it.

While this is a great thing in terms of making computers more accessible, no argument from me, I think we must be careful to avoid confusing of use" with technologically limiting. I fervently believe that its possible to design powerful software that is also easy to use, and I think that as often as not, a confusing technology is an opportunity to provide a teaching experience as much as it presents an opportunity to improve a given technology.

And if it comes down to it, there are situations where it doesn’t matter so much if you’re the one entering the commands into the server. It doesn’t much matter if you are the one managing the server or if you’ve hired someone to configure it for you. As I think about it, there’s probably something of a niche here for people to offer management services in a very boutique sort of style.

If we have to contract to people to do our administration for us, is that really a step in the right direction?

I think, it is. At the moment we pay for our networked computing services (i.e. gmail) by looking at google’s ads next to our mail and giving Google access to the aggregate of our mail spools so that they can mine it for whatever data they need. The other price that we pay for these services is “lock in:” once we commit to using a service it’s quite difficult to change to an alternate provider. Since these are “real costs,” it seems reasonable to expect and want to pay (money) for services that don’t have these costs. Which is where cooperative and boutique-style services make a lot of sense.

I’m not a systems administrator, I just want to do [the thing that I do] and not have to tinker with my computer. This is a lousy idea.

And that’s a lousy question.

To dig in a bit further. I don’t think that “doing the [whatever you do],” would necessarily require a lot of tinkering. It might, of course, and the chances are that we’ve all had to tinker with our technology at one point or another. In most cases tinkering is an upfront rather than ongoing cost. Ideally, the other thing that having full control over your network services you’ll be able to use have services which are more tailored to [the thing you do] than the one size fits all application provided by a third party.

Ok, so what’s the stack look like.

I’m not sure. There’s clearly a common set tasks that we currently use in the networked context. I’m not sure what the application is, exactly, but here’s a beginning of what this “application stack” looks like.

• An XMPP Server like Prosody.im, with PyAIMt and other convectional IM network transports.
• Some sort of Email Service: Citadel springs instantly to mind as an “all in one solution,” but some postfix+procmail+fetchmail+horde/squirrelmail seems to make some sense
• A web server, either for hosting personal websites, or with some sort of authentication scheme (digest?) for sharing files with yourself. The truth is that web servers, are pretty darn lightweight and it doesn’t make sense to not install one. Having said that, people see “web hosting,” and probably often think “Well, I don’t really need web hosting,” when that’s almost beside the point.
• SSH and some system for FUSE (or FUSE-like) mount points, so that they can use and store remote files.
• There’s probably a host of web-based applications that would need to be installed as a matter of course: some sort of web-based RSS reader, wiki-like note taking. Bookmarking. Some sort of notification service, Etc.
• [your suggestion here.]

People have been thinking about databases recently. Even I’ve been thinking about databases, and I’m not particularly prone to thinking about databases. It’s fair given the ongoing drama of the Oracle/Sun, and even mainstream press of the NoSQL Movement. I’d like to take a step back and think a bit more honestly and holistically about the database application, aboth this “NoSQL” phenomena, and about the evolving role of relational database management systems in our technology “ecosystems.”

(Seriously folks this is what I think about for fun in my free time.)

I’ve been milling over the notion that databases, like MySQL and PostgreSQL and Oracle’s RDBM products, are not particularly “Unix-like.” Sure they run on Unix systems, and look and feel like Unix applications, but the niche fulfill--providing quick access to structured data with a specialized query language, doesn’t jive with the Unix philosophies: small specialized tools for precise tasks. “Plain text” as lingua franca of system tools, and so forth.

Databases solve a problem. Indeed they solve a problem in a very functional and workable manner. I don’t want to suggest that the relational database model is somehow broken; however, I would like to suggest that industrial strength database systems are over utilized, and have become the go-to solution for storing and interacting with data of any kind, even in cases where they’re not a good fit for the job at hand.

I’m not the first person to suggest this, not by a long shot. The NoSQL “movement,” addresses this issue from a couple different direction. It’s true that NoSQL refers to a collection of practices and approaches related to providing systems for storing data that goes above and beyond the type and model of a database system. In the end NoSQL is about addressing the scaling problem: what happens when we have so much data that it can’t easily fit in one database system, or in situations where centralized model is untmaintable for any number of reasons. I think NoSQL is also relevant as we think about storing data that doesn’t easily fit into RDBMs’es: I’ve seen a lot of very poorly architected database systems, that suffer from a “square peg in round hole” problem.

Indeed, as we try and put all of our data in these RDBMs systems, particularly data that doesn’t fit very well, these databases loose their ability to scale. The complex logic required to pull more complex data back out of a database and reassemble it for use and analysis is computationally expensive and doesn’t scale particularly well.

But let’s focus for a moment on the scaling question, apart from the data modeling and storage question. The real problem at the core of the scaling question is: we need a way, a thing, that allows multiple systems to access a shared data store in a reliable and consistent manner.

The ongoing work around clustered file systems seems to address this issue from a much different direction, and perhaps a more interesting perspective. Beyond a certain point--and its a fuzzy point--database systems basically become file system replacements. So rather than work on making databases more like file systems, the thought is (I assume) lets make file systems a bit more “database like.” Like I said, I don’t know a lot about the ins-and-outs of clustered file systems, but I think, in addition to worrying and thinking the future of current database systems, we need to also think about the future of these very scalable and clustered manner.

I’m not sure what the next-generation data storage technology really looks like, the NoSQL stuff is a step in the right direction, but I’m not sure if it’s a large enough step in a lot of ways, as its focus is a bit narrow. To be honest, I’m not incredibly familiar with the work that’s going on in the clustered file system space. Nonetheless, I think it’s important to not just think about the future of the relational database platforms as such, but the model and the underlying problems that these kinds of data storage methods address, and to think about other possible ways of addressing the original issues.

I’m very close to declaring feed reader bankruptcy. And not just simple “I don’t think I’ll ever catch up with my backlog,” but rather that I’ll pull out of the whole RSS reading game all together. Needless to say, because of the ultimate subject matter--information collection and utilization and cultural participation on the Internet--and my own personal interests and tendencies this has provided some thinking… Here goes nothing:

Problems With RSS

Web 2.0 in a lot of ways introduced the world to ubiquitous RSS. There were now feeds for everything. Awesome right?

I suppose.

My leading problem with RSS is probably a lack of good applications to read RSS with. It’s not that there aren’t some good applications for RSS, its that RSS is too general of a format, and there are too many different kinds of feeds, and so we get these generic applications that simply take the chronology of RSS items from a number of different feeds and present them as if they were emails or one giant feed, with some basic interface niceties. RSS readers, at the moment, make it easier to consume media in a straightforward manner without unnecessary mode switching, and although RSS is accessed by way of a technological “pull,” the user experience is essentially “push.” The problem then, is that feed reading applications don’t offer a real benefit to their users beyond a little bit of added efficiency.

Coming up a close second, is the fact that the publishers of RSS sometimes have silly ideas about user behaviors with regards to RSS. For instance there’s some delusion that if you truncate the content of posts in RSS feeds, people will click on links and visit your site, and generate add revenue. Which is comical. I’m much more likely to stop reading a feed if full text isn’t available than I am to click through to the site. This is probably the biggest single problem with that I see with RSS publication. In general, I think publishers should care as much about the presentation of their content in their feed as they do about the presentation of content on their website. While it’s true that it’s “easier” to get a good looking feed than it is to get a good looking website, attending to the feed is important.

The Solution

Web 2.0 has allowed (and expected) us to have RSS feeds for nearly everything on our sites. Certainly there are so many more rss feeds than anyone really cares to read. More than anything this has emphasized the way that RSS has become the “stealth data format of the web,” and I think it’s pretty clear, that for all its warts, RSS is not a format that normal people are really meant to interact with.

Indeed, in a lot of ways the success of Facebook and Twitter have been as a result of the failure of RSS-ecosystem software to present content to us in a coherent and usable way.

Personally, I still have a Google Reader account, but I’m trying to cull my collection of feeds and wean myself from consuming all feeds in one massive stew. I’ve been using notifixlite for any feed where I’m interested in getting the results in very-near-real time. Google alerts, microblogging feeds, etc.

I’m using the planet function in ikiwiki, particularly in the cyborg institute wiki as a means of reading collection of feeds. This isn’t a lot better than the conventional feed reader, but it might be a start. I’m looking at plagger for the next step.

I hope the next “thing” in this space are some feed readers that add intelligence to the process of presenting the news. “Intelligent” features might include:

• Noticing the order you read feeds/items and attempting to present items to you in that order.
• Removing duplicate, or nearly duplicate items from presentation.
• Integrate--as appropriate--with the other ways that you typically consume information: reading email and instant messaging (in my case.)
• Provide notifications for new content in an intelligent sort of way. I don’t need an instant message every time a flickr tag that I’m interested in watching updates, but it might be nice if I could set these notifications up on a per-folder or per-feed manner. Better yet, the feed reader might be able to figure this out.
• Integrate with feedback mechanisms in a clear and coherent way. Both via commenting systems (so integration with something like Disqus might be nice, or the ability auto-fill a comment form), and via email.

It’d be a start at any rate. I look forward to thinking about this more with you in any case. How do you read RSS? What do you wish your feed reader would do that it doesn’t?