Beyond Lists in Org Mode

I’ve written about this problem in org-mode, the emacs outlining and organization tool that I us, before, but I’m readdressing it for my benefit as well as yours.

Org mode is an outlining tool, fundamentally. It provides a nice interface for editing and manipulating information arranged in an outline format. Additionally, and this is the part that everyone is drawn to, it makes it very easy to mark and treat arbitrary items in the outline as “actionable,” or todo items in need of done. The brilliance of org-mode, I think, is the fact that you spend all your time working on building useful outlines and then it has a tool which takes all this information and compiles it into a useful todo list. How awesome is that. For more information on org-mode, including good demonstrations, check out this video.

The problem is a common and recurring one for me. I basically live in the agenda mode--that compiled list of todo items--and I don’t so much use org-mode for making outlines. Truth is, I have a “Tasks” heading in most org files, and I use the automatic capture option (e.g. org-remember) to stuff little notes into the files, and beyond that, I mostly don’t interact with the outlines themselves.

This isn’t a bad thing, I suppose, but it means that org-mode can’t really help you, and you’ve short-circuted the ability of org-mode to improve the organization. Under ideal circumstances, org allows you to embed and extract todo lists from the recorded record of your thought process. If you’re not actively maintaining your thoughts in your org-mode files, it’s just another todo list. That isn’t without merit, but it doesn’t allow the creation of tasks and the flow of a project to spring organically from your thoughts about the project, which is the strength of org mode.

Intermission: I took a break from writing this post to go and reorganize my org files. What follows are a list of “things I’ve been doing wrong” and “things I hope to improve.”

  • I don’t think I had enough org-files. There are lots of approaches to organizing information in org: one giant file, lots of small files for individual projects, a few mid to large files for each “sphere” of your life.

    Initially I took the “medium sized files for major ongoing projects.” I had a writing file, and a work file, and a writing file, and files for the fiction projects that I’m working on, and a notes file, and a clippings file, and so forth. Say about 8-10 files. It works, but I think the thing it did was it caused me to use the org-remember functions to just dump things in a “tasks” heading, and then work from the agenda buffer, and not ever really have to touch the files themselves. Org files need to be specific enough that you would want to keep them open in another window while you’re working on a project. I think the point where you know you’ve gone too far is when the first level headings start to replicate organization that might better be handled by the file-system.

  • Use the scheduling and deadline functions to filter the todo list into something that is workable. It’s easy to just look at the task list and say “oh no, I don’t want to work on this task right now because it depends on too many things that aren’t done, and there are other things that I could work on.” Scheduling an item, if not setting a deadline, forces me (at least) to think practically about the scope of a given project, what kind of time I’ll have to work on it, and what other tasks depend upon it.

  • When you’re using org to manage huge blocks of text--or any system, really--it can be difficult if you have multiple hierarchies and depths of greater than two or three. It just gets hard to manage and keep track of things and figure out where things are, particularly given how useful and prevalent search tools are.

    Having said that, When you’re organizing tasks in org, that limitation, one that I find myself imposing upon myself doesn’t really work terribly well, and leads to files that might actually be more difficult to read and to work out of.

  • I started using the “org-archive-subree” function for archiving content when I was through with parts of the outline, This sends the archive to a separate file and while it works, I find it… less than useful. I’ve since discovered “org-archive-to-archive-sibling” which is a great deal of awesome, and I recommend using it exclusively.

  • Write content in org mode when possible. Though some people (hi Matt!) are keen on using org as a publication system, I’m not sure if this is the right answer, but I do think that its good during very creative phases of projects to do the work in org, mostly as I think it facilitates focusing on the current problem (through collapsing of the tree to show you just what you’re working on,) and also for working non-linearly as you can leave yourself TODO items for later action.

At the same time, if you tend to maintain org files that contain planning for more than one project, I find it cumbersome to also draft in these files. So I think “keep smaller very focused org files, and maybe do drafting in them if appropriate.”

That’s a start at least. I’ve made these changes--which are really quite subtle--and I like the way it feels, but we’ll have to see how things shake down in a few weeks. As much as I want to avoid tinkering with things--because tinkering isn’t the same as getting things done--I really do find it helpful to review processes from time to time and make sure that I’m really working as effectively as I can.

fact files

I wrote a while back about wanting to develop a “fact file” or some way of creating a database of notes and clippings that wouldn’t (need to be) project specific research, but that I would none the less like the keep track of. Part of the notion was that I felt like I was gathering lots of information and reading lots of stuff, that I didn’t really have any good way of retaining this information beyond whatever I could recall based on what I just happen to remember.

I should note that this post is very org-mode focused, and I’ve not subtitled very much. You’ve been warned.

Ultimately I developed an org-remember template, and I documented that in the post linked to above.

Since then, however, I’ve changed things a bit, and I wanted to publish that updated template.

(setq org-remember-templates'(
  ("annotations" ?a
    "* %^{Title} %^g \n  :PROPERTIES:\n  :date: %^t\n  :cite-key: %^{cite-key}\n  :link: %^{link}\n  :END:\n\n %?"
    "~/org/data.org" "Annotations and Notes")
  ("web-clippings" ?w
    "* %^{Title} %^g \n  :PROPERTIES:\n  :date: %^t\n  :link: %^{link}\n  :END:\n\n %x %?"
    "~/org/data.org" "Web Clippings")
  ("fact-file" ?f
    "* %^{Title} %^g \n  :PROPERTIES:\n  :date: %^t\n  :link: %^{link}\n  :END:\n\n %x %?"
    "~/org/data.org" "Fact File")
  ))

What this does, reflects something I noticed in the way I was using the original implementation. I noticed that I was collecting quotes from both a variety of Internet sources and published sources. Not everything had a cite-key (a key that tracks the information in my bibtex database,) and I found that I also wanted to save copies of blog posts and other snippets that I found useful and interesting, but that still didn’t seem to qualify as a “fact file entry.”

So now there are three templates:

  • First, annotations of published work, all cross referenced against cite-keys in the bibtex database.
  • Second, web clippings, this is where I put blog posts, and other articles which I think will be interesting to revisit and important to archive independently for offline/later reading. Often if I respond to a blogpost on this blog, the chances are that post has made it into this section of the file.
  • Third, miscellaneous facts, these are just quotes, in general. Interesting facts that I pull from wikipedia/wherever, but nothing teleological, particularly. It’s good to have a place to collect unstructured information, and I’ve found the collection of information in this section of the file to be quite useful.

General features:

  • Whatever text I select (and therefore add to the X11 clipboard) is automatically inserted into the remember buffer (with the %? tag)
  • I make copious use of tags and tag compleation which makes it easier to use the “sparse tree by tag” functionality in org-mode to just display heading which are tagged in a certain way.) So that I can see related content easily. Tags include both subject and project-related information for super-cool filtering.
  • All “entires” exist on the second level of the file. I’m often sensative to using too much hierarchy, at the expense of clarity or ease of searching. This seems to be particularly the case in org-mode, given the power of sparse trees for filtering content.

So that’s what I’m doing. As always, alternate solutions feedback are more than welcome.

useful emacs and org-mode hacks

After a long time of intentionally avoiding tweaking my emacs file, I’ve gotten back into tweaking and hacking on my setup a bit in emacs land. Rather than wax philosophical about emacs and plain text, I thought I’d share a few things with you all in the hopes that this will prove helpful for you.

I’ve given some thought to publishing a git repository with my emacs files, my awesome config, and the useful parts of my bashrc files. My only hesitation is that all of these files aren’t in one repository right now, and I’d need to do some clean up to avoid publishing passwords and the like. Encouragement along this direction might be helpful in inspiring me to be a little more forthcoming in this direction.

Keybinding “Name Spaces”

I’ve begun reorganizing key-bindings in a standard pattern, in order to avoid collision of bindings in certain spaces. The problem with the “C-x C-[a-z]” bindings is that it’s hard to get really good mnemonic bindings for whatever you’re trying to do, and there are few of them. I’ve taken to putting all of my custom bindings (mostly) under “C-c [a-z],” and then grouping them together, based on mode or function.

(global-set-key (kbd "C-c o a") 'org-agenda-list)
(global-set-key (kbd "C-c o t") 'org-todo-list)
(global-set-key (kbd "C-c o p") 'org-insert-property-drawer)
(global-set-key (kbd "C-c o d") 'org-date)
(global-set-key (kbd "C-c o j") 'org-journal-entry)
(global-set-key (kbd "C-c r") 'org-remember)
(global-set-key (kbd "C-c a") 'org-agenda)

(global-set-key (kbd “C-c w s”) ‘w3m-search) (global-set-key (kbd “C-c w t”) ‘w3m-goto-url-new-session) (global-set-key (kbd “C-c w o”) ‘w3m-goto-url) (global-set-key (kbd “C-c w y”) ‘w3m-print-this-url) (global-set-key (kbd “C-c w l”) ‘w3m-print-current-url)

You can see here, org-mode related bindings and w3m related bindings. “C-c o” is wide open, and I haven’t yet found anything in that space that I’ve overwritten. Same with “C-c w”. Even though the command key-chains are a bit longer than they might be if I piled things more sporadically, I can remember them more quickly.

Org-journal is something I got from metajack, and I don’t use it as much as I should. Everything else is standard org or w3m functionality.

I suppose I should make mode-specific key-bindings so that I’m not eating away global name space for mode-specific functionality, but I’m not sure that would make things too much clearer or easier to remember.

Also I really like the (kbd ") syntax for specifying key sequences. Much easier to read and edit.

Custom File settings

A while back I pulled my customize-set variables out of my main init-file, and gave them their own file, which means my init-file isn’t quite so long, and the variables that I’m not setting.

Nevertheless, I like to set as many variables by hand with setq just so that I can be in better touch with what settings I’m changing. This code, moves custom-set variables out of main file:

(setq custom-file "~/path/to/emacs.d/custom.el")
(load custom-file 'noerror)

Window Transparency and Font Settings

At the top of my init file, I have the following four lines to set font and window transparency.

(add-to-list 'default-frame-alist '(font . "Monaco-08"))
(set-default-font "Monaco-08")
(set-frame-parameter (selected-frame) 'alpha '(86 84))
(add-to-list 'default-frame-alist '(alpha 86 84))

Note that this depends on running a composting manager like xcompmngr, and the transparency is quite subtle. With great pleasure, running this code at the begining of the init file means that emacs’ looks and behaves correctly when I start it using a plain,

emacs --daemon

command from a regular bash prompt. I’m running fairly recent (but perhaps not the actual release?) builds of emacs 23. Note that I’d had trouble getting daemonized versions of emacs to start and capture the right information about font and transparency. That seems to be resolved.

Aliases

Here’s the alaises I use to make key-commands less work to type. It’s sort of a space between “creating a key binding” and just using the function from M-x Here’s the current list:

(defalias 'wku 'w3m-print-this-url)
(defalias 'wkl 'w3m-print-current-url)

(defalias ‘afm ‘auto-fill-mode) (defalias ‘mm ‘markdown-mode) (defalias ‘rm ‘rst-mode) (defalias ‘wc ‘word-count) (defalias ‘wcr ‘word-count-region) (defalias ‘qrr ‘query-replace-regexp) (defalias ‘fs ‘flyspell-mode) (defalias ‘oa ‘org-agenda) (defalias ‘uf ‘unfill-region) (defalias ‘ss ‘server-start) (defalias ‘se ‘server-edit) (defalias ‘nf ‘new-frame) (defalias ‘eb ‘eval-buffer) (defalias ‘mbm ‘menu-bar-mode) (defalias ‘hs ‘hs-org/minor-mode)

There are a number of these that I don’t use much any more, but it’s not worth it to edit the list down.

New Modes

A few new modes that I’ve been using

yassnippet

I’ve started using yasnippet more, and I’m quite fond of it for managing and inserting little templates into files as I’m working. There’s not a lot of example code that I can share with you, as it just works, but I do have a couple of notes/complaints:

  • I have to use C-i to expand snippets. The “tab” key doesn’t seem to work to expand snippets ever.
  • The organization of the snippets directory is absurd. I understand how the structure of the hierarchy mirros the way modes are derived from one another, and having the expansion triggers as file names also makes sense, but it’s really hard to organize things. Do people use modes that aren’t derived from “text-mode”? Are there any? There should be a “global” directory in the snippets folder (next to text-mode) where all of the files in any number of folders beneath “global” are available in all modes.
  • It’s amazing useful, and there are some things that I need to create snippets for that I haven’t. This is on my list of things to do.

w3m

w3m is an external text-mode browser that emacs hackers have written a good bridge to emacs for. What this means is you get a text-mode browser that works in emacs, but it’s speedy because page rendering happens outside of emacs.

It works, and it’s immensely use-able, though the key-bindings are a bit hard to remember and there are too many of them to change at once without completely driving yourself crazy.

I read a thread on the emacs-devel list a few months back about embedding something like uzbl inside of emacs (making emacs more like a window-manager) and I think the project presents an interesting possibility, but I think w3m succeeds because it makes the text of a website accessible within emacs.

Embedding a “real” browser in emacs, would just duplicate window manager functionality, and add complication. I think better to make a uzbl config file that was emacs-friendly, and some sort of “create emacs buffer with selected uzbl text” bridge would be nice, but anything more than that seems foolish.

My (few) w3m key-bindings are above.

nxml mode

With all this web-design work I’ve been doing, (eg. cyborg institute) I’ve needed to stray into using HTML and CSS modes. There’s this newer mode called nxml-mode which is delightful because it validates your html/xhtml/xml file on the fly (great!) but I’ve found it less than helpful for situations where I just have a snippet of HTML/XML in a given file, because it gets included later. Nonetheless, powerful stuff.


That’s about it for now. There are few other things, but I don’t feel ready to really explore them at this point, mostly because I haven’t gotten familiar enough to know if my modifications have been useful. Muse-mode, etc.

Any good emacs code that I should be looking at?

writing in org mode

With all luck, I’ll have most of a draft of the short story I’ve been working on done by the time this goes live, but if not certainly rather soon there after. This is an exciting announcement in and of itself, but perhaps the more interesting thing is that in the process of doing this I sank into writing this story in org mode.

My general M.O. for writing for the last several years has just been to write and store the files in markdown and use whatever text editor I fancy. I write the blog this way, I write papers this way. Everything seems to work fine, there are converters for LaTeX, HTML, and the plain text format is absolutely and completely readable to people who aren’t as obsessive about text files as I am.

While I’m a huge org-mode proponent, I don’t tend to think that org-mode makes a particularly good writing environment (or haven’t, heretofore) because unless you use org-mode org files are sometimes a bit ugly, and the syntax is enough different from markdown to confuse me, and…

The general consensus, that I’ve seen is that while org-mode is indeed a great boon to the intensive-emacs user, that it’s not an ideal production editing environment. muse-mode, or my favored markdown-mode might be better if you’re actually writing text.

And then, as I got into the writing of this story, I realized that I was flipping rather seriously (and annoyingly) between my notes for the story and the story I was writing. Also, when I’m writing book-length (or conceptually book-length) work, I tend to break up the text into more manageable chapter-length or scene-length files, which is conceptually useful for me.

In a short story, it didn’t seem to make sense to break things up into more than one file, and after I’d written a couple thousand words, I realized that something needed to be done. I created a file, with some header meta-data (using the yaml form that jekyll), an org-mode statement to define custom-status words that seem relevant to the writing/editing process, and then first level headers define key scenes or breaks in the story. I’ve never written (or read, to the best of my memory) a story that required more than one level of organization (but ymmv), and then--and this is the clever part as far as I’m concerned--property drawers for notes about what happens in the scene.

Property drawers stay folded by default, and are intended to store a collection of key-value pairs, but they don’t get exported by default, and so are a good way to keep your notes and your writing together and then export, as needed when drafting is done.

Also, I’ve recently added the following to my key-binding list, which adds a property drawer to the current heading, which is indeed a good thing:

(global-set-key "\M-p" 'org-insert-property-drawer)

I’ve posted a copy of my template file for your review and edification.

Comments?

Mobile Emacs

I have a confession. I last week (briefly) considered getting a Nokia N810 so that I could sync and use org-mode when I was away from my computer/laptop. The N800/810 is a small tablet that runs a Debian based operating system, which means it could run emacs, and I could write little clickable scripts that could do all of the syncing and awesomeness that I’ve grown accustomed to.

Then I realized how absurd this is, and cast it aside. My laptop is really mobile, and if I needed it to be lighter or more mobile, I could buy a new battery for it. And it has a full sized keyboard.

This is a sickness right?

Fact File Code

In my post about my fact file I said that I was going to “try things out and see how it goes” before I posted code to see how things work in the system. Well, I think things are pretty stable (I haven’t tweaked much), so I’m going to post my remember template, for the system described in that post.

(setq org-remember-templates
   '(("data" ?d "* %^{Title} %^g \n :PROPERTIES:\n :date:
       %^t\n :cit e-key: %^{cite-key}\n :link: %^{link}\n
       :END:\n\n %x %?"  "~/org/data.org")))`

If you want to tweak further, check out the relevant section of the org manual.

Enjoy and collect facts with abandon!

Things to Learn in Emacs

For my good (and yours), here’s a list of things that I think I should know how to do in emacs, but somehow don’t:

  • Use the version control package so that I’m not constantly popping between bash (terminal) and emacs to talk to git, and I don’t think that’s necessary.
  • Batch renaming with Dired. It should work, it should be simple, I’ve just not gotten there.
  • File specific org-mode settings, for properties, tags, and TODO keywords. I understand that it’s possible, but my brain/system doesn’t have room for that yet.

getting emacs daemon to work right

In the latest/forthcoming version of GNU Emacs there exists a daemon mode. The intent is to make it easier to run in one instance of emacs, and rather than starting new instances, you can run everything in one instance. Less overhead on your system, and everyone’s happy. Without daemon mode, you can still run in server mode and get nearly the same effect, but you end up with one emacs frame that’s the “host,” which means, if you’re dumb and close it (or it’s running inside of another process which crashes…) all of the frames close.

I suppose, as an interjection, that my attempt to explain why this is cool to a generalized audience is somewhat of a lost cause.

In any case, there’s a problem daemon mode doesn’t behave like anyone would want it to. This post explains it better than anything else I’ve read thus far, but it’s not all that clear. Basically when you start emacs --daemon it doesn’t load your .emacs init file correctly, and so things like display settings are off kilter, and can’t really be fixed. I resorted to running my “host” emacs frame inside of a terminal/screen session, because that worked and was basically the same from my perspective.

Nevertheless I’ve discovered “the fix” to the emacs daemon it to work like you’d expect. Run the following command at the command line (changing username and font/fontsize):

emacs -u [USERNAME] --daemon --eval "(setq default-frame-alist \ '((font-backend . "xft") (font . "[FONT]-[SIZE]")))" -l ~/.emacs

And then, open an emacsclient.

and there was much rejoicing

There are a couple of things that I can’t get to work reliably. Most notably, though the emacsclient works in terminal instances, it has some sort of problem with attaching new clients to X after an X crash/restart. No clue what this is about. Not quite sure what the deal is with this, but needing to reboot every time X goes down is a bummer. Other than that? Emacs bliss. For the moment.