This post is adapted from a post I made to the org-mode email list a few weeks ago. I proposed an application to compliment MobileOrg for writing. Where MobileOrg collects the core bits of org-mode’s task planning functionality in a form that makes sense for smart phone users, the parts of org-mode functionality that people use to for writing and organizing the content of larger form projects isn’t particularly accessible.

I spend (or should spend) 70% or more of my time in front of a computer writing or editing something in org-mode. Most of my org files have tens of thousands of words of blog posts, notes, drafts of articles, and so forth. While I can store that data on an android device with only minor problems using a little script that I put together, and I can capture content into my org-files using email and some nifty filters, and there are text editors that can let me edit these files: it could be better.

The proposal is simple. Can we build something like Epistle for org-mode? It might just render org-mode text to HTML, and frankly that would be enough for me. If the editing interface had an org-indent-mode equivalent, org-syntax highlighting, and even collapsing trees or org-narrow-to-subtree, that’d be kind of like heaven.

I’m not a mobile developer, so I can’t promise to start making an app this instant if there’s interest but if anyone’s bored and thinks this might be a good idea (or knows of something that might work better for this.) I’d love to hear about it. If someone wants to start work on this, I’ll do whatever I can to help make this a reality.

Onward and Upward!

I wonder if, at some point, this constant state of overload and flux in my world will begin to seem normal and I’ll just adjust to that normal. In the mean time, exciting things are happening and I’m not quite sure of the best way to write about them. Perhaps soon. For now, I’m trying to get better about updating more regularly and I have a bunch of links of stuff that have happened on the wiki in the past couple of weeks that I’d like to share. Here we go:

Discussion of Rhizomes

jfm and I had a good exchange about an old post, /posts/ideology-and-systems-administration. Basically the posts says, “systems administrators have a unique approach to solving technological problems,” and discussed the implications of systems administrators background on technology development. I think our clarifications were useful.

There are a couple of comments on my recent series on a productivity. First, I wrote a post about task planning and creating task items, and Matt posted a comment. Second, a number of us had an ongoing conversation on mobile productivity in response to the “Mobile Productivity Challenges” post that touched on emacs (of course!) input, and context switching.

Site Tweaks

This is a pretty minor point, but I’ve been subtly tweaking the design a little in the site. There are now links to the tags page and the site map in the upper right hand corner. I’ve also made links to as-of-yet-uncreated wiki pages red (according to wiki-convention.) I think (and hope) that red links are easier to spot when they’re red. Feedback on the design would be most welcome. My goal is to make the site welcoming, easy to use, and to minimize the amount of “fussiness.” It might be time for a full refresh, but feedback on the subject might be good.

Critical Futures and Wiki Fiction

Eventually the story will move to the Critical Futures domain, but that’s a bit down the road. Right now I’d rather focus my time/energy on writing some stories, for now (on this wiki.) Infrastructure can come next.

I hope to work on a series of posts that explore collaborative fiction organizing over the next few weeks. If people are interested, that is.

External Links

I came across a blog from the comment on the make emacs better post that I wanted to offer as a link The Babbage Files is a great cyborg/emacs/free software blog.

This is probably not news to anyone, but from John Wiegley I learned about the following two emacs gems that merit mention:

• paredit which makes handling all of the parentheses in Lisp coding much easier.
• redshank, which basically adds a number of tempting systems and associated tools for

And in another direction, I’ve been playing with pylookup. This emacs add on makes it possible to access python documentation from within emacs, from a local copy. The interface is a little bit fiddly but it’s pretty much heaven. More things should work like this.

Onward and Upward!

This document outlines the use of emacs' require and provide functions to help new users understand how to better configure the text editor. While there are a number of different strategies for organizing emacs configuration files and lisp systems and there is no single dominant best practice, consider this document if you find your .emacs or [init.el]{.title-ref}` file growing out of control.

Background and Overview

After using emacs for any period of time, one begins to develop a rather extensive emacs configuration. Emacs comes with very little default configuration and large number of configuration possibilities. Because writers, programmers, and researchers of all persuasions and backgrounds use emacs for a larger array of tasks and work profiles, the need for customization is often quite high. n Rather than have a massive emacs configuration with thousands of lines, I’ve broken the configuration into a number of files that are easier to manage, easier to troubleshoot, and easier to make sense of. These files are then linked together and loaded using emacs' native require function. This document explains that organizational principal and provides the code needed to duplicate my configuration.

I store all of my emacs configuration in a folder that I will refer to as ~/emacs/, in actuality this is a sub-folder within a git repository that I use to store all of my configuration folders, and you should modify this location to suit your own needs. Additionally, I have the habit of prepending the characters tycho- to every function and emacs file name that are my own writing. This namespace trick helps keep my customization separate from emacs' own functions or the functions of loaded packages and prevents unintended consequences in most cases. You might want to consider a similar practice.

Configuring .emacs

My .emacs file is really a symbolic link to the ~/emacs/config/$HOSTNAME.el file. This allows the contents of .emacs to be in version control and if you have your emacs configuration on multiple machines to use the same basic configuration on multiple machines with whatever machine specific configuration you require. To create this symlink, issue the following command: :

ln -s ~/emacs/config/$HOSTNAME.el ~/.emacs

Make sure that all required files and directories exist. My .emacs file is, regardless of it’s actual location, is very minimal because the meat of the configuration is in ~/emacs/tycho-init.el. Take the following skeleton for ~/.emacs: :

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Startup and Behavior Controls
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(setq load-path (cons "~/emacs" load-path))

(setq custom-file "~/emacs/custom.el")
(add-to-list 'load-path "~/emacs/snippet/")
(add-to-list 'load-path "/usr/share/emacs/site-lisp/slime/")

(require 'tycho-display)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Machine Specific Configuration Section
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(tycho-font-medium)
(setq bookmark-default-file "~/garen/emacs/bookmarks/arendt"
  w3m-session-file "~/garen/emacs/bookmarks/w3m-session-arendt"
  bookmark-save-flag 1)

(if (file-directory-p "~/garen/emacs/backup")
(setq backup-directory-alist '(("." . "~/garen/emacs/backup")))
  (message "Directory does not exist: ~/garen/emacs/backup"))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Load the real init
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(require 'tycho-init)

(menu-bar-mode -1)

The first seq defines the load path. Like other configuration paths, this is the directory that emacs will look for files to load when you use require later. load-path does not crawl a directory hierarchy, so if you store emacs lisp within ~/emacs/, you’ll need to add those directories here. To see the value of the load-path use “C-h v” in emacs. I then define “custom.el” as it’s own file to prevent customize from saving configuration in my init file. Then I use require to load a number of display-related functions (from the file ~/emacs/tycho-display.el,) including the tycho-font-medium function.

Then I have a number of machine-specific configuration opens set, mostly to keep multiple machines from overwriting state files.

Finally, I load the file with the real configuration with the (require 'tycho-init) sexp. The configuration is located in the ~/emacs/tycho-init.el file. The file closes with the (menu-bar-mode -1) sexp, which is the last part of the configuration to evaluate and ensures that there isn’t a menu-bar at all.

Require and Provide

require, however, does not simply load .el files in the load path. Rather, the file needs to be announced to emacs. Accomplish this with provide functions in the file. For ~/emacs/tycho-display.el the relevant parts are as follows: :

(provide 'tycho-display)

(defun tycho-font-medium ()
  (interactive)
  (setq default-frame-alist '((font-backend . "xft")
              (font . "Inconsolata-13")
              (vertical-scroll-bars . 0)
              (menu-bar-lines . 0)
              (tool-bar-lines . 0)
              (alpha 86 84)))
  (tool-bar-mode -1)
  (scroll-bar-mode -1))

(global-set-key (kbd "C-c f m") 'tychoish-font-medium)

(setq-default inhibit-startup-message 't
      initial-scratch-message 'nil
      save-place t
      scroll-bar-mode nil
      tool-bar-mode nil
      menu-bar-mode nil
      scroll-margin 0
      indent-tabs-mode nil
      flyspell-issue-message-flag 'nil
      size-indication-mode t
      scroll-conservatively 25
      scroll-preserve-screen-position 1
      cursor-in-non-selected-windows nil)

The provide call, identifies this file as the location of the tycho-display functionality. tycho-font-medium describes the font and display parameters that I called in the .emacs file. And the file ends with a keybiding to call that function and a number of default settings.

Init and Conclusion

While the tycho-init.el file holds all of the interesting configuration options, functions and settings, it’s mostly beyond the scope of this file. When you download contributed emacs.lisp files from emacswiki, put them in ~/emacs/ and put the require call in tycho-init.el. By convention provide names map to file names but be sure to check files to ensure that this is the case.

Using this setup as a framework, you can create--without confusion--a number of configuration files to properly collect and organize your settings, emacs modes, and other emacs code and functions that you’ve gotten from other users. Good luck!

You may also be interested in a couple other tutorials I’ve collected on emacs, notably:

• /posts/running-multiple-emacs-daemons-on-a-single-system
• /posts/decreasing-emacs-start-times

Though short, this week has been pretty good. I’ve been doing cool things at work, I’ve been writing and posting blog entries, and fiction(!), I’m on top of email, and the sweater is growing. I hope this isn’t just a fluke and that I can keep this up and also expand slightly into doing a bit more reading. Small steps.

I’ve done a little bit of work on the wiki and site. Notably, selected entries are mirrored on Planet Emacsen. Also consider the following links to updates on the wiki and other sites:

• Discussion of “/posts/make-emacs-better”, which has been incredibly productive and an interesting discussion of emacs adoption and use by non-programmer niches. I think this discussion and the original post connect pretty well with a post that made the rounds a few weeks ago: Lets Just Use Emacs
• A link from my father on the history of computing. I replied to him with Code Quarterly Interview with Hal Abelson.
• I’ve also collected a few LaTeX System links, of related projects, ideas and collaborators. I’ve also had a few conversations that I need to transcribe into the wiki. Watch this space.
• Then there’s emacs-instapaper, which isn’t exactly a FaiF web service, but the functionality is great for the subway, and I like being able to keep Firefox closed more of the time.

That’s all for now!

I love emacs. I’m also aware that emacs is a really complex piece of software with staggering list of features and functionality. I’d love to see more people use emacs, but the start up and switch cost is nearly prohibitive. I do understand that getting through the “emacs learning curve” is part of what makes the emacs experience so good.

That said, there really ought to be a way to make it easier for people to start using emacs. Think of how much more productive some developers and writers would be if the initial experience of emacs was less overwhelming. And if emacs were easier to use, developers could use emacs as a core (embeded, even) component of text-editing applications, for instance, some sort of specific IDE built with emacs tools, or a documentation creation and editing toolkit built with emacs. I’d go for it, at least.

To my mind there are three major challenges for greater emacs usability. Some of these may be pretty easy to change non-intrusively, others less so. Feedback is, of course, welcome:

1. The biggest problem is that there’s no default configuration. While I appreciate that this provides a neutral substrate for people to customize emacs for themselves, you have to write lisp in order to do pretty much anything in emacs other than write lisp. And customize-mode is well unmentioned, but not particularly usable.

Perhaps one solution to this problem would be to create a facility within emacs to build “distributions,” that come configured for specific kinds of work. That way, emacs can continue to be the way it is, and specialized emacs can be provided and distributed with ease.

2. Improve the customize interface. I like the idea of customize, but I find it incredibly difficult to use and navigate, and end up setting all configuration values manually because that’s easier to keep track of and manage. I’d prefer an option where you configure your emacs instance the way you want (through some sort of conventional menu system), and then have the option of “dumping state” to an arbitrary file that makes a little more sense than the lisp structure that customize produces. Then, as needed, you could load these “state file(s),” But then I’ve never used the menu-bar at all, so perhaps I’m not the best person to design such a system.

This strikes me as a more medium term project, and would make it easier for people who want to modify various basic behaviors and settings. I don’t think that it would need to totally supplant customize, but it might make more sense.

3. Improve and add the ability to extend emacs beyond emacs-lisp. I initially thought emacs-lisp was a liability for emacs adoption and I don’t think this is uncommon, but I’ve since come to respect and understand the utility of emacs lisp. Having said that, I think offering some sort of interopperability between emacs and other languages and interperators, might be a good thing. Ideas like ParrotEmacs and using the Guile VM to run existing emacs-lisp in addition to other new code would be great.

This is a longer term project, of course, but definitely opens emacs up to more people with a much more moderate learning curve.

I’ve been working (slowly) on getting my base configuration into a presentable state that I can push it to a git repository for everyone to see and use, which (at least for me) might start to address problems one and two, but three is outside of the scope of my time and expertise. The truth is that emacs is so great and so close to being really usable for everyone, that a little bit of work on these, and potentially other, enhancements could go a long way toward making emacs better for everyone.

Who’s with me? Let’s talk!

Org-mode is great. Org is this super-task management package that merges outlining and structured document editing abilities with task management glue. It sounds like a weird combination at first, when you realize that it means that you can effectively do work and organize your work in the same set of files without needing to “switch modes,” between a planning/task list interface and a “doing things” interface, the effects are amazing.

I’ll leave “the awesomeness org-mode” to other people and other posts, and spend time here focusing on why “org-mode is awesome but won’t do your work for you,” and “if you’re new to org-mode you will probably want to do things in a certain way, but don’t” issues. Org-mode is great and it can be even better if you avoid developing a few bad habits.

Use `org-capture <http://orgmode.org/manual/Capture.html>`_ as much as possible. Org-capture is a quick interface within emacs that lets you open a temporary buffer, take a few notes, and save the notes into a file. It has advanced features for more complex data entry features and data types. The temptation is to use it as a “quick entry” tool for task list items, but don’t just use it to capture new tasks. Capture links and bookmarks, store notes and important information, If org-mode is your outboard emacs-based brain, then org-capture is it’s main input device. Fighting it, means that your org files end up being less useful.

Avoid using org-mode as a simple task list, and particularly avoid constructing the content in your org files to “game” your agenda view. The agenda compiles a working task list for your actionable notes, working backwards from this means your notes are less than useful things can get lost and all of the really cool things that org lets you do aren’t accessible to you.

This may just be the application of a good general information management practice, but: distribute information evenly throughout all levels of the organizational hierarchy of the file. Use the org-narrow-to-subtree function to focus your current work on a specific portion of a file, and don’t bury information or have it sitting around in one big unorganized pile.

Hope this helps!

Surely I’m not the only person who’s wanted to run multiple distinct instances of the emacs daemon at once. Here’s the use case:

I use one laptop, but I work on a number of very distinct projects many of which involve having a number of different buffers open, most of which don’t overlap with each other at all. This wouldn’t be a huge problem except that I’ve easily gotten up to two hundred buffers open at once. It can get a bit confusing. Particularly since I never really need to touch my work related stuff when I’m writing blags, and my blogging and website design buffers never intersect with fiction writing.

If I weren’t using emacs in daemon mode (that is, invoked with the “emacs --daemon” command) I’d just open separate instances of emacs. The problem with that is, when X11 crashes (as it is so wont to do) the emacs instances crash too and that’s no good. Under normal conditions if you start emacs as a daemon, you can only run one at a time, because it grabs a socket and the emacsclient program isn’t smart enough to be able to decide which instance of emacs you want. So it’s a big ball of failure.

Except I figured out a way to make this work.

In your .emacs file, at the very beginning, put the following line:

(setq server-use-tcp t)

In the default configuration, the emacs daemon listens on a UNIX/system socket. However, in emacs can also, with the above option set, can also listen for connections over TCP. I’ve not yet figured out how to create the required SSH tunnel to make this particularly cool, but it makes this use case possible.

Now, when you start emacs, use commands in the following form:

emacs --daemon=tychoish
emacs --daemon=work

Each server process creates a state file in the “~/.emacs.d/server/” folder. If you are using version control on this file, you may want to consider explicitly ignoring this folder to avoid confusion.

To open an emacs client (i.e. an emacs frame attached to the emacs daemon,) use commands in the following form

emacsclient --server-file=tychoish -c -n
emacsclient --server-file=work -c -n

You may append a file name to open a specific client with one of these emacsclient invocations, or use any of the other emacsclient options. Although these commands are long, I have integrated them into my default zsh configuration as aliases, and as key shortcuts in my window manager. So opening a frame on a specific emacs instance isn’t particularly difficult.

And that’s it. It just works. I have the following two lines in my user’s crontab file:

@reboot    emacs --daemon=tychoish
@reboot    emacs --daemon=work

These lines ensure that the usual (and expected) named emacs daemons are started following reboot. More generally, the @reboot cronjob is great for making the “my computer just rebooted, and now I have to fuss over it for ten minutes before I can work” problem seem much less daunting.

In conclusion I’d like to present one piece of unsolicited advice, and ask a question the assembled.

• Advice: Even though it’s possible to create a large number of emacs instances, and on modern systems the required RAM is pretty low, avoid this temptation. The more emacs instances you have to juggle the greater the chance that you’ll forget what buffers are open in what instance. Undesirable.
• Question: Is there a way to get the value of server-name in into emacs lisp so that I can if things against it? Haven’t figured this one out yet, but it seems like it would be nice for conditionally loading buffers and things like org-mode agenda. Any ideas?

Onward and Upward!

One oft made complaint about emacs is that it takes forever to start up, particularly if you’ve got a lot of packages to load it can take a few seconds for everything to start up. In a lot of respects this is an old problem, that isn’t as relevant given contemporary hardware. Between improvements to emacs, and the fact that computers these days are incredibly powerful, it’s just not a major issue.

Having said that, until recently an emacs instance took as much as 7 seconds to start up. I’ve beaten it down to under two seconds, and using emacsclient and starting emacs with “emacs --daemon” makes the start up time much more manageable.

Step One: Manage your Display Yourself

I’ve written about this before, but really even a 2 second start time feels absurd, if I had to start a new emacs session each time I needed to look into a file. “emacs --daemon” and emacsclient mean that each time you “run” emacs rather than start a new emacs instance, it just opens a new frame on the existing instance. Quicker start up times. It means you can open a bunch of buffers in one frame, settle into work on one file, and then open a second buffer and edit one of the previous files you opened. Good stuff. The quirk is that if you’ve set up your emacs files to load the configuration for your window displays late in game, the windows won’t look right. I have a file in my emacs files called gui-init.el, and it looks sort of like this:

(provide 'gui-init)

(defun tychoish-font-small () (interactive) (setq default-frame-alist ‘((font-backend . “xft”)(font . “Inconsolata-08”) (vertical-scroll-bars . 0) (menu-bar-lines . 0) (tool-bar-lines . 0) (left-fringe . 1) (right-fringe . 1) (alpha 86 84))) (tool-bar-mode -1) (scroll-bar-mode -1) )

(if (string-match “laptop” system-name) (tychoish-font-big))

Modifying, of course, the system name, and the settings to match your tastes and circumstances. The (if) statement allows you to have a couple of these -font- functions defined and then toggle between them based on which machine you load emacs on. Then in your init file (e.g. .emacs), make sure the first two lines are:

(setq load-path (cons "~/confs/emacs" load-path))
(require 'gui-init)

Establish the load path first so that emacs knows where to look for your required files, and then use the (require) sexep to load in the file. Bingo.

Package Things Yourself

We saw this above, but as much as possible avoid using the load function. When you use load emacs has to (I’m pretty sure) do a fairly expensive file system operation and then load the file and then compile and load the file. This takes time. Using the require function is not without it’s own cost, but it does save some time compared to load because it lets you take advantage of the work emacs does with the library loading. At least in my experience.

In your various .el files, insert the following statement:

(provide 'package)

And then in your .emacs, use the following statement

(require 'package)

To load it in. You’re probably already familiar with using these to configure packages that you download. Better yet, don’t require at all, but use the auto-load function. This just creates a little arrow inside of emacs that says “when this function is called, load this file, and hopefully the ‘real’ function by this name will be in there.” This lets you avoid loading packages that you don’t use frequently until you actually need them. The following example provides an auto-load for the identica-mode:

(autoload 'identica-mode "identica-mode.el" "Mode for Updating Identi.ca Microblog" t)

Byte Compile files as much as you can.

Contrary to whatever you’ve been told, emacs isn’t a text editor, as much as it is a virtual machine with a good deal of low level functions established for interacting with text and textual environments and some editing-based interfaces. But really at the core, it’s just virtual machine that interprets a quirky Lisp dialect.

The execution model is pretty simple and straightforward, particularly to people who are used to Java and Python: you load source files, emacs imports them and compiles them half way, they’re not the kind of thing that you could read on your own or would want to write, but it’s not quite machine code either. Byte-compiled files are easier for the machine to read, and quicker to process, but they’re not human intelligible. Then when you need to do something with the function that it’s byte-compiled, emacs compiles it the rest of the way into machine code and executes it. Usually this all happens too fast that we don’t really notice it.

One tried and true means of speeding up emacs load times is to byte-compile files manually so that emacs doesn’t have to do it itself when it loads. The emacs-lisp libraries are byte compiled when emacs installs itself, but your files probably aren’t. Now generally, only byte-compile files that you’re not going to be editing yourself regularly. Byte compiled files have an .elc extension, and as soon as there’s a .el file and a .elc of the same name in a directory, emacs will ignore the .el file even if there have been changes made. To byte compile an emacs-lisp file, simply type M-x to get the execute-extended-command prompt, and then run the function byte-compile (i.e. “M-x byte-compile"). Viola!

I hope these all help you all and lead to a slightly more efficient emacs experience.