OK, this is a weird post, but after reading a post about running emacs with systemd, ¹ I’ve realized that the my take on how I run and manage processes is a bit unique, and worth enumerating and describing. The short version is that I regularly run multiple instances of emacs, under systemd, in daemon mode, and it’s pretty swell. Here’s the rundown:

• On Linux systems, I use a build of emacs with the lucid toolkit, rather than GTK, because of this bug with GTK, which I should write more about at some point. Basically, if the X session crashes with GTK emacs, even if you don’t have windows open, the daemon will crash, even if the dameon isn’t started from a GUI session. The lucid toolkit doesn’t have this problem.

• I run the process under my user’s systemd instance, rather than under the PID 1 systemd instance. I like keeping things separate. Run the following command to ensure that your user’s systemd will start at boot rather than at login: :

  sudo loginctl enable-ligner $(whoami)
  

• I have a systemd service file named emacs@.service in my ~/.config/systemd/user/ directory that looks like this: :

  [Unit]
  Description=Emacs-tychoish: the extensible, self-documenting text editor.
  
  [Service]
  Type=forking
  ExecStart=/usr/bin/emacs --daemon=%i --chdir %h
  ExecStop=/usr/bin/emacsclient --server-file=hud --eval "(progn (setq kill-emacs-hook 'nil) (kill-emacs))"
  Restart=always
  TimeoutStartSec=0
  
  [Install]
  WantedBy=default.target
  

  I then start emacs dameons: :

  systemctl --user start emacs@work
  systemctl --user start emacs@personal
  systemctl --user start emacs@chat
  

  To enable them so that they start following boot: :

  systemctl --user enable emacs@work
  systemctl --user enable emacs@personal
  systemctl --user enable emacs@chat
  
  Though to be honest, I use different names for daemons.
  

• I have some amount of daemon specific code, which might be useful: :

  (setq server-use-tcp t)
  
  (if (equal (daemonp) nil)
  (setq tychoish-emacs-identifier "solo")
    (setq tychoish-emacs-identifier (daemonp)))
  
  ;; this makes erc configs work less nosily. There's probably no harm in
  ;; turning down the logging
  (if (equal (daemonp) "chat")
  (setq gnutls-log-level 0)
    (setq gnutls-log-level 1))
  
  (let ((csname (if (eq (daemonp) nil)
       "generic"
       (daemonp))))
    (setq recentf-save-file (concat user-emacs-directory system-name "-" csname "-recentf"))
    (setq session-save-file (concat user-emacs-directory system-name "-" csname "-session"))
    (setq bookmark-default-file (concat user-emacs-directory system-name "-" csname "-bookmarks"))
    (setq helm-c-adaptive-history-file (concat user-emacs-directory system-name "-" csname "--helm-c-adaptive-history"))
    (setq desktop-base-file-name (concat system-name "-" csname "-desktop-file"))
    (setq desktop-base-lock-name (concat system-name "-" csname "-desktop-lock")))
  

  Basically this just sets up some session-specific information to be saved to different files, to avoid colliding per-instance.

• Additionally, I use the tychoish-emacs-identifier from above to provide some contextual information as to what emacs daemon/window I’m currently in: :

  (setq frame-title-format '(:eval (concat tychoish-emacs-identifier ":" (buffer-name))))
  
  (spaceline-emacs-theme 'daemon 'word-count)
  (spaceline-define-segment daemon tychoish-emacs-identifier)
  

  Also, on the topic of configuration, I do have a switch statement that loads different mu4e configurations in different daemons.

• To start emacs sessions, I use operations in the following forms: :

  # create a new emacs frame. Filename optional.
  emacsclient --server-file=<name> --create-frame --no-wait <filename>
  
  # open a file in an existing (last-focus) frame/window. Filename required.
  emacsclient --server-file=<name> --no-wait <filename>
  
  # open a terminal emacs mode. Filename optional .
  emacsclient --server-file=<name> --tty --no-wait <filename>
  

  That’s a lot to type, so I use aliases in my shell profile: :

  alias e='emacsclient --server-file=personal --no-wait'
  alias ew='emacsclient --server-file=personal --create-frame --no-wait'
  alias et='emacsclient --server-file=personal --tty'
  

  I create a set of aliases for each daemon prefixing e/ew/et with the first letter of the daemon name.

And that’s about it. When I’ve used OS X, I’ve managed something similar using launchd but the configuration files are a bit less elegant. On OS X, I tend to install emacs with cocoa toolkit, using homebrew.

Using multiple daemons is cool, though not required, for a number of reasons:

• you can have good separation between personal things and professional/work things, which is always nice, but particularly gratifying during the pandemic when it’s easy to work forever.
• Managing multiple separation of email. While mu4e has profiles and contexts, and that’s great, I like a firmer boundary, and being able maintain separate email databases.
• Running emacs lisp applications that do a lot of networking, or do other blocking operations. The main case where this matters in my experience is running big erc instance, or something else that isn’t easily broken into async/subprocesses.

I’ve been using emacs for a long time, and it’s mostly pretty unexceptional. A few months ago, I started using LSP mode, and it’s been pretty great… mostly. This post reflects on some of the dustier corners, and some of the broader implications.

History and Conflict

Language Server is a cool idea, that (as far as I know) grew out of VS Code, and provides a way to give a text editor “advanced, IDE-like” features without having to implement those features per-language in the text editor. Instead, editor developers implement generic integration for the protocol, and language developers produce a single process that can process source code and supports the editor features. Sounds great!

There are challenges:

• The LSP protocol uses JSON for encoding data between the server and emacs. This is a reasonable choice, and makes development easy, but it’s not a perfect fit. Primarily, in emacs, until version 27, which hasn’t reached an official release yet (but is quite usable on Linux), parsed JSON in lisp rather than C, which made everything rather slow, but emacs 27 helps a lot. Second,

• There is some overhead for the lsp, particularly as you attempt to scale very large numbers of files or very large files. Emacs is really good with large files, as an artifact of having had to support editing larger files on computers 20 or 30 years ago, and lsp feels sluggish in these situations.

  I think some of this overhead is probably depends on the implementation of the language server’s themselves, and some percentage is probably protocol itself: http’s statelessness is great in big distributed web services, and for the local use cases where responsiveness matters, it’s less good. Similarly JSON’s transparency and ubiquity is great, but for streaming streaming-type use cases, perhaps less so.

I’ve not poked around the protocol, or any implementations, extensively, so I’m very prepared to be wrong about these points.

There’s also some tension about moving features, like support for adding editor support for a language, out of lisp and into an external process. While I’m broadly sympathetic, and think there are a bunch of high-quality language integrations in emacs lisp, and for basic things, the emacs lisp ones might be faster. Having said that, there’s a lot of interface inconsistency between different support for languages in emacs, and LSP support feels spiffy my comparisons. Also at least for python (jedi,) go (gocode), and C/C++ (irony,) lisp (slime,) the non-lsp services were already using external processes for some portion of their functionality.

It’s also the case that if people spend their emacs-lisp-writing time doing things other than writing support for various development environments it’s all for the better anyway.

Suggestions / Notes

• things will break or get wedged, call lsp-restart-workspace to restart the server (I think.) It helps.
• use lsp-file-watch-ignored to avoid having the language servers process emacs process files from vendored code, intermediate/generated code, or similar, in cases where you might have a large number of files that aren’t actually material to your development work
• I believe some (many? all?) servers are sourced from the PATH (e.g. clang/llvm, gopls), which means you are responsible for updating them and their dependencies.
• I’ve found it useful to update lsp-mode itself more often than I typically update other emacs packages. I also try and keep the language servers as up to date as possible in coordination with lsp-mode updates.
• If Emacs 27 can work on your system, it’s noticeably faster. Do that.
• If you have an existing setup for a language environment, the LSP features end up layering over existing functionality, and that can have unexpected results.

My Configuration

None of this is particularly exciting, but if you use use-package, then the following might be an interesting starting point. I stole a lot of this from other places, so shrug.

I must say that I don’t really use dap or treemacs, because I tend to keep things pretty barebones.

(use-package helm-lsp
  :ensure t
  :after (lsp-mode)
  :commands (helm-lsp-workspace-symbol)
  :init (define-key lsp-mode-map [remap xref-find-apropos] #'helm-lsp-workspace-symbol))

(use-package lsp-mode
  :diminish (lsp-mode . "lsp")
  :bind (:map lsp-mode-map
("C-c C-d" . lsp-describe-thing-at-point))
  :hook ((python-mode . #'lsp-deferred)
(js-mode . #'lsp-deferred)
(go-mode-hook . #'lsp-deferred))
  :init
  (setq lsp-auto-guess-root t       ; Detect project root
lsp-log-io nil
lsp-enable-indentation t
lsp-enable-imenu t
lsp-keymap-prefix "C-l"
lsp-file-watch-threshold 500
lsp-prefer-flymake nil)      ; Use lsp-ui and flycheck

  (defun lsp-on-save-operation ()
(when (or (boundp 'lsp-mode)
     (bound-p 'lsp-deferred))
  (lsp-organize-imports)
  (lsp-format-buffer))))

(use-package lsp-clients
  :ensure nil
  :after (lsp-mode)
  :init (setq lsp-clients-python-library-directories '("/usr/local/" "/usr/")))

(use-package lsp-ui
  :ensure t
  :after (lsp-mode)
  :commands lsp-ui-doc-hide
  :bind (:map lsp-ui-mode-map
     ([remap xref-find-definitions] . lsp-ui-peek-find-definitions)
     ([remap xref-find-references] . lsp-ui-peek-find-references)
     ("C-c u" . lsp-ui-imenu))
  :init (setq lsp-ui-doc-enable t
     lsp-ui-doc-use-webkit nil
     lsp-ui-doc-header nil
     lsp-ui-doc-delay 0.2
     lsp-ui-doc-include-signature t
     lsp-ui-doc-alignment 'at-point
     lsp-ui-doc-use-childframe nil
     lsp-ui-doc-border (face-foreground 'default)
     lsp-ui-peek-enable t
     lsp-ui-peek-show-directory t
     lsp-ui-sideline-update-mode 'line
     lsp-ui-sideline-enable t
     lsp-ui-sideline-show-code-actions t
     lsp-ui-sideline-show-hover nil
     lsp-ui-sideline-ignore-duplicate t)
  :config
  (add-to-list 'lsp-ui-doc-frame-parameters '(right-fringe . 8))

  ;; `C-g'to close doc
  (advice-add #'keyboard-quit :before #'lsp-ui-doc-hide)

  ;; Reset `lsp-ui-doc-background' after loading theme
  (add-hook 'after-load-theme-hook
   (lambda ()
     (setq lsp-ui-doc-border (face-foreground 'default))
     (set-face-background 'lsp-ui-doc-background
              (face-background 'tooltip))))

  ;; WORKAROUND Hide mode-line of the lsp-ui-imenu buffer
  ;; @see https://github.com/emacs-lsp/lsp-ui/issues/243
  (defadvice lsp-ui-imenu (after hide-lsp-ui-imenu-mode-line activate)
(setq mode-line-format nil)))

;; Debug
(use-package dap-mode
  :diminish dap-mode
  :ensure t
  :after (lsp-mode)
  :functions dap-hydra/nil
  :bind (:map lsp-mode-map
     ("<f5>" . dap-debug)
     ("M-<f5>" . dap-hydra))
  :hook ((dap-mode . dap-ui-mode)
(dap-session-created . (lambda (&_rest) (dap-hydra)))
(dap-terminated . (lambda (&_rest) (dap-hydra/nil)))))

(use-package lsp-treemacs
  :after (lsp-mode treemacs)
  :ensure t
  :commands lsp-treemacs-errors-list
  :bind (:map lsp-mode-map
     ("M-9" . lsp-treemacs-errors-list)))

(use-package treemacs
  :ensure t
  :commands (treemacs)
  :after (lsp-mode))

For a while, I’ve been envious of some of the project and file navigation features in emacs for browsing bigger projects/programs, things like imenu and tags have always seems awesome but given that I spend most of time editing restructured text and markdown files (I’m a technical writer), these tools have been distant and not a part of my day to day work.

It’s not that it would be impossible to write interfaces for imenu or etags, for the formats I use regularly, but more that I’ve never gotten around to it until now.

We’re still a ways away on the question of etags, but it turns out that when I wasn’t looking rst mode got imenu support, and with the following little bit of elisp you can get imenu for markdown.

(setq markdown-imenu-generic-expression
   '(("title"  "^\\(.*\\)[\n]=+$" 1)
     ("h2-"    "^\\(.*\\)[\n]-+$" 1)
     ("h1"   "^# \\(.*\\)$" 1)
     ("h2"   "^## \\(.*\\)$" 1)
     ("h3"   "^### \\(.*\\)$" 1)
     ("h4"   "^#### \\(.*\\)$" 1)
     ("h5"   "^##### \\(.*\\)$" 1)
     ("h6"   "^###### \\(.*\\)$" 1)
     ("fn"   "^\\[\\^\\(.*\\)\\]" 1)
))

(add-hook 'markdown-mode-hook
   (lambda ()
      (setq imenu-generic-expression markdown-imenu-generic-expression)))

Pretty awesome! I hope it helps you make awesome things.

In no particular order:

Org Mode Guilt and a Lisp Function

I have some guilt about having mostly forsaken org-mode,¹ in particular because I was watching Sacha Chua’s chat with John Wiegley, and I think both are such nifty hackers, and have done so many things that are pretty darn nifty.

I liked what I heard about johnw’s org mode setup so much that I might give it a try again. But in the mean time, I wanted to make my “recompile my tasklist function” to be a bit more clean. The result is follows:

(defun tychoish-todo-compile ()
   (interactive)
   (if (get-buffer "*todo-compile*")
       (progn
          (switch-to-buffer-other-window (get-buffer "*todo-compile*"))
          (recompile))
       (progn
          (compile "make -j -k -C ~/wiki")
          (switch-to-buffer-other-window "*compilation*")
          (rename-buffer "*todo-compile*")))
       (revbufs))

Notables:

• This is the first time I’ve used progn which is somewhat embarrassing, but it’s a great thing to have in the toolkit now. Link: progn
• I hadn’t realized until now that there wasn’t an else-if form in emacs lisp. Weird, but it makes sense.
• Compilation Mode is pretty much my current favorite thing in emacs.
• revbufs is this amazing thing that reverts buffers if there aren’t local modifications, and also reports to you if a buffer has changed outside of emacs and there are local modifications. So basically “does everything you want without destroying anything and then tells you what you need to do manually.” Smart. Simple. Perfect.

I might need to “macro-ize” this, as I have a lot of little compile processes for which I’d like to be able to trigger/maintain unique compile buffers. That’s a project for another day.

Emacs Thoughts

I’m even thinking about putting together a post about how, although I’m a diehard emacs user, and I’ve spent a fair bit of time learning how to do really great things with emacs, there are a lot of vim-ish things in my workflow:

• I read email with mutt and I’ve tried to get into GNUS, and I try it again every now and then, but I always find it so unbelievably gnarly. At least the transition. Same with Notmuch, which I like a lot more (in theory,) but I find the fact that Notmuch and mutt have this fundamental misunderstanding about what constitutes a “read” email, to be tragic.

• I use a crazy ikiwiki + deft + makefile setup for task tracking. As (obliquely) referenced above.

  I might give org another shot, and I’ve been looking at task warrior, but the sad truth is that what I have works incredibly well for in most cases, and switching is hard.

• I tend jump to a shell window to do version control and other things, even though I’m familiar with magit and dired, my use of these tools is somewhat spotty.

I learned about artbollocks-mode.el from Sacha Chua’s post, and it’s pretty freaking amazing.

Basically, it does some processing of your writing--while you work--to highlight passive sentences and affected jargon.¹ And that’s all. There are some functions for generating statistics about your writing, but I find I don’t use that functionality often. You can enable it all of the time, or just turn it on when you’re doing editing.

After a few weeks, I’ve noticed a marked improvement in the quality of my output. I leave it on all the time, but I’m pretty good at resisting the urge to edit while I’m writing. Or at least I’m pretty good at picking up again after going back to tweak a wording. In general it’s hard to keep more than a few things in an editing pass at any time.

It turns out that the instant feedback on passive sentences, even though it’s not perfect, is great for improving the quality of my content the first time out. And it’s even better for doing editing work. It’s harder to ignore a passive sentence when the editor is highlighting you see a screen full of them for you.

It’s of course important to be able to ignore its suggestions from time to time, and it’s no harder to ignore than “flyspell-mode” (the on-the-fly spell checker in emacs.)

I’ve been subject to a rather annoying emacs bug for months. Basically, when you start emacs with the --daemon switch, and the X11 session exits, and any emacs frames are open, then the emacs process dies. No warning. The whole point, to my mind, of the daemon mode is to allows emacs sessions to persist beyond the current X11 session.

This shouldn’t happen. I think this is the relevant bug report, but I seem to remember that the issue has something to do with the way that GTK interacts with the X11 session and emacs’s frames. It’s something of a deadlock: the GTK has no real need to fix the bug (and/or it’s a behavior that they rely on for other uses,) and it might not really be possible or feasible for emacs to work around this issue.¹

I also think that it’s probably fair to say that daemon mode represent a small minority all emacs usage.

Regardless, I’ve figured out the workaround:

Turns out, it’s totally possible to build GNU emacs without GTK, by using the “Lucid” build. Which is to say, use the windowing system kit built for Lucid Emacs (i.e. XEmacs,) rather than GTK. I was able, using the code below, to get an emacs experience with the new build that seems identical² to the one I used to get with GTK, except without the frustrating crashes every time that X11 spazzed when I decided to unplug a monitor or some such. A welcome improvement, indeed.

The following emacs-lisp covers all of the relevant configuration of the “look and feel” of my emacs session. Install the Inconsolata font if you haven’t already, you’ll be glad you did.

(setq-default inhibit-startup-message 't
              initial-scratch-message 'nil
              save-place t
              scroll-bar-mode nil
              tool-bar-mode nil
              menu-bar-mode nil
              scroll-margin 0
              indent-tabs-mode nil
              flyspell-issue-message-flag 'nil
              size-indication-mode t
              scroll-conservatively 25
              scroll-preserve-screen-position 1
              cursor-in-non-selected-windows nil)

(setq default-frame-alist '((font-backend . "xft")
                            (font . "Inconsolata-14")
                            (vertical-scroll-bars . 0)
                            (menu-bar-lines . 0)
                            (tool-bar-lines . 0)
                            (alpha 86 84)))

(tool-bar-mode -1)
(scroll-bar-mode -1)
(menu-bar-mode -1)

Hope this helps you and/or anyone else that might have run into this problem.

I’m a huge fan of emacs' org-mode on so many levels: as an IDE for knowledge workers, as a task management system, as a note taking system, and as the ideal basic mode for so many tasks. However, I’ve been bucking against org-for a number of tasks recently. The end result is that I’m becoming less org-dependent. This post is a reflection on how I’ve changed the way I work, and how my thinking has changed regarding org-mode.

Fair warning: this is a really geeky post that has a somewhat specialized context. If you’re lost or bored. check back later in the week.

The Perils of Org

The problem I keep running into with org is that I really don’t prefer to work in org-mode.¹ Org is great and very flexible, but I don’t like that it means that all text-based work is dependent on emacs. My brain is already wired for Markdown and reStructured Text from years of blogging and work projects respectively.

And then there’s this organization problem. There are two ways you can organize content in org-mode. The first is to just dump every thing in one org-mode file and use the hierarchical outlining to impose organization to organize everything. The second is to have every project inside of it’s own file and use outlining incidentally as the project needs it. Content aggregation happens in the agenda.

The problem with the “large files” approach is that you end up with a small handful of files with thousands of lines and imposing useful organization is difficult (too many levels and things get buried; not enough and inevitably your headings aren’t descriptive enough and you get confused. Furthermore, I end up living in clone-indirect-buffer-other-window’d and org-narrow-to-subtree’d buffers, which is operationally the same as having multiple files it just takes longer to set up.

The problem with the other approach, having lots of different files, is that I have a hard time remembering what is in each file, or in logically splitting big projects into multiple files. The agenda does help with this, but the truth is that the kinds of org-headings for organization and tasks are not always the same kinds of headings that make sense for the project itself. I often need more tasks than organizational divides in a project. I tried this approach a couple of times, and ended up with useless mush in my files.

Typically, I can never make the “lots of file approach” really work, and the big files problem lead me to general avoidance of everything. Not good. The key to success here is good aggregation tools.

Hodgepodge

In response, I’ve made a couple of tweaks to how I’m doing… pretty much everything. That is:

• I’ve moved most of my open projects into a locally ruining and compiling ikiwiki instance. Both laptops have this setup, and there’s a central remote to keep both (all?) machines in sync.
• I’m using ikiwiki tasklist to basically replicate the functions of org-agenda. Basically this crawls the entire wiki looking for lines that begin with certain keywords and generates a “todo” page based on these notes. Really simple, incredibly useful and it solves much of my aggregation needs.
• I still have some stuff in org-mode: notes for the nearly-finished novel, lots of random old (legacy) data, 12 various open tasks, and org-capture. I’m thinking of pointing various org-capture templates at files in the wiki but haven’t gotten there yet.
• I’ve basically taken the “lots of little files,” approach to my writing and work. I’ve not over-leaded the system yet. Each major project gets a page in the root level of the wiki for overview and planing, and then sub-pages for all related project files (if/as needed)
• It turns out that the markdown-mode for emacs has gotten a few improvements since the last time I downloaded the file, including better support for wiki-links that are mostly compatible with ikiwiki. Also from the same developer deft which implements a pretty nifty incremental search for text files in a given directory. So between these tools, ikiwiki, and the ikiwiki-tasklist there’s support for the most important things.
• In terms of publishing, beyond ikiwiki for tychoish.com and the personal organization instance, I have a couple of other smaller wikis (also ikiwiki powered,) and I’ve been playing with Sphinx as publishing for more structured documents and resources (i.e. documentation, novels, and collections,) particularly those that need multiple formats and presentations.

I’m sure there will be more shifts in the future, I’m sure. I think this is a good start. Thoughts?

The ideal writing application is emacs, at least for me. In the absence of emacs (as on a tablet,) I’ve been thinking about what features I actually need in a writing application. While I’ve grown to admire the power of a full Lisp machine in my text editor, I accept that it’s not, strictly speaking required. Here’s a first stab at the list of requirements. Feel free to comment or submit a patch to this page.

• Simple, primarily full screen editing.
• The ability edit very large files, 100kbs should present no issue.
• Some sort of syntax highlight during the editing process, preferably support for LaTeX, Markdown, and org-mode.
• Word count generation for the entire files and for current selections.
• Auto-save as crash protection.
• Undo/Redo last typing action.

Nice to have (but not crucial) features:

• The ability to edit one file and reference another (or potentially edit) at the same time. Bonus points for being able to switch between to parts of a single file at once.
• The ability to hide or collapse some sections of a file.
• Optional spell checking.
• Parenthetical and double-quote matching.
• Soft and hard line/word wrapping.

Other than that I don’t think there’s anything that I really need to have to get writing done 90% of the time. How about you?