So I made a thing, or at least, I took a thing I’ve built and made it presentable for other people. I’m talking, of course, about my Emacs configuration.

Check it out at github.com/tychoish/.emacs.d! The README is pretty complete, and giving it a whirl is simple, just do something like: :

mv ~/.emacs.d/ ~/emacs.d-archive
git clone --recurse-submodules git@github.com:tychoish/.emacs.d.git ~/.emacs.d/

If you’re using Emacs 27, you might also be able to clone it into ~/.config/emacs, for similar effect. It doesn’t matter much to me. Let me know what you think!

The tl;dr of “what’s special about this config,” is that, it has:

• a good set of defaults out of the box.
• a lot of great wiz-bang features enabled and configured.
• very quick start times, thanks to lazy-loading of libraries.
• great support for running as a daemon, and even running multiple daemons at once.

I’ve sporadically produced copies of my emacs config for other folks to use, but those were always ad hoc, and difficult to reproduce and maintain, and I’ve been working on and off to put more polish on things, so making it usable for other people seemed like a natural step at this point.

I hope other people find it useful, but also I think it’s a good exercise for me in keeping things well maintained, and it doesn’t bother me much one way or another.

Discussion

I think a lot about developer experience: I’ve spent my career, thus far, working on infrastructure products (databases, CI systems, build tools, release engineering,) that help improve developer experience and productivity, and even my day-to-day work tends to focus mostly on problems that impact the way that my teams write software: system architecture, service construction, observability, test infrastructure, and similar.

No matter how you slice it, editor experience is a huge factor in people’s experience, and can have a big impact on productivity, and there are so many different editors with wildly different strengths. Editors experience is also really hard: unlike other kinds of developer infrastructure (like buildsystems, or even programming languages) the field conceptualizes editors as personal: your choice in editor is seen to reflect on you (it probably doesn’t), and your ability to use your editor well is seen to be a reflection of your skills as a developer (it isn’t). It’s also the case that because editors are so personal, it’s very difficult to produce an editor with broad appeal, and the most successful editors tend to be very configurable and can easily lack defaults, which means that even great editors are terrible out of the box, which mostly affects would-be and new developers.

Nevertheless, time being able to have an editor that you’re comfortable with and that you can use effectively without friction does make it easier to build software, so even if folks often conceptualize of editors in the wrong way, improving the editing experience is important. I think that there are two areas for improvement:

• editor configurations should--to some extent--be maintained at the project (or metaproject) level, rather than on the level individual engineer. The hard part here is how to balance individual preference with providing a consistent experience, particularly for new developers.¹
• there should be more “starter kits” (like this one! or the many other starter kits, but also for (neo)vim, vscode, and others.) that help bootstrap the experience, while also figuring out ways to allow layering other project-based extensions on top of a base configuration.

Also, I want to give chemacs a shout out, for folks who want to try out other base configurations.

What a year, am I right?

I’ve avoided writing a more “journal”-type post because there’s both too much to say and somehow not enough all at the same time. This is a tough time to be a human, and I’ve been incredibly lucky in many regards: I can stay busy, I have a living situation that makes quarantining easy and tolerable, and while I wish the produce at my local grocery had better produce, it’s nice to live in a very walkable and accessible neighborhood. Even though “nowtimes” are fundamentally different from the “beforetimes,” the speed at which things become routine--if not normal--is almost disturbing. Delightfully, though, I’ve felt like I’m still getting things done. In no particular order:

• As the title of the post suggests I have firmly entered the home pickling phase of quarantine, and have begun fermenting rather a lot of cauliflower, with additional vegetables to follow. Frankly I don’t know why I didn’t think of this earlier: I love pickles and it’s really hard to get compelling pickles for things that aren’t cucumbers. My first batch is cauliflower, and I’m also excited to do things like: baby okra, cabbage, greenbeans, and maybe some of my favorite root vegetables (shallots, raddishes, horseradish, jerusalem artichokes, etc.) Details forcoming, perhaps.
• I’ve begun cooking more often. My go to, is to make a pot of chicken stock, and then figure out things to use the stock to cook. I’ve done some nice proto-beef stew things, and have made some pretty great lentil dishes. Nothing fancy or complex. I’m also delightfully, getting to a point where I can produce a meal that consists of more than one flavor profile, which is nice.
• I’ve written enough common lisp that writing little bits of software in CL seem doable. It’s still a bit harder to get into bigger projects, and I have a lot to learn still, but I’ve crossed a line, and it’s nice to have an additional skill.
• I replaced the video card in my home desktop, and I feel like I have a totally new computer! It’s amazing. I got this computer in 2013, and have mostly been using laptops for the past few years. Before quarantine, I was even planning to de-accession it, but I’ve been using it a lot more and the new video card really changes things. Since I don’t do a lot of graphics things and my computer use is not particularly performance intensive, I forget how much having an underpowered GPU can really impact perceptions: I was suffering through a lot of rendering-hangs and glitches that just don’t happen any more.
• I’ve been writing blog posts more, and am working on a plan to have something that resembles a publication schedule. I wrote professionally for so long, that writing for fun never seemed like a viable way to spend time, but as time goes on, it’s clear that I really want to be able to experiment with ideas and reflect on things in a more regular way, and continuing to write posts here seems like a good way to do that. My lisp-related posts are now appearing on Planet Lisp, and I’m interested in expanding my reach… slightly.
• I completed a sort of long term project to modernize the way that I use and configure emacs: I got all of the hardcoded weirdness undercontrol, I started using use-package, and I got lsp-mode to work, and I fixed a number of hilarious little bugs in how my config worked. The end result, is I get to use the editor I want in the way that I want, and it all starts up on my (old) computers in well under 2 seconds, not that I start up all that often. I expect I’ll write a bit more on this soon.
• Since quarantine started I’ve gone through an entire complete box of Chemex filters (plus the tail end of a box, and I’m a few pots into the next box.) I started drinking coffee during the summer of 2015, and have primarily consumed coffee produced by other people (i.e. at work) in that time. While I’ve definitely gone through a few boxes of Chemex filters over the years, my new rate is noticeable. But quarantine means making your own coffee, and apparently I’ve made a lot of coffee. It feels like a milestone.

Alternate title: yolo lentils

I’ve been cooking more in the last few months that I have, basically ever. I’m not a bad cook, and my instincts and raw skills are pretty decent, but I’m not super inspired, I guess. Left to my own devices I eat the same few things again and again. So the exercise of mostly cooking for myself has been a lot about figuring out things to cook that taste good (and different!) and that I can reproduce reliably. Sometimes I’m more successful than other times, and frankly a lot of what I’ve come up with isn’t super inspiring, but I recently stumbled upon something that was pretty great:

Ingredients:

• 1 Cup of Black Lentils
• 2 Cups of Chicken Stock
• 1 Onion chopped reasonably finely.
• 3 cloves of garlic sliced.
• 1 inch piece of fresh ginger microplaned or minced.
• (some) Olive Oil
• Spices: Sweet Curry Powder, Garam Masala, Tumeric
• Salt and Pepper

Procedure:

• Saute garlic onion, and ginger in olive oil until soft and becoming translucent, adding spices near the end. (5 minutes.)
• Add stock and lentils, and cook until lentils soften. I did the whole thing in an instant pot for 8-10 minutes and let the pressure release naturally (without keep warm). I’m sure other kinds of pots and cooking methods work well too.

That’s it.

Modifications and notes:

• I put the spices in at the end of the sauteing phase, and I might have put them in slightly earlier, but it’s fine.
• I like that black lentils retain their form after cooking, but if you like more mushy texture other lentils would be fine.
• If you don’t use a pressure cooker, it seems reasonable that more liquid would be useful.
• I used chicken stock because it’s delicious and I had just made a batch of right before I made the lentils so it seemed reasonable. The stock flavor is secondary to the spices.
• I didn’t measure the spices (which were not particularly fresh, I must confess,) but I listed them roughly in terms of amount.

Enjoy!

OK, this is a weird post, but after reading a post about running emacs with systemd, ¹ I’ve realized that the my take on how I run and manage processes is a bit unique, and worth enumerating and describing. The short version is that I regularly run multiple instances of emacs, under systemd, in daemon mode, and it’s pretty swell. Here’s the rundown:

• On Linux systems, I use a build of emacs with the lucid toolkit, rather than GTK, because of this bug with GTK, which I should write more about at some point. Basically, if the X session crashes with GTK emacs, even if you don’t have windows open, the daemon will crash, even if the dameon isn’t started from a GUI session. The lucid toolkit doesn’t have this problem.

• I run the process under my user’s systemd instance, rather than under the PID 1 systemd instance. I like keeping things separate. Run the following command to ensure that your user’s systemd will start at boot rather than at login: :

  sudo loginctl enable-ligner $(whoami)
  

• I have a systemd service file named emacs@.service in my ~/.config/systemd/user/ directory that looks like this: :

  [Unit]
  Description=Emacs-tychoish: the extensible, self-documenting text editor.
  
  [Service]
  Type=forking
  ExecStart=/usr/bin/emacs --daemon=%i --chdir %h
  ExecStop=/usr/bin/emacsclient --server-file=hud --eval "(progn (setq kill-emacs-hook 'nil) (kill-emacs))"
  Restart=always
  TimeoutStartSec=0
  
  [Install]
  WantedBy=default.target
  

  I then start emacs dameons: :

  systemctl --user start emacs@work
  systemctl --user start emacs@personal
  systemctl --user start emacs@chat
  

  To enable them so that they start following boot: :

  systemctl --user enable emacs@work
  systemctl --user enable emacs@personal
  systemctl --user enable emacs@chat
  
  Though to be honest, I use different names for daemons.
  

• I have some amount of daemon specific code, which might be useful: :

  (setq server-use-tcp t)
  
  (if (equal (daemonp) nil)
  (setq tychoish-emacs-identifier "solo")
    (setq tychoish-emacs-identifier (daemonp)))
  
  ;; this makes erc configs work less nosily. There's probably no harm in
  ;; turning down the logging
  (if (equal (daemonp) "chat")
  (setq gnutls-log-level 0)
    (setq gnutls-log-level 1))
  
  (let ((csname (if (eq (daemonp) nil)
       "generic"
       (daemonp))))
    (setq recentf-save-file (concat user-emacs-directory system-name "-" csname "-recentf"))
    (setq session-save-file (concat user-emacs-directory system-name "-" csname "-session"))
    (setq bookmark-default-file (concat user-emacs-directory system-name "-" csname "-bookmarks"))
    (setq helm-c-adaptive-history-file (concat user-emacs-directory system-name "-" csname "--helm-c-adaptive-history"))
    (setq desktop-base-file-name (concat system-name "-" csname "-desktop-file"))
    (setq desktop-base-lock-name (concat system-name "-" csname "-desktop-lock")))
  

  Basically this just sets up some session-specific information to be saved to different files, to avoid colliding per-instance.

• Additionally, I use the tychoish-emacs-identifier from above to provide some contextual information as to what emacs daemon/window I’m currently in: :

  (setq frame-title-format '(:eval (concat tychoish-emacs-identifier ":" (buffer-name))))
  
  (spaceline-emacs-theme 'daemon 'word-count)
  (spaceline-define-segment daemon tychoish-emacs-identifier)
  

  Also, on the topic of configuration, I do have a switch statement that loads different mu4e configurations in different daemons.

• To start emacs sessions, I use operations in the following forms: :

  # create a new emacs frame. Filename optional.
  emacsclient --server-file=<name> --create-frame --no-wait <filename>
  
  # open a file in an existing (last-focus) frame/window. Filename required.
  emacsclient --server-file=<name> --no-wait <filename>
  
  # open a terminal emacs mode. Filename optional .
  emacsclient --server-file=<name> --tty --no-wait <filename>
  

  That’s a lot to type, so I use aliases in my shell profile: :

  alias e='emacsclient --server-file=personal --no-wait'
  alias ew='emacsclient --server-file=personal --create-frame --no-wait'
  alias et='emacsclient --server-file=personal --tty'
  

  I create a set of aliases for each daemon prefixing e/ew/et with the first letter of the daemon name.

And that’s about it. When I’ve used OS X, I’ve managed something similar using launchd but the configuration files are a bit less elegant. On OS X, I tend to install emacs with cocoa toolkit, using homebrew.

Using multiple daemons is cool, though not required, for a number of reasons:

• you can have good separation between personal things and professional/work things, which is always nice, but particularly gratifying during the pandemic when it’s easy to work forever.
• Managing multiple separation of email. While mu4e has profiles and contexts, and that’s great, I like a firmer boundary, and being able maintain separate email databases.
• Running emacs lisp applications that do a lot of networking, or do other blocking operations. The main case where this matters in my experience is running big erc instance, or something else that isn’t easily broken into async/subprocesses.

I’ve been using emacs for a long time, and it’s mostly pretty unexceptional. A few months ago, I started using LSP mode, and it’s been pretty great… mostly. This post reflects on some of the dustier corners, and some of the broader implications.

History and Conflict

Language Server is a cool idea, that (as far as I know) grew out of VS Code, and provides a way to give a text editor “advanced, IDE-like” features without having to implement those features per-language in the text editor. Instead, editor developers implement generic integration for the protocol, and language developers produce a single process that can process source code and supports the editor features. Sounds great!

There are challenges:

• The LSP protocol uses JSON for encoding data between the server and emacs. This is a reasonable choice, and makes development easy, but it’s not a perfect fit. Primarily, in emacs, until version 27, which hasn’t reached an official release yet (but is quite usable on Linux), parsed JSON in lisp rather than C, which made everything rather slow, but emacs 27 helps a lot. Second,

• There is some overhead for the lsp, particularly as you attempt to scale very large numbers of files or very large files. Emacs is really good with large files, as an artifact of having had to support editing larger files on computers 20 or 30 years ago, and lsp feels sluggish in these situations.

  I think some of this overhead is probably depends on the implementation of the language server’s themselves, and some percentage is probably protocol itself: http’s statelessness is great in big distributed web services, and for the local use cases where responsiveness matters, it’s less good. Similarly JSON’s transparency and ubiquity is great, but for streaming streaming-type use cases, perhaps less so.

I’ve not poked around the protocol, or any implementations, extensively, so I’m very prepared to be wrong about these points.

There’s also some tension about moving features, like support for adding editor support for a language, out of lisp and into an external process. While I’m broadly sympathetic, and think there are a bunch of high-quality language integrations in emacs lisp, and for basic things, the emacs lisp ones might be faster. Having said that, there’s a lot of interface inconsistency between different support for languages in emacs, and LSP support feels spiffy my comparisons. Also at least for python (jedi,) go (gocode), and C/C++ (irony,) lisp (slime,) the non-lsp services were already using external processes for some portion of their functionality.

It’s also the case that if people spend their emacs-lisp-writing time doing things other than writing support for various development environments it’s all for the better anyway.

Suggestions / Notes

• things will break or get wedged, call lsp-restart-workspace to restart the server (I think.) It helps.
• use lsp-file-watch-ignored to avoid having the language servers process emacs process files from vendored code, intermediate/generated code, or similar, in cases where you might have a large number of files that aren’t actually material to your development work
• I believe some (many? all?) servers are sourced from the PATH (e.g. clang/llvm, gopls), which means you are responsible for updating them and their dependencies.
• I’ve found it useful to update lsp-mode itself more often than I typically update other emacs packages. I also try and keep the language servers as up to date as possible in coordination with lsp-mode updates.
• If Emacs 27 can work on your system, it’s noticeably faster. Do that.
• If you have an existing setup for a language environment, the LSP features end up layering over existing functionality, and that can have unexpected results.

My Configuration

None of this is particularly exciting, but if you use use-package, then the following might be an interesting starting point. I stole a lot of this from other places, so shrug.

I must say that I don’t really use dap or treemacs, because I tend to keep things pretty barebones.

(use-package helm-lsp
  :ensure t
  :after (lsp-mode)
  :commands (helm-lsp-workspace-symbol)
  :init (define-key lsp-mode-map [remap xref-find-apropos] #'helm-lsp-workspace-symbol))

(use-package lsp-mode
  :diminish (lsp-mode . "lsp")
  :bind (:map lsp-mode-map
("C-c C-d" . lsp-describe-thing-at-point))
  :hook ((python-mode . #'lsp-deferred)
(js-mode . #'lsp-deferred)
(go-mode-hook . #'lsp-deferred))
  :init
  (setq lsp-auto-guess-root t       ; Detect project root
lsp-log-io nil
lsp-enable-indentation t
lsp-enable-imenu t
lsp-keymap-prefix "C-l"
lsp-file-watch-threshold 500
lsp-prefer-flymake nil)      ; Use lsp-ui and flycheck

  (defun lsp-on-save-operation ()
(when (or (boundp 'lsp-mode)
     (bound-p 'lsp-deferred))
  (lsp-organize-imports)
  (lsp-format-buffer))))

(use-package lsp-clients
  :ensure nil
  :after (lsp-mode)
  :init (setq lsp-clients-python-library-directories '("/usr/local/" "/usr/")))

(use-package lsp-ui
  :ensure t
  :after (lsp-mode)
  :commands lsp-ui-doc-hide
  :bind (:map lsp-ui-mode-map
     ([remap xref-find-definitions] . lsp-ui-peek-find-definitions)
     ([remap xref-find-references] . lsp-ui-peek-find-references)
     ("C-c u" . lsp-ui-imenu))
  :init (setq lsp-ui-doc-enable t
     lsp-ui-doc-use-webkit nil
     lsp-ui-doc-header nil
     lsp-ui-doc-delay 0.2
     lsp-ui-doc-include-signature t
     lsp-ui-doc-alignment 'at-point
     lsp-ui-doc-use-childframe nil
     lsp-ui-doc-border (face-foreground 'default)
     lsp-ui-peek-enable t
     lsp-ui-peek-show-directory t
     lsp-ui-sideline-update-mode 'line
     lsp-ui-sideline-enable t
     lsp-ui-sideline-show-code-actions t
     lsp-ui-sideline-show-hover nil
     lsp-ui-sideline-ignore-duplicate t)
  :config
  (add-to-list 'lsp-ui-doc-frame-parameters '(right-fringe . 8))

  ;; `C-g'to close doc
  (advice-add #'keyboard-quit :before #'lsp-ui-doc-hide)

  ;; Reset `lsp-ui-doc-background' after loading theme
  (add-hook 'after-load-theme-hook
   (lambda ()
     (setq lsp-ui-doc-border (face-foreground 'default))
     (set-face-background 'lsp-ui-doc-background
              (face-background 'tooltip))))

  ;; WORKAROUND Hide mode-line of the lsp-ui-imenu buffer
  ;; @see https://github.com/emacs-lsp/lsp-ui/issues/243
  (defadvice lsp-ui-imenu (after hide-lsp-ui-imenu-mode-line activate)
(setq mode-line-format nil)))

;; Debug
(use-package dap-mode
  :diminish dap-mode
  :ensure t
  :after (lsp-mode)
  :functions dap-hydra/nil
  :bind (:map lsp-mode-map
     ("<f5>" . dap-debug)
     ("M-<f5>" . dap-hydra))
  :hook ((dap-mode . dap-ui-mode)
(dap-session-created . (lambda (&_rest) (dap-hydra)))
(dap-terminated . (lambda (&_rest) (dap-hydra/nil)))))

(use-package lsp-treemacs
  :after (lsp-mode treemacs)
  :ensure t
  :commands lsp-treemacs-errors-list
  :bind (:map lsp-mode-map
     ("M-9" . lsp-treemacs-errors-list)))

(use-package treemacs
  :ensure t
  :commands (treemacs)
  :after (lsp-mode))

Last week, I did a release, I guess, of cl-grip which is a logging library that I wrote after reflecting on common lisp logging earlier. I wanted to write up some notes about it that aren’t covered in the read me, and also talk a little bit4 about what else I’m working on.

cl-grip

This is a really fun and useful project and it was really the right size for a project for me to really get into, and practice a bunch of different areas (packages! threads! testing!) and I think it’s useful to boot. The read me is pretty comprehensive, but I thought I’d collect some additional color here:

Really at it’s core cl-grip isn’t a logging library, it’s just a collection of interfaces that make it easy to write logging and messaging tools, which is a really cool basis for an idea, (I’ve been working on and with a similar system in Go for years.)

As result, there’s interfaces and plumbing for doing most logging related things, but no actual implementations. I was very excited to leave out the “log rotation handling feature,” which feels like an anachronism at this point, though it’d be easy enough to add that kind of handler in if needed. Although I’m going to let it stew for a little while, I’m excited to expand upon it in the future:

• additional message types, including capturing stack frames for debugging, or system information for monitoring.
• being able to connect and send messages directly to likely targets, including systemd’s journal and splunk collectors.
• a collection of more absurd output targets to cover “alerting” type workloads, like desktop notifications, SUMP, and Slack targets.

I’m also excited to see if other people are interested in using it. I’ve submitted it to Quicklisp and Ultralisp, so give it a whirl!

See the cl-grip repo on github.

Eggqulibrium

At the behest of a friend I’ve been working on an “egg equilibrium” solver, the idea being to provide a tool that can given a bunch of recipes that use partial eggs (yolks and whites) can provide optimal solutions that use a fixed set of eggs.

So far I’ve implemented some prototypes that given a number of egg parts, attempt collects recipes until there are no partial eggs in use, so that there are no leftovers. I’ve also implemented the “if I have these partial eggs, what can I make to use them all.” I’ve also implemented a rudimentary CLI interface (that was a trip!) and a really simple interface to recipe data (both parsing from a CSV format and an in memory format that makes solving the equilibrium problem easier.)

I’m using it as an opportunity to learn different things, and find a way to learn more about things I’ve not yet touched in lisp (or anywhere really,) so I’m thinking about:

• building a web-based interface using some combination of caveman, parenscript, and related tools. This could include features like “user submitted databases,” as well as links to the sources the recpies, in addition to the basic “web forms, APIs, and table rendering.”
• storing the data in a database (probably SQLite, mostly) both to support persistence and other more advanced features, but also because I’ve not done database things from Lisp at all.

See the eggquilibrium repo on github it’s still pretty rough, but perhaps it’ll be interesting!'

Other Projects

• Writing more! I’m trying to be less obsessive about blogging, as I think it’s useful (and perhaps interesting for you all too.) I’ve been writing a bunch and not posting very much of it. My goal is to mix sort of grandiose musing on technology and engineering, with discussions of Lisp, Emacs, and programming projects.
• Working on producing texinfo output from cl-docutils! I’ve been toying around with the idea of writing a publication system targeted at producing books--long-form non-fiction, collections of essays, and fiction--rather than the blogs or technical resources that most such tools are focused on. This is sort of part 0 of this process.
• Hacking on some Common Lisp projects, I’m particularly interested in the Nyxt and StumpWM.

I’ve made the decision to make all of personal project code that I write to do in Common Lisp. See this post for some of the background for this decision.

It didn’t take me long to say “I think I need a logging package,” and I quickly found this wonderful comparsion of CL logging libraries, and only a little longer to be somewhat disappointed.

In general, my requirements for a logger are:

• straightforward API for logging.
• levels for filtering messages by importance
• library in common use and commonly available.
• easy to configure output targets (e.g. system’s journal, external services, etc).
• support for structured logging.

I think my rationale is pretty clear: loggers should be easy to use because the more information that can flow through the logger, the better. Assigning a level to all log messages is great for filtering practically, and it’s ubiquitous enough that it’s really part of having a good API. While I’m not opposed to writing my own logging system,¹ but I think I’d rather not in this case: there’s too much that’s gained by using the conventional choice.

Configurable outputs and structured logging are stretch goals, but frankly are the most powerful features you can add to a logger. Lots of logging work is spent crafting well formed logging strings, when really, you just want some kind of arbitrary map and makes it easier to make use of logging at scale, which is to say, when you’re running higher workloads and multiple coppies of an application.

Ecosystem

I’ve dug in a bit to a couple of loggers, sort of using the framework above to evaluate the state of the existing tools. Here are some notes:

log4cl

My analysis of the CL logging packages is basically that log4cl is the most mature and actively maintained tool, but beyond the basic fundamentals, it’s “selling” features are… interesting.² The big selling features:

• integration with the developers IDE (slime,) which makes it possible to use the logging system like a debugger, almost. This is actually a phenomenal feature, particularly for development and debugging. The downside is that it wouldn’t be totally unreasonable to use it production, and that’s sort of terrifying.
• it attempts to capture a lot of information about logging call sites so you can better map back from log messages to the state of the system when the call was made. Again, this makes it a debugging tool, and that’s awesome, but it’s overhead, and frankly I’ve never found it difficult to grep through code.
• lots of attention to log rotation and log file management. There’s not a lot of utility in writing log data to files directly. In most cases you want to write to standard out: the program is being used interactively, and users may want to see the log of what happens. In cases where you’re running in a daemon mode, any more you’re not, systemd or similar just captures your output. Even then, you’re probably in a situation where you want to send the log output to some other process (e.g. an external service, or some kind of local socket for fluentd/etc.)
• hierarchical organization of log messages is just less useful than annotation with metadata, in practice, and using hierarchical methods to filter logs into different streams or reduce logging volumes just obscures things and makes it harder to understand what’s happening in a system.

Having said that, the API surface area is big, and it’s a bit confusing to just start using the logger.

a-cl-logger

The acl package is pretty straightforward, and has a lot of features that I think are consistent with my interests and desires:

• support for JSON output,
• internal support for additional output formats (e.g. logstash,)
• more simple API

It comes with a couple of pretty strong draw backs:

• there are limited testing.
• it’s SBCL only, because it relies on SBCL fundamentals in collecting extra context about log messages. There’s a pending pull request to add ECL compile support, but it looks like it wouldn’t be quite that simple.
• the overhead of collecting contextual information comes at an overhead, and I think logging should err on the side of higher performance, and making expensive things optional, just because it’s hard to opt into/out of logging later.

Conclusion

So where does that leave me?

I’m not really sure.

I created a scratch project to write a simple logging project, but I’m definitely not prioritizing working on that over other projects. In the mean time I’ll probably end up just not logging very much, or maybe giving log4cl a spin.

Notes

I’ve been intermittently working on a common lisp library to produce a binary encoding of arbitrary objects, and I think I’m going to be abandoning the project. This is an explanation of that decision and an reflection on my experience.

Why Common Lisp?

First, some background. I’ve always thought that Common Lisp is a language with a bunch of cool features and selling points, but unsurprisingly, I’ve never really had the experience of writing more than some one-off bits of code in CL, which isn’t surprising. This project was a good experience for really digging into writing and managing a conceptually larger project which was a good kick in the pants to learn more.

The things I like:

• the implementations of the core runtime are really robust and high quality, and make it possible to imagine running your code in a bunch of different contexts. Even though it’s a language with relatively few users, it feels robust in a way. The most common implementations also have ways of producing fully self contained static binaries (like Go, say), which makes the thought of distributing software seem reasonable.
• quicklisp, a package/library management tool is new (in the last decade or so,) has really raised the level of the ecosystem. It’s not as complete as I’d expect in many ways, but quicklisp changed CL from something quaint to something that you could actually imagine using.
• the object system is really nice. There isn’t quite compile time-type checking on the values of slots (attributes) of objects, though you can opt in. My general feeling is that I can pretty easily get the feeling of writing statically typed code with all of the freedom of writing dynamic code.
• multiple dispatch, and the conceptual approach to genericism, is amazing and really simplifies flow control in a lot of cases. You implement the methods you need, for the appropriate types/objects and then just write the logic you need, and the function call machinery just does the right thing. There’s surprisingly little conditional logic, as a result.

Things I don’t like:

• there are all sorts of things that don’t quite have existing libraries, and so I find myself wanting to do things that require more effort than necessary. This project to write a binary encoding tool would have been a component in service of a much larger project. It’d be nice if you could skip some of the lower level components, or didn’t have your design choices so constrained by gaps in infrastructure.
• at the same time, the library ecosystem is pretty fractured and there are common tools around which there aren’t really consensus. Why are there so many half-finished YAML and JSON libraries? There are a bunch of HTTP server (!) implementations, but really you need 2 and not 5?
• looping/iteration isn’t intuitive and difficult to get common patterns to work. The answer, in most cases is to use (map) with lambdas rather than loops, in most cases, but there’s this pitfall where you try and use a (loop) and really, that’s rarely the right answer.
• implicit returns seem like an over sight, hilariously, Rust also makes this error. Implicit returns also make knowing what type a function or method returns quite hard to reason about.

Writing an Encoder

So the project I wrote was an attempt to write really object oriented code as a way to writing an object encoder to a JSON-like format. Simple enough, I had a good mental model of the format, and my general approach to doing any kind of binary format processing is to just write a crap ton of unit tests and work somewhat iteratively.

I had a lot of fun with the project, and it gave me a bunch of key experiences which make me feel comfortable saying that I’m able to write common lisp even if it’s not a language that I feel maximally comfortable in (yet?). The experiences that really helped included:

• producing enough code to really have to think about how packaging and code organization worked. I’d written a function here and there, before, but never something where I needed to really understand and use the library/module/packaging (e.g. systems and libraries.) infrastructure.
• writing and running lots of tests. I don’t always follow test-driven development closely, but writing lots of tests is part of my process, and being able to watch the layers of this code come together was a lot of fun and very instructive.
• this project for me, was mostly about API design and it was nice to have a project that didn’t require much design in terms of the actual functionality, as object encoding is pretty straight forward.

From an educational perspective all of my goals were achieved.

Failure Mode

The problem is that it didn’t work out, in the final analysis. While the library I constructed was able to encode and decode objects and was internally correct, I never got it to produce encoding that other implementations of the same specification could reliably read, and the ability to read data encoded by other libraries only worked in trivial cases. In the end:

• this was mostly a project designed to help me gain competence in a programming language I don’t really know, and in that I was successful.
• adding this encoding format isn’t particularly useful to any project I’m thinking of working on in the short term, and doesn’t directly enable me to do anything in particular.
• the architecture of the library would not be particularly performant in practice, as the encoding process didn’t deploy a buffer pool of any kind, and it would have been harder than not to back fill that in, and I wasn’t particularly interested in that.
• it didn’t work, and the effort to debug the issue would be more substantive than I’m really interested in doing at this point, particularly given the limited utility.

So here we are. Onto a different project!