Emacs 31.1 is finally out! Unlike earlier Emacs versions, there is not a singular big-bang feature in this release. From what I could gather, the new garbage collector was possibly planned for inclusion in Emacs 31.1, but it has been postponed to Emacs 32. But more on that in a future post; it’s an interesting subject.
Some of the more notable features in Emacs 31.1 are small, quality of life fixes, and one deprecation that marks the end of an era.
As always, my book, Mastering Emacs is 31% off for the next week to celebrate the release also.
The unexec/pdumper controversy and subsequent deprecation
Emacs is… not a normal application. When you compile and link it, you get temacs which is the heart of Emacs but without most of the libraries that ship with it. It’s a bare-bones Emacs with little more than the C core and the interpreter; it’s not really that useful.
To get the Emacs binary you know and love, you have to run temacs and tell it to load the standard library into memory. That is slow. There is a lot of elisp and housekeeping that has to happen. It can take several minutes and a fair bit of CPU and ram to start Emacs this way; it’s untenable.
This has been a problem that has dogged Emacs for decades. It’s not a huge deal today, but back in the day it could break the back on home computers or even shared multi-user environments if a brace of enthusiastic Emacs users all decide to launch Emacs at the same time in the morning.
The solution to this problem? Load it all in once and then literally dump the text/data/bss/etc. segments of Emacs’s memory to a new binary. Do that, and you don’t have to bootstrap all that Emacs lisp state again and again. It feels like a wrestling move almost. You corral a top-heavy Emacs into position and apply The Attitude Adjustment, body slamming Emacs into a new binary, and everything’s all set up and ready to go.
It’s a pretty boss move.
But to make this, uh, wrestling move work, Emacs depended on a number of snowflake functions in glibc. After a couple of decades of enabling this sort of bad behavior the glibc team called it quits, and Emacs had to find another way of doing it.
Daniel Colascione built a much better solution, though not everyone was happy about it, that — put simply — standardizes the serialization of Emacs’s internal structures into something that is not a 1:1 dump of its internal memory structures.
The portable dumper’s been the default for a number of years now. It was first introduced around ten years ago, and in keeping with Emacs’s long history of backwards compatibility, the old unexec dumper was kept around ostensibly for the one or two users who found the idea of a portable dumper risible or unworkable.
But now it is finally gone for good. The end of an era.
User Lisp Directory
Classic problem: you git clone or download an Emacs package somewhere and now you want it to work. But how? It’s not that trivial; there are quite a few competing ways of doing it. The simplest one is to tell users to drop their package into the user-lisp/ in your .emacs.d directory and Emacs will sort out loading and setting up autoload (so the right stuff appears in M-x.)
Minibuffer and Completions
Emacs 30.1 gained completion-preview-mode, a native “pop-up window” system not unlike Company and Corfu, but more attuned to Emacs’s own way of doing things: using the *Completions* window instead of a floating child frame like Company and friends.
Emacs 31.1 builds on that with a wide range of customizable options you’re sure to want to customize if you want to go native.
Rotating Window Layouts
M-x window-layout-rotate-clockwise (see C-x w C-h for the manifold new options) and suchlike rotate your window layouts. Another little UI winner.
Exchanging the point and mark without activating the region
I’ve talked (mostly in the my book) about how transient-mark-mode is a rather awkward one-size-fits-all that was draped over Emacs’s multitude of “region-affecting” commands, like kill-region (C-w).
So C-x C-x, that exchanges point and mark, also activates the region whether you want it to or not. Fixing the mark commands in transient mark mode is an old article of mine where I demonstrate how to do exactly that. But now there’s a builtin option to not have it do that — sweet.
Tree-sitter now offers to install its grammars for you
Two blockers work in tandem to hold back the wider adoption of tree-sitter in Emacs:
The fact that TS demands a special major mode to work; and that said mode is often a thread-bare re-implementation of the original.
That installing grammars, especially on Windows, is a giant pain in the neck, as you have to not only thread the needle with the exacting ABI version of the tree-sitter library itself, but also ensure you just the exacting version of each language grammar, or everything goes up in smoke.
The former is still a problem, but the latter is now mostly resolved. Emacs can now finally offer to install the right language grammar for TS modes it knows about.
Now there’s no excuse not to try out Combobulate: Structured Movement and Editing with Tree-Sitter.
and so much more
Lots of little tweaks and changes. Have a read.
Installation Changes in Emacs 31.1
unexec dumper removed. The traditional unexec dumper, deprecated since Emacs 27, has been removed.
The portable dumper now works on m68k a.out targets.
As I wrote in the introduction at the top, this is indeed the end of an era.
Emacs’s old ‘ctags’ program is no longer built or installed. You are encouraged to use Universal Ctags <https://ctags.io/> instead. For now, to get the old ‘ctags’ behavior you can can run ‘etags –ctags’ or use a shell script named ‘ctags’ that runs ‘etags –ctags “$@“’.
If you’re a TAGS user you should check with where and make sure you’ve got a newer one installed. (If you don’t know if you use TAGS, you do not.)
Changed GCC default options on 32-bit x86 systems. When using GCC 4 or later to build Emacs on 32-bit x86 systems, ‘configure’ now defaults to using the GCC options ‘-mfpmath=sse’ (if the host system supports SSE2) or ‘-fno-tree-sra’ (if not). These GCC options work around GCC bug 58416, which can cause Emacs to behave incorrectly in rare cases.
New configure option ‘–with-systemduserunitdir’. This allows specifying the directory where the user unit file for systemd is installed; the default is ‘${prefix}/usr/lib/systemd/user’.
You can tell Emacs to install a systemd service to run Emacs’s server that way. I recommend doing this.
Startup Changes in Emacs 31.1
In compatible terminals, ‘xterm-mouse-mode’ is turned on by default. For these terminals the mouse will work by default. A compatible terminal is one that supports Emacs setting and getting the OS selection data (a.k.a. the clipboard) and mouse button and motion events. With ‘xterm-mouse-mode’ enabled, you must use Emacs keybindings to copy to the OS selection instead of terminal-specific keybindings.
You can keep the old behavior by customizing ‘xterm-mouse-mode’ to nil.
Most people do not know this but Emacs added mouse support to terminal Emacs years ago but left it off. Terminal capabilities vary widely so that was a nice and safe decision. But now it just works as you’d expect it to: menus are clickable and so forth. Good stuff.
site-start.el is now loaded before the user’s early init file. Previously, the order was early-init.el, site-start.el and then the user’s regular init file, but now site-start.el comes first. This allows site administrators to customize things that can normally only be done from early-init.el, such as adding to ‘package-directory-list’.
If you’re on a single user system like your laptop or home computer, this is unlikely to matter much to you.
New User Lisp directory feature. If you have a subdirectory “user-lisp/” in your Emacs configuration directory, then Lisp files in it and any subdirectories are now recursively byte-compiled, scraped for autoload cookies and added to ‘load-path’.
You can disable the feature by setting ‘user-lisp-auto-scrape’ to nil, and you can customize the option ‘user-lisp-directory’ to process some other directory instead. There is also a new command ‘prepare-user-lisp’ that you can invoke at any time. See the Info node “(emacs) User Lisp Directory” for more details.
Oh this is so useful. I have been cargo culting the same snippets of code around for 23 years to load directories with my stuff in it; yes use-package helps but it’s still a lot of manual hassle. About time!
The first client frame now shows warnings from daemon startup. When there are warnings emitted during Emacs startup, usually due to problems in your initialization file, these are shown in a “*Warnings*” buffer. Until now such warnings were not made visible in the case that Emacs was started as a daemon. Now the first frame after daemon startup will show the “*Warnings*” buffer. So for example, starting Emacs with a command like ‘emacsclient -a “” -c’ will now show “*Warnings*” just like a plain invocation of ‘emacs’ would.
Bad news. Emacs’s insistence on telling you about every minor stubbed toe in some random package will now plague you even if you’re running Emacs as a daemon. Such a cursed feature. Nobody cares. If it was important it’d be an error.
Changes in Emacs 31.1
‘line-spacing’ now supports specifying spacing above the line. Previously, only spacing below the line could be specified. The user option can now be set to a cons cell to specify spacing both above and below the line, which allows you to vertically center text.
This is a global value to all of Emacs, it’s not a face setting, so you cannot use M-x customize-face to change it. Set it with setopt or customize ui.
New face ‘margin’ for the window margin display. A new basic face ‘margin’ is used by default for text displayed in the left and right margin areas, which are used by various packages for per-line annotations. Its background defaults to the frame default background, so existing behavior is unchanged for users who do not customize this new face.
Display strings shown in the margins now inherit unspecified face attributes from the ‘margin’ face, if the string itself does not fully specify its face. If your code relied on the face of the underlying buffer text to serve as a default for any unspecified face attributes of strings displayed in the margin, you must now apply those face attributes to the margin string itself using ‘propertize’.
‘prettify-symbols-mode’ attempts to ignore undisplayable characters. Previously, such characters would be rendered as, e.g., white boxes.
‘standard-display-table’ now has more extra slots. ‘standard-display-table’ has been extended to allow specifying glyphs that are used for borders around child frames and menu separators on TTY frames.
Call the command ‘standard-display-unicode-special-glyphs’ to set up the ’standard-display-table’s extra slots with Unicode characters. See the documentation of that command to see which slots of the display table it changes.
Child frames are now supported on TTY frames. This supports use-cases like Posframe, Corfu, and child frames acting like tooltips. To enable tooltips on TTY frames, call ‘tty-tip-mode’.
The presence of child frame support on TTY frames can be checked with ’(featurep ‘tty-child-frames)’.
Recent versions of Posframe and Corfu are known to use child frames on TTYs if they are supported.
This is a welcome change for terminal users. Frames in the terminal do not work as they do in GUI — they behave more like tmux/screen “windows”. Here child frames are just inset popups like the ones you find in GUI Emacs.
Several font-lock face variables are now obsolete. The following variables are now obsolete: ‘font-lock-builtin-face’, ‘font-lock-comment-delimiter-face’, ‘font-lock-comment-face’, ‘font-lock-constant-face’, ‘font-lock-doc-face’, ‘font-lock-doc-markup-face’, ‘font-lock-function-name-face’, ‘font-lock-keyword-face’, ‘font-lock-negation-char-face’, ‘font-lock-preprocessor-face’, ‘font-lock-string-face’, ‘font-lock-type-face’, ‘font-lock-variable-name-face’, and ‘font-lock-warning-face’.
These variables contributed both to confusion about the relation between faces and variables, and to inconsistency when major mode authors used one or the other (sometimes interchangeably). We always recommended using faces directly, and not creating variables going by the same name.
If you have customized these variables, you should now customize the corresponding faces instead, using something like:
M-x customize-face RET font-lock-string-face RET
If you have been using these variables in Lisp code (for example, in font-lock rules), simply quote the symbol, to use the face directly instead of its now-obsolete variable.
Note this is not about the faces but about variables named the same as the faces. Yeah that is confusing. Emacs has faces like font-lock-string-face that you probably have customized already. But it also has variables named the same as the faces. The variables are deprecated.
If you have configured your faces with M-x customize-face (you should!) you have nothing to worry about.
New char-table ‘special-mirror-table’ for mirroring special glyphs. This char-table is used to mirror special glyphs (truncation and continuation) when the user has defined an alternative representation for those characters via display tables.
find-func.el commands now have history enabled. The ‘find-function’, ‘find-library’, ‘find-face-definition’, and ‘find-variable’ commands now allow retrieving previous input using the usual minibuffer history commands. Each command has a separate history.
Huh. I never noticed they did not have their own history; now they do. That is good to know I guess but unlikely to affect me much.
New minor mode ‘find-function-mode’ replaces ‘find-function-setup-keys’. The new minor mode defines the keys at a higher precedence level than the old function, one more usual for a minor mode. To restore the old behavior, customize ‘find-function-mode-lower-precedence’ to non-nil.
You’re unlikely to have much of a need to customize this.
‘find-function’ can now find ‘cl-defmethod’ invocations inside macros.
New minor mode ‘prettify-special-glyphs-mode’. The new minor mode prettifies the special character glyphs (truncation and continuation) on TTY frames (and GUI frames without fringes). You can customize the associated new face ‘special-glyphs’.
Minibuffer and Completions
Support for immediate display of the “*Completions*” buffer. Whenever a minibuffer with completion is opened, then if the completion table sets the ‘eager-display’ completion property to non-nil, the “*Completions*” buffer will now be displayed immediately. This property can be overridden for different completion categories by customizing ‘completion-category-overrides’. Alternatively, the new user option ‘completion-eager-display’ can be set to t to force eager display of “*Completions*” for all minibuffers, or nil to suppress this for all minibuffers.
Support for updating “*Completions*” as you type. If the “*Completions*” buffer is displayed and the completion table sets the completion property ‘eager-update’ to non-nil, then the “*Completions*” buffer will be updated as you type. This property can be overridden for different completion categories by customizing ‘completion-category-overrides’. Alternatively, the new user option ‘completion-eager-update’ can be set to t to make “*Completions*” always be updated as you type, or nil to suppress this always. Note that for large or inefficient completion tables, this can slow down typing.
‘RET’ chooses the completion selected with ‘M-<UP>/M-<DOWN>’. If a completion candidate is selected with ‘M-<UP>’ or ‘M-<DOWN>’, typing ‘RET’ will exit completion with that candidate as the result. This works both in minibuffer completion and for in-buffer completion. This feature supersedes ‘minibuffer-completion-auto-choose’, which previously provided similar behavior; that variable is now nil by default.
This goes hand in hand with the changes in Emacs 30.1 to make Emacs’s minibuffer completion system behave a little bit more like traditional company/corfu-style completers.
I really rate these new inclusions but I do warn they require a fair bit of customization to really get them to behave like something that does not get in your way.
Support for completion category inheritance. You can now define completion categories that inherit properties from existing categories, using the new function ‘define-completion-category’.
New optional value of ‘minibuffer-visible-completions’. If the value of this option is ‘up-down’, only the ‘<UP>’ and ‘<DOWN>’ arrow keys move point between candidates shown in the “*Completions*” buffer display, while ‘<RIGHT>’ and ‘<LEFT>’ arrows move point in the minibuffer.
New user option ‘completion-pcm-leading-wildcard’. This option configures how the partial-completion style does completion. It defaults to nil, which preserves the existing behavior. When it is set to t, the partial-completion style behaves more like the substring style, in that the input can match a candidate anywhere in the candidate string.
Another minor tweak to a completion style to make it behave more like something it once did. Emacs has a diverse set of completion styles. The default have changed a lot over the years, sometimes to the chagrin of people who were used to the quirks of a now-relegated default style. For example there’s both an emacs21 and an emacs22 completion style in completion-styles-alist. But see Understanding Minibuffer Completion for more information.
‘completion-styles’ now can contain lists of bindings. In addition to a symbol naming a completion style, an element of ‘completion-styles’ can now be a list of the form ‘(STYLE ((VARIABLE VALUE) …))’ where STYLE is a symbol naming a completion style. VARIABLE will be bound to VALUE (without evaluating it) while the style is executing. This allows multiple references to the same style with different values for completion-affecting variables like ‘completion-pcm-leading-wildcard’ or ‘completion-ignore-case’. This also applies to the styles configuration in ‘completion-category-overrides’ and ‘completion-category-defaults’.
Oh man. That is niche. completion-styles is a shopping list of how Emacs must match things in stuff like the minibuffer’s completer. Now you can make it so initials ignores case but substring does not.
Navigating “*Completions*” now accommodates ‘completions-format’. When ‘completions-format’ is set to ‘vertical’, typing ‘n’, ‘TAB’ or ‘M-<DOWN>’ in the “*Completions*” buffer (the latter also in the minibuffer) now moves point to the completion candidate in the next line in the current column, and wraps to the next column after the last completion candidate of the current column. Likewise, typing ‘p’, ‘S-TAB’ or ‘M-<UP>’ moves point to the completion candidate in the previous line or wraps to the previous column. Previously, these keys ignored the vertical format, i.e., they moved point only to the item in the same line of the next or previous column, in accordance with the default horizontal format. In the vertical format, typing ‘<LEFT>’ and ‘<RIGHT>’ in the “*Completions*” buffer (and when ‘minibuffer-visible-completions’ is non-nil, also in the minibuffer) moves point only within the current line, analogously to how, in the horizontal format, ‘<DOWN>’ and ‘<UP>’ move point only within the current column.
You’ll want to configure this for sure if you are intent on using the Completions buffer and window for in-buffer completion. I always found navigating between the tabular structure in completions to be a bit weird and offputting; it’s a good use of space, for sure, but a flat list of matches is much easier to reason about.
Selected completion candidate is preserved across “*Completions*” updates. When the window point is on a completion candidate in the “*Completions*” buffer (because of ‘minibuffer-next-completion’ or for any other reason), it will remain on that candidate after the “*Completions*” is updated with a new list of completions. The candidate is deselected when the “*Completions*” buffer is hidden.
“*Completions*” is now displayed faster when there are many candidates. As before, if there are more completion candidates than can be displayed in the current frame, only a subset of the candidates is displayed. This process is now faster: only that subset of the candidates is actually inserted into “*Completions*” until you run a command which interacts with the text of the “*Completions*” buffer. This optimization only applies when ‘completions-format’ is ‘horizontal’ or ‘one-column’.
New user option ‘crm-prompt’ for ‘completing-read-multiple’. This option configures the prompt format of ‘completing-read-multiple’. By default, the prompt indicates to the user that the completion command accepts a comma-separated list. The prompt format can include the separator description and the separator string, which are both stored as text properties of the ‘crm-separator’ regular expression.
It’s a pretty rare feature, that. You can “toggle-select” multiple matches from the minibuffer; few things use it, to be honest. I find the user experience rather poor if I am perfectly honest, no matter the completer. Helm is one of the few tools I think that does it well.
For a practical example of multi-select see Fuzzy Finding with Emacs Instead of fzf.
New user option ‘completion-preview-sort-function’. This option controls how Completion Preview mode sorts completion candidates. If you use this mode together with an in-buffer completion popup interface, such as the interfaces that the GNU ELPA packages Corfu and Company provide, you can set this option to the same sort function that your popup interface uses for a more integrated experience.
(‘completion-preview-sort-function’ was already present in Emacs 30.1, but as a plain Lisp variable, not a user option.)
New user option ‘completion-preview-inhibit-functions’. This option provides fine-grained control over Completion Preview mode activation. You can use it to specify arbitrary conditions in which to inhibit the mode’s operation.
Another thing you’ll want to customize. You may want certain movement commands like those used in paredit or combobulate commands to not trigger the completion window.