Shadow files are Emacs' built in file synchronization system. They let you link two files or directories together such that when a change occurs in one, that file is synced with the other. For example, this could be used to create a backup on a separate drive. By using a Tramp connection, the files can be synchronized across a network of machines.
The documentation on this feature is light, at least that I could find, and the feedback the code provides when you make a mistake is minimal. As it stands, the implementation has some peculiarities that make it a bit awkward to use. If you require a more complete synchronization tool, SyncThing is a solid choice.
This is not a complete guide on what is built in, I only got the basics working but that was enough for me. On that note, I have only tested this on Linux (WSL2) so some of the syntax may be different for other operating systems.
For the design of this very website, I used a few Emacs plugins to add Typescript support as well as code formatting. Even with the necessary back-ends installed on my remote server, I could not get the packages to work across a Tramp connection. With shadow files, I can edit a local copy of the website, taking advantage of the tools, and then sync the changes with the server. This process is so seem-less that running npm run dev naturally updates the website after changes as if they were made locally.
There are three main variables used in the synchronization of files:
Each of these lists has a command to define a new element, M-x shadow-define-X. These commands offer little in the way of assistance as the auto-fill suggestions are plain wrong. I found it easier to define the variables manually, as you'll see below, and leave it at that.
To begin, require shadow-files in your configuration setup:
(require 'shadowfile) (add-hook 'after-save-hook 'shadow-add-to-todo)
Additionally, the block adds a hook calling shadow-add-to-todo after a file is saved. This checks if edits were made in a file that has a shadow equivalent, and if so, sets up the process to copy the changes over.
I have the definitions of the shadow lists stored in a file named shadows in my .emacs.d directory. Maintaining the variable definitions in a separate file is potentially an artifact of my attempts to make shadow files work, but I see no reason to change it now. These settings are then loaded by the init.el file with
(load-file "~/.emacs.d/shadows")
Shadow clusters are shortcut names for systems that contain files you want to link. For example, you may name your main computer HomeBase as follows:
(setq shadow-clusters '((shadow-cluster "HomeBase" "/MyComputer:" "\\`/MyComputer:\\'")))
where MyComputer is whatever follows your username in the terminal user@MyComputer. The three inputs to the function shadow-cluster are the name the cluster will go by, the file path, a regexp that matches to the file path. The variable shadow-clusters, note the ending s, maintains a list of clusters. Clusters also work over Tramp connections:
(setq shadow-clusters '((shadow-cluster "RemoteServer" "/ssh:user@123.45.67.890:" "\\`/ssh:user@123\\.45\\.67\\.890:\\'") (shadow-cluster "Homebase" "/MyComputer:" "\\`/MyComputer:\\'")))
Shadow literals directly connect two specific files. These files can have different paths and different names, however, if one of them is saved in Emacs, you will be prompted to sync them.
(setq shadow-literal-groups '(("/RemoteServer:/remote-path-to-file/remote-file.md" "/HomeBase:/home-path-to-file/home-file.md")))
The above example demonstrates the use of clusters to identify where the files are located. The clusters do not need to be different. The direct paths with MyComputer and the remote IP can be used as well.
To me, this method is far more useful but at a small cost. Shadow regexp groups allow you to sync any files that match a full regular expression. This means you can sync any file in specific directories or that have a given naming structure. The main caveat, at least through my testing, is that the file paths must be the same. I use shadow files to link two directories between my main computer and a remote server. This doesn't affect me as I've set up a uniform environment across my devices to facilitate version control, but it could a deal breaker. I could be using shadow files incorrectly, whose to say.
To set up file synchronization between a directory on your home machine, and one on a remote server, you proved two regular expressions with different clusters:
(setq shadow-regexp-groups '(("/ssh:user@123.45.67.890:\\`/home/user/path-to-dir/.*\\'" "/MyComputer:\\`/home/user/path-to-dir/.*\\'")))
When any file that matches these regular expressions changes, the system records that the file has changed and waits for synchronization.
After changes are made, run M-x shadow-copy-files to sync the changes. This will ask for confirmation to sync each file with a recorded change. This will create new files if necessary; it will NOT create new directories, so keep that in mind.