Synchronization of .dot files

Tired of manually copying settings on different computers? This article will show you how to synchronize your dotfiles so that your environment and application settings remain the same.

Denis Rasulev ·   · 
Synchronization of .dot files

In this article, let’s talk about how to use dotfiles to keep system settings unchanged across devices. By following simple steps with Git, you can easily manage everything from bash profiles to application settings. This approach ensures a seamless experience no matter what platform you’re using.

If you’re in a hurry, go straight to step-by-step guide to setting up dotfile synchronization. And if you want to understand what’s done there and what it’s for, read on.

Dot files (dotfiles) are configuration files in Unix-like systems such as macOS, Linux, and BSD. They store settings for applications, tools, and services running on your system, including things like git, ssh, .bashrc, .zshrc, .nanorc, .vimrc, and many others.

The name of such files starts with a dot (.) to hide these files from the user by default and to simplify the lists of files in directories. And why they are hidden from the user - read the footnote below (click to open it).

How .dot files became hidden files

This story begins many years ago, when the first file systems were created in UNIX. To allow easy navigation, one dot file (.) was added to each directory. And another double-dot (…) file was added to easily navigate up through the directory structure.

Since there was no real data in these files, a quick hack was added to the LS command code to hide such files from default output. There’s a full story of this hack here about how laziness allowed such hidden files to be created 😉.

Many developers share their configuration files. You can always find links to popular repositories with other people’s configs on the Internet.

Synchronizing dotfiles is necessary for those who use multiple computers (and/or servers) and want to have familiar environment, services and application settings on each. This increases productivity, reduces setup time for new machines, and ensures that the same tools and configurations of services and applications are available regardless of the computer being used.

For example, I recently had to set up two new MacBooks in a row. On the main one, all the settings have been done for a long time and are rarely changed. There I have already got used to certain utilities, keyboard shortcuts and many actions are performed automatically. But on the new one, I had to configure everything from scratch.

The first problem was to remember what to install and how to config it so that everything would be familiar. Then the second problem arose - how to synchronize the settings of programs, and even so that when they are changed on one computer, all this will automatically change on other computers.

First of all, you should understand that synchronization of absolutely everything - installed applications and services, their settings, environment variables, etc. is still not 100% solved. There are things that are tied to specific hardware, and simply copying them to another computer will result in a non-functional application.

So everyone is still agonizing, using Ansible, Chef.io, OpenTofu and other tools, or solving the issue with at least basic apps. I have also only set up synchronization for the following configuration files so far:

  • .gitconfig - configuration for working with Git.
  • .gitignore - which files to ignore. (👉 .gitignore configurator)
  • .muttrc - mail in terminal.
  • .p10k.zsh - theme for ZSH.
  • .vimrc - settings for Vim.
  • .zimrc - modules for Zim.
  • .zshrc - ZSH settings.

Any file can be added and removed from synchronization at any time.

There are tons of articles on this topic on the Internet. You will find a list of the most sensible and useful ones I have read on this topic at the very end of this article. Most of them describe the use of various version control systems.

The most popular system is, of course, Git. In the picture you can see the market shares of the different systems and that Git is the clear leader among all of them.

Market shares of different version control systems.
Market shares of different version control systems.

Link to up-to-date information on the Top 5 Version Control technologies.

What Git and GitHub are and how they differ

Git - is a free and open source distributed version control system for tracking changes in files, designed to work quickly and efficiently with any project - from small to very large.

GitHub - is a commercial company offering cloud hosting for Git repositories. It allows individual enthusiasts and entire teams to store, track, and control changes to project code (files).

Simply put, you can use Git without Github, but you can’t use GitHub without Git.

Git is a popular, convenient, and familiar system, so I decided to use it. When using Git, there are two most common ways to synchronize configuration files - through the usual directory and through the bare directory.

How to synchronize environment settings through a regular directory? To do this, you need to create a special directory (folder) in your home directory, where all the settings and configuration files you need will be saved. Then symbolic links are created for them in your home directory.

Here are the specific steps to set up synchronization this way. All commands are executed in the console.

# Go to your home directory
cd ~

# Create a directory to store the configuration files
# The name can be anything, I used dotfiles
mkdir dotfiles

# Move the existing configuration files into the created directory
mv .vimrc .zimrc .zshrc dotfiles/

# Create symbolic links in the home directory
# -s: this option establishes a symbolic link
# -f: overwrites the link file without notification
ln -sf ~/dotfiles/.vimrc ~/.vimrc
ln -sf ~/dotfiles/.zimrc ~/.zimrc
ln -sf ~/dotfiles/.zshrc ~/.zshrc

In simple words: the commands above move files of settings to a special directory (dotfiles) and replace them in your home directory with links to them. When a program, such as vim, tries to read its settings file, which is by default in your home directory, it opens a link that takes it to the file itself in the dotfiles directory.

The next step is to use your browser to go to your GitHub (if you don’t have one, register an account, it’s free) and create a repository there where you will store your configuration files. Then go back to the console on your computer and navigate to the dotfiles directory we created

cd dotfiles

Initialize it as a Git directory:

git init
git add .
git commit -m ‘First commit’
# In the command below, replace the link to your repository address on GitHub
git remote add origin https://github.com/YOURNAME/dotfiles
git push origin main

This directory is easy to work with using the usual Git commands - status, push, pull, etc. When you need to add a new settings file for synchronization, e.g. .nanorc, do the following:

mv .nanorc dotfiles/
ln -sf ~/dotfiles/.nanorc ~/.nanorc

cd dotfiles
git add .nanorc
git commit -m ‘Add nanorc config’
git push

IMPORTANT! Note that Git commands are executed inside the project directory that was initialized, in the .git folder. Standard Git logic assumes that the project’s working tree is in the same directory. Therefore, to perform actions to synchronize files (add, modify, download, etc.), you will need to go inside the dotfiles directory each time.

However, optionally you can create a special alias that will allow you to execute Git commands in the dotfiles directory from anywhere without having to go inside the dotfiles directory every time. You can name this alias whatever you want, I named it cfg. Here’s what you need to do.

# Create an alias with a name that's easy for you to remember
alias cfg='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'

# Save it in the environment settings, so we don't have to create it every time.
echo "alias cfg='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'" >> $HOME/.zshrc

# And turn off untraceable files so we can see the ones we want to see
cfg config --local status.showUntrackedFiles no

If you use this alias, the process of adding new files or versions will look like this:

cd ~
mv .nanorc dotfiles/
ln -sf ~/dotfiles/.nanorc ~/.nanorc

cfg add .nanorc
cfg commit -m ‘Add nanorc config’
cfg push

Note that there is no need to enter the dotfiles directory - we did not execute the cd dotfiles command. So you can execute these commands from anywhere. Once again, let me remind you that this step is completely optional, it’s just for convenience. If you are not too lazy to go to the dotfiles directory every time to add or update the configuration files, you don’t need to create an alias.

How to synchronize settings on another computer?

Now. On the other computer, to synchronize the settings with the source computer, start the console and run the following commands:

# Go to your home directory
cd ~

# To avoid recursion, add the .cfg directory to global ignore beforehand
echo ".cfg" >> .gitignore

# Clone your repository from GitHub
git clone https://github.com/YOURNAME/dotfiles $HOME/.cfg

# If you're using an alias, save it in your environment settings
echo "alias cfg='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'" >> $HOME/.zshrc

# Disable the display of untracked files
cfg config --local status.showUntrackedFiles no

# And move the contents of the repository to a new computer
cfg checkout

You may get a warning that some of the existing local files, such as .bashrc or .gitignore may be overwritten. Save them if you need them or delete them, then repeat the command - cfg checkout. Done!

Some users note that this method does not negatively affect the performance of other programs.

The inconvenience is that you must always remember about symbolic links, and that the configuration files themselves now lie in a special directory. If you change something, the links need to be updated or edited. And if there is no need for synchronization, you will have to delete the links and return the files themselves to their place.

And you should also keep in mind that some applications simply do not work with symbolic links. They need only the original configuration files.

Another group of solutions uses the --bare option when creating a Git directory. I used this method because I really didn’t want to mess around with a separate directory and symbolic links. Although everything is automated, I found this method to be more elegant and convenient.

👉 I won’t go into details here, but you can read the difference between regular and bare catalogs in this informative question What’s the practical difference between a Bare and non-Bare repository? on StackOverflow.

So, what do I need to do to set up dotfile synchronization using the bare directory method? To do this, perform the following steps.

# Go to your home directory
cd ~

# Create a bare Git repository in your home directory
git init --bare $HOME/.cfg
alias cfg='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'
cfg config --local status.showUntrackedFiles no
echo "alias cfg='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'" >> $HOME/.zshrc

When you’re setting up dot file synchronization, it’s important to keep security in mind. Sometimes configuration files can include sensitive data such as access keys and passwords. Here are some simple tips to help you protect your dot files:

  1. Private repositories: If you use GitHub or another platform to store your dotfiles, be sure to make the repository private. This will help avoid unauthorized access to your configuration files.

  2. Data Encryption: Don’t add files containing passwords, API keys, private SSH keys and other sensitive information to your repository. For such data, it is better to use special secret managers, such as git-crypt or SOPS or GPG.

  3. Regular checks: Periodically check who has access to your repository and update passwords and keys. This will help protect your data from unauthorized access.

  4. Use SSH keys: Connect to remote servers and repositories using SSH keys rather than passwords. This will increase the security of your connections.

  5. Access Restrictions: Make sure only those who really need to access your dot files have access. Enable two-factor authentication for extra protection.

In conclusion, I would like to share a few points.

First, you should only bother with synchronization if you have more than two servers/computers for which it is important to have the same tools and settings. Otherwise it will be more trouble than it’s worth. Secondly, try to understand what exactly you need to synchronize. If it’s just one or two utilities, then again it may not be worth it. And don’t forget about security!

When I started setting up dotfile sync, it seemed very complicated. Once I got everything set up and learned the service commands, then it was quick and easy. Works great, so in my case the effort was worth it.

However, after some time, I realized that I was much more likely to have to bring up a new server or set up a new computer than I was to change settings. And that’s a whole different story that requires a slightly different approach and toolset. I’m already researching another set of information on this topic, and there are some very interesting findings. As usual, I plan to summarize and simplify the experiences of smart people and share it all on my blog, so stop by again.

Here is a list of materials I had to dig through to understand the topic.

Good luck and if you have any questions, contact me via any of the channels.

😎

 Tools, Sync, Dots
Published on Aug 17, 2024
Back  |  Home