We're an ISO27001:2013 Certified Supplier

blog-post-featured-image

You’re trying to resolve a configuration problem for a complex application. There’s a growing trend for applications to have configuration directories, typically named conf.d, rather than a single configuration file, and this application uses that technique. That means there are multiple configuration files, so before you start you copy the original, not-quite-working, conf.d to conf.d.original.

Finally, after you’ve lost count of the number of edits and consultations with Mr Google, you finally get it working. The question now is: what was the actual change or changes that resolved the problem? What’s the easiest way of comparing conf.d to conf.d.original?

I’ve seen some really complex answers to this problem, involving shell scripts or guru-level Perl incantations, but there are few much simpler, and arguably better, solutions.

Let’s take a look at a few.

The diff Command

The diff command is associated with showing the differences between files, but it can also be used to show differences between directories. We can get a summary of the changes with diff -rq:

$ diff -rq conf.d.original/ conf.d
Files conf.d.original/10-auth.conf and conf.d/10-auth.conf differ
Files conf.d.original/10-mail.conf and conf.d/10-mail.conf differ
Files conf.d.original/15-lda.conf and conf.d/15-lda.conf differ
Files conf.d.original/20-imap.conf and conf.d/20-imap.conf differ
Only in conf.d: 20-managesieve.conf
Files conf.d.original/90-acl.conf and conf.d/90-acl.conf differ
Files conf.d.original/90-plugin.conf and conf.d/90-plugin.conf differ
Only in conf.d: 90-sieve.conf
Only in conf.d: 90-sieve-extprograms.conf
Only in conf.d: auth-ldap.conf.ext

The -r switch makes this a recursive comparison, descending into directories, and the -q makes this a quick comparison, only listing the files that are different rather than showing what the differences are. That’s a good starting point. Of course, we can use diff again to look at the differences in more detail:

$ diff -u conf.d.original/10-auth.conf conf.d/10-auth.conf
--- conf.d.original/10-auth.conf 2017-10-16 08:46:01.188700855 +0100
+++ conf.d/10-auth.conf 2017-10-16 08:44:43.823364825 +0100
@@ -121,7 +121,7 @@

!include auth-system.conf.ext
#!include auth-sql.conf.ext
-#!include auth-ldap.conf.ext
+!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-vpopmail.conf.ext

Other tools – I favour vimdiff and meld – can be used to merge those differences if required. All changes to all files can be listed with diff -rq conf.d.original/ conf.d, but that may give too much output to deal with easily.

The dirdiff Utility

If you favour a graphical tool, dirdiff may be worthy of consideration. Here, we see the same directory comparison as before:

dirdiff listing

By selecting the 10-auth.conf file and clicking on the Diff menu, we can see a visual diff of the file:

dirdiff differences

The dirdiff tool has many other facilities, including the option to generate patch files to take a file from one version to another.

Just Use git

Some shy away from git, fearing its complexity, and it is certainly a very powerful tool. Like most good, powerful tools, though, you only need to know what you need to use. Let’s assume you’ve never used git before. Here’s how to manage the directory change problem we’ve been discussing.

Before you start making any changes, issue this command:

$ git init
Initialized empty Git repository in /tmp/git/.git/
$ git add .
$ git commit -m "Original files"
[master (root-commit) 2b7bc59] Original files
26 files changed, 1611 insertions(+)
create mode 100644 10-auth.conf
[...]

This is the equivalent of making a backup copy of conf.d that we used earlier.

Now make all the changes you need. Once the system is working as required, you can explore the changes. First, which files changed?

$ git status
On branch master
Changes not staged for commit:
(use "git add ..." to update what will be committed)
(use "git checkout -- ..." to discard changes in working directory)

modified: 10-auth.conf
modified: 10-mail.conf
modified: 15-lda.conf
modified: 20-imap.conf
modified: 90-acl.conf
modified: 90-plugin.conf

Untracked files:
(use "git add ..." to include in what will be committed)

20-managesieve.conf
90-sieve-extprograms.conf
90-sieve.conf
auth-ldap.conf.ext

no changes added to commit (use "git add" and/or "git commit -a")

The first list shows files that have been modified; the second shows new files that have been created. To compare a filed that has changed:

$ git diff 10-auth.conf
diff --git a/10-auth.conf b/10-auth.conf
index 1c59eb4..cfb6e46 100644
--- a/10-auth.conf
+++ b/10-auth.conf
@@ -121,7 +121,7 @@ auth_mechanisms = plain

!include auth-system.conf.ext
#!include auth-sql.conf.ext
-#!include auth-ldap.conf.ext
+!include auth-ldap.conf.ext
#!include auth-passwdfile.conf.ext
#!include auth-checkpassword.conf.ext
#!include auth-vpopmail.conf.ext

To launch a visual tool to examine and manage differences:

$ git difftool 10-auth.conf

There are git configuration options to change the tool used.

Once you’ve finished the work, if you no longer want to keep a git repository, just remove the .git directory:

$ rm -rf .git

Conclusion

Tracking the changes when configuring an application can help both with understanding what changes are necessary and with documenting the changes. Either approach above works well, but if you’re unfamiliar with git I would encourage its use: after a short while, you’ll wonder how you managed without it.

Could This Linux Tip Be Improved?

Let us know in the comments below.