dl-config-compare v1.4 [29 Mar 2012] by Dominic
Specific program for users of Devil-Linux (http://www.devil-linux.org) to view differences between the default Devil-Linux configuration files and your own configurations. Intended for running after a Devil-Linux system upgrade, so you can compare your retained configuration files (at /shm/etc-mods/*) with the default files (at /etc-cd/*), and then (manually) update your configuration files with any changes rather than having them fall behind with deprecated or no-longer-supported settings. Trivial variations (comments, blank lines) are ignored.
This is not a replacement or alternative to the file selection process offered upon first boot when upgrading Devil-Linux; but it is normal (and advisable) during such process to retain any configuration files identified for retention (because otherwise you might break something.) Yet by retaining old configurations in this way you might fail to adopt new settings or continue to use deprecated ones. This is where dl-config-compare.sh can help.
Based upon the output from dl-config-compare.sh (which doesn't itself make any changes to your configuration files), you can either manually delete an identified differing configuration file (e.g. /shm/etc-mods/lpd/lpd.conf), thus substituting the default file at /etc-cd/ or manually update the configuration file, retaining any specific settings of your own but adopting any other 'latest and greatest' settings and eliminating deprecated or broken settings.
As explained below you can 'tune' dl-config-compare.sh to ignore, in any given file, specific differences that you are ok with, so that it only warns you about other unexpected differences.
A standard list of files to be ignored entirely is automatically used (this can be seen by running with -s option). As part of the process of using dl-config-compare.sh you should also create your own 'ignores' file (see examples below) which lists configuration files (regex format, one pattern per line) which can be ignored, or, if you add a sed command after a comma on the same line, which are to be compared with the 'default' version at /etc-cd/ *after* applying your sed command(s) to both file versions.
Listing a file pattern without a sed command means that files matching the pattern are ignored entirely - use this for files which contain no important 'default' settings. Listing a file with a sed command (or commands) means that you will be warned about any differences that remain between your configuration file at /etc-mods/ and the equivalent default file at /etc-cd/ *after* the sed command has been applied to both files.
Your 'ignores' listed in your own 'ignores' file have higher priority than the standard 'ignores', so for instance you can force a file listed as a standard ignore (say /etc/crontab) to be processed by adding a line with a comma and a dummy sed pattern, for example:
No original files are modified by this program. The exit code is the number of files which do not match (after applying any sed commands) or which exist in both locations but are the same (and therefore could be deleted from /shm/etc-mods/). With verbose output '-v' the differences between the files are also shown. Normal output is formatted for a console with colors, use the plain option '-p' if you want to save the output.
Start by running dl-config-compare.sh with -v option, then with the information returned you can build your ignores_file which you then specify on the command line when you re-run dl-config-compare.sh. In the case of configuration files in which you have *some* special settings (as well as other settings inherited as defaults) it is better to add a sed command to a line in your ignores_file which excludes your special settings from the comparison, rather than excluding the file from comparison altogether, so that if the default settings change in a later release of Devil-Linux they will be flagged up when you run this program.
dl-config-compare.sh [options] [ignores_file]
# lines beginning with hash are comments - do not append
# hashes and comments to active lines
# note that periods & other special characters should be escaped
# process ntp.conf (for xntpd), ignoring lines starting 'server'
# a more complex example involving multiple sed commands:
/etc/syslog-ng/syslog-ng.conf,s/stats_freq(7200); //;/destination logfile /d;/log . source(src). destination(logfile)/d
-a i.e. 'all': list skipped files too
-d i.e. 'debug': don't remove temporary files
-h show this help and exit
-l show changelog and exit
-p plain output format (no colors, ANSI sequences)
-s show the standard or default 'ignores' and exit
-v i.e. 'verbose': if files differ, show the differences
Devil-Linux OS, awk, basename, bash, diff, find, fold, save-config, sed, stty, tar
Copyright 2014 Dominic Raferd. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
1.4 [29 Mar 2012] - improved help text
1.3 [11 Feb 2012] - check only /etc-mod/ files saved by save-config
1.2 [10 Feb 2012] - check all /etc/ files, with 'ignores' file
1.1 [09 Feb 2012] - use conf file, allow non-verbose output
1.0 [08 Feb 2012] - initial release
- TimeDicer - Onsite/offsite data backup for Windows (uses rdiff-backup)
- Finding a 4D Backup Solution
- Web Scraping How To - extracting data from web sites
Here is a selection of some (other) programs I have written, most of which run from the command line (CLI), are freely available and can be obtained by clicking on the links. Dependencies are shown and while in most cases written for a conventional Linux server, they should run even on a Raspberry Pi, and many can run under Windows using Cygwin. Email me if you have problems or questions, or if you think I could help with a programming requirement.
- TimeDicer - Onsite/offsite data backup for Windows (uses rdiff-backup) [GNU/Linux & MS Windows©: 2008-14]
- rdiff-backup-install - GNU/Linux script to install rdiff-backup. [ GNU/Linux: 2012-12 ]
- rdiffweb-install - GNU/Linux script to install rdiffWeb, fixing various bugs that otherwise prevent it working correctly. [ GNU/Linux: 2011-15 ]
- rdiff-backup-regress - GNU/Linux script to regress an rdiff-backup archive. [ GNU/Linux: 2012-15 ]
Debian/Ubuntu kernel and LVM Utilities
- kernel-remove - GNU/Linux script to list the installed GNU/Linux kernels in a Debian-based distro (e.g. Ubuntu), and can be used to remove an unwanted kernel and related packages, updating grub appropriately. (Ubuntu Tweak can do the same but kernel-remove.sh is a command-line script so does not require GUI.) [ GNU/Linux-Debian/Ubuntu: 2010-15 ]
- kernel-update - GNU/Linux script to install/update Ubuntu kernel (also optionally btrfs-progs and duperemove) with latest version. [ GNU/Linux-Ubuntu: 2015-15 ]
- lvm-usage - GNU/Linux script to show available disk space and how it is used; run as cron job to warn if usage is above a set percentage. Provides additional information if LVM is in use. [ GNU/Linux-Debian/Ubuntu: 2012-15 ]
- lvm-delete-snapshot - GNU/Linux script to remove LVM snapshot that has been left over by another process. [ GNU/Linux-Debian/Ubuntu: 2012-15 ]
Dellmont / Three / Giffgaff / Vodafone - VoIP and Mobile Phone Account Utilities
- dellmont-credit-checker - GNU/Linux script to check credit balance on many Dellmont / Finarea / Betamax portals such as voicetrading.com and voipdiscount.com. [ GNU/Linux: 2008-15 ]
- get-vt-cdrs - GNU/Linux script to download CDRs (call detail records) from Dellmont’s voicetrading.com or voippro.com. [ GNU/Linux: 2010-15 ]
- saynoto0870 - For people in UK, a GNU/Linux script which performs automated lookup of the www.saynoto0870.com database, finding cheap or free geographic number replacements for expensive non-geographic (087* or 084*) numbers. [ GNU/Linux: 2012-12 ]
- three-credit-checker - GNU/Linux script which checks credit/calls/text/data remaining on a mobile phone account with three.co.uk. [ GNU/Linux: 2014-15 ]
- giffgaff-credit-checker - GNU/Linux script which checks credit/calls/text/data remaining on a mobile phone account with giffgaff.com. [ GNU/Linux: 2014-15 ]
- vodafone-compile-bills - GNU/Linux script which reprocesses downloaded call record 'csv' files from vodafone.co.uk so that they can be easily analysed via spreadsheet - including analysis of bundled minutes which even Vodafone do not seem able to perform! [ GNU/Linux: 2012-12 ]
- sleepwalker - Windows© program which can be run from a remote machine to 'wake up' a Windows© machine behind a router, wait for it to start and then initiate Remote Desktop session. [MS Windows©: 2008-14]
- form-extractor - GNU/Linux program to extract form tags from a web page or downloaded file. [ GNU/Linux: 2012-15 ]
- recover-space - GNU/Linux program to enable a virtual disk volume to be compacted. [ GNU/Linux: 2014-15 ]
- tiny-device-monitor - GNU/Linux program to test webpages (including password-protected) or machines to check they are live; use as a cron job for your own websites, for hardware presenting a webpage, or for any machines with a presence on your local LAN or on the internet. [ GNU/Linux: 2009-15 ]
- dutree - GNU/Linux program to show a tree-style list of files and directories at the specified location and greater than the specified size (default 1GB). [ GNU/Linux: 2012-15 ]
- disk-wiper - GNU/Linux script to wipe a disk drive comprehensively and also check it for bad blocks. For use on a surplus drive (not SSD) before passing to a third party. [ GNU/Linux: 2011-15 ]
- myip-upload - GNU/Linux and Windows (Cygwin) script to obtain external ip and upload it to remote site/file by ftp. [ GNU/Linux & MS Windows©: 2014-15 ]
- man2text - GNU/Linux one-liner program to convert man page output to straightforward text. [ GNU/Linux: 2012-12 ]
- Accounts - Multi-business multi-currency accounting software, uses Access [MS Windows©: 1996-2014]
- Rents Program - Residential lettings/landlord front office program, with many special features for UK market [MS Windows©: 1991-2014]