Linux - filtered backups with preview
Currently, I experiment with a possibility to backup the whole filesystem to a single compressed file. During the backup process, I want to conveniently preview the directory structure which will be actually backed up.
I have prepared separate scripts for multiple stages of this process. While doing a backup, I run them in order.
The scripts have hardcoded paths - review and edit the scripts and files below before using them.
Stage 1: Mount a filtered filesystem
rofs-filtered.rc file contains the filter rules as regular expressions. E.g. the contents of my file are:
^/bin$ ^/dev$ ^/lib$ ^/lib64$ ^/lost\+found$ ^/media$ ^/misc$ ^/mnt$ ^/net$ ^/opt$ ^/proc$ ^/run$ ^/sbin$ ^/sys$ ^/tmp$ ^/usr$ ^/var/cache$ ^/var/lib/docker$ ^/var/lib/jenkins/caches$ ^/var/lib/jenkins/\.gradle$ ^/var/lib/jenkins/\.npm$ ^/var/lib/jenkins/\.nuget$ ^/var/lib/jenkins/jobs/[^/]+/builds$ ^/var/log$ ^/home/user/\.android/avd$ ^/home/user/\.cache$ ^/home/user/\.config/Code/User/workspaceStorage$ ^/home/user/\.config/unity3d/cache$ ^/home/user/\.local/share/NuGet$ ^/home/user/\.local/share/Trash$ ^/home/user/\.local/lib$ ^/home/user/\.gradle$ ^/home/user/\.npm$ ^/home/user/\.nuget$ ^/home/user/\.minikube/cache$ ^/home/user/Unity/Hub$ ^/home/user/VirtualBox VMs$ ^/root/\.cache$ ^/root/\.gradle$ ^/root/\.npm$ ^/root/\.nuget$ /\.metadata$ /node_modules$
Stage 2: Check what occupies the most space
filelight is a GUI application which shows you a graphical representation of the space occupied by the parts of the filesystem tree. Mostly, this application can be obtained from standard Linux distribution repositories.
Stage 3: Check the backup size
This only tries to create the compressed backup, without storing it anywhere. It is faster than Stage 4 and the result does not occupy any disk space.
If the result of this step shows that the backup is too large, I return to Stage 3 to see the largest directories, and then possibly return to Stage 2 to edit the filesystem filter.
Stage 4: Execute the backup
This creates the same backup file as Stage 3 and actually stores it somewhere.