Tape Archive

Following guide is meant to help you archive your data using simple bash commands and afterwards send it to us using internal kista where we can collect the data and store it on a long term tape medium.

Attention

This is a pilot service for tape archiving. Contact us (opens new window) for further information.

1. Identification of data

1.1 Identify what data you want stored on the tape. Mark down whole directories. Generally, we expect your data to be stored in /mnt/archive directory

1.2 Do you use symbolic links ? If you are not sure, you can identify symlink directories by first letter in permission string. Letter l stands for a symbolic link. You can also use command below to count your symlinks. If output is other than 0, contact us on slack for further instructions.

Example:

lrwxrwxrwx 1 root    root      28 Mar 21 07:59 test-symbolic-link -> /path/of/symbolic-link/ 

Command:

ls -lR /mnt/archive | grep ^l |wc -l

2. Compress data using tar

2.1 Check disk space available with command below.

df -H

For compression of data we will use tar command and store archive in /lab/scratch directory.

2.2 Create a file_list text file of all files you want to save using below command.

find -L /mnt/archive -not -type d >  labname-filelist.txt # this will check /mnt/archive directory and list all files.

Options used:
-c: Create an archive.
-z: Compress the archive with gzip.
-v: Display progress in the terminal while creating the archive, also known as “verbose” mode. (optional)
-f: Allows you to specify the filename of the archive.
-h: Follow symlinks; archive and dump the files they point to. (optional)

Examples:

tar -czvf single_file.tar.gz /mnt/archive/data.dta    # single file or directory

tar -chzvf archive_symlink.tar.gz /mnt/archive    # includes symbolic links

tar -czvf archive_multiple_dir.tar.gz /mnt/archive/test.txt /mnt/scratch/data.dta  # compress multiple directories

tar -czvf excluded_files.tar.gz /mnt/archive --exclude=/mnt/archive/test/data.txt --exclude=/mnt/archive/test1/data1.txt # excludes specified directories

3. Hash the data

Once you have compressed your files, you need to create a hash of the files using sha256sum command.

single hash:

sha256sum your_archive.tar.gz > hash_archive.txt

4. Upload compressed data, hash file and file_list.txt to kista.

5. Notify us