Skip to content

Directory Analysis with Bash

Introduction

It is possible to do a lot of evidence hunting with just bash. Sometimes using FTK is overkill, this is where bash comes in. Using bash you can create a listing of a folder and create a list of files and it's statistics. This works particulary well when analyzing image files on Linux. Below I'll write up the process of setting up an image for mounting and how to perform some basic directory analysis.

Image Setup

If you have a raw image then mounting is farely straightforward. First you have to run fdisk -l in order to find the offset that the partition starts in. 

root@FORENSICS-PC:~# fdisk -l 'Snapshot2.001'

Disk Snapshot2.001: 64.4 GB, 64424509440 bytes
255 heads, 63 sectors/track, 7832 cylinders, total 125829120 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x92eaca5f

         Device Boot      Start         End      Blocks   Id  System
Snapshot2.001p1   *        2048   125827071    62912512    7  HPFS/NTFS/exFAT

In this image the offset starts at 2048. We use this number and multiply by 512 in order to find the starting point to mount the image, for this image it is 2048 * 512 = 1048576.

This is all that we need in order to begin mounting. In order to mount the file we need specify a directory. Usually it's good practice to mount it inside the /mnt directory. Since we are gonna work on the image we'll mount the image inside a temp folder that resides inside the /mnt directory. I used a /mnt/tmp/forensics directory. We will use this as our working directory in order to house our findings. The command mkdir -p /mnt/tmp/forensics will create the needed directory and any parent directories if they don't exist.

The following image will mount the image to the specific directory. One thing to note is that this mount is not permanent, if the computer is powered down, the image will become unmounted.

mount -o offset=1048576 Snapshot2.001 /mnt/tmp/forensics

Directory Analysis Script

After creating the image we can then list all the files in the image and order it by last modified. The most recent and relevant items will be at the top of the list. I created a script that does this automatically and it even includes the accompanying sha1 hashes to every file.

#!/bin/bash

if [ $# == 0 ] ; then
    echo "Usage: ./dirAnalysis.sh <DIRECTORY>"
    exit 1;
fi

TIME=`date +%s`

echo "[*] Running directory scan, #$TIME"
echo "  [>] Using find for gathering stats"
echo "  [=] Gathering dates and sha1sums"
{
        nohup find $1 -type f -exec stat --format '%Y :%y %n' "{}" \; | tee date.list &
        nohup find $1 -type f -exec sha1sum {} \; | cut -d " " -f1 | tee sha1sum.list &
} &> /dev/null
wait
echo "  [=] Preparing all output into one file"
paste -d" " date.list sha1sum.list > merged.list
cat merged.list | sort -nr | cut -d: -f2- > directory_$TIME.list
echo "  [=] Cleaning..."
rm date.list
rm sha1sum.list
rm merged.list
echo "[*] All successful, the results are located at $(realpath directory_$TIME.list)"

Script Breakdown

The script has multiple parts in order to create a list. One of the first things it does is assign a variable called TIME. This is just a way to differentiate the result file at the end by assigning it a value based on time.

The meat of the script are the two find commands. They both run in parallel in order to reduce , this is done by using nohup and putting a & at the end of the line. The brackets surrounding the two find commands are used to display unwanted text into an outside pipe, it just serves to keep the terminal clean from any unnecessary output. The first line, uses find $1 -type f -exec stat --format '%Y :%y %n' "{}" \, to create a list of all the files in the image and displays the timestamp of the last modified date. The second line, find $1 -type f -exec sha1sum {} \, creates a list of all the files as well as calculating the sha1 hashes for each one. It then cuts out duplicated data that the first find found.

After everything is done, the paste -d" " date.list sha1sum.list > merged.list line merges both the files using a space to seperate contents. It then sorts all the items from most recently modified to last modifed.

Using Grep to Find Relevent Output

With the newly created list of directory contents you can then begin analyzing files. If you have a timeframe of stuff to look out for you can easily grep by a year and month. Something like the following should work cat directory_1492648314.list | grep "2017-04". This would only show items from the month of April in 2017. If you're looking for specific information from a user or a certian extension you can do something like grep "docx" or grep "Users/JohnSmith" to find the item of relevance. You can also reverse grep using the -v flag. An example is if you want to avoid output from a directory that displays a lot of temp or system files like the AppData directory. This can be done with grep -v "AppData". Additionally we can combine all these things in one single line in order to be very specific. 

root@FORENSICS-PC cat directory_1492650274.list | grep "2017-04" | grep "Users/Default.WIN-SUBQGETTI1M" | grep -v "AppData"
2017-04-10 18:53:56.185966300 -0500 5af287953f82e795f5b87e9a2c74dc404c37bfaa /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/NTUSER.DAT 
2017-04-10 18:53:56.154766300 -0500 586f9e62cbbf444b51bab7467056ab91f6d2c79c /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/ntuser.dat.LOG1 
2017-04-10 17:41:34.523208800 -0500 5ebda0ff9a09376ad3278a44bb12da2b9649d386 /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Documents/usr-pwd.7z 
2017-04-10 17:24:21.196770900 -0500 39195d8b0237fbaca533f5ccf7f790e2f5752f54 /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Documents/random/;lsakdjfa.rtf 
2017-04-10 17:23:47.297911400 -0500 41908241a9ff4a0e378af2204ed4f6fff2d1ee25 /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Documents/random/999sdlkfjaso33iqpe09w4nmnsrnsdoigf.txt 
2017-04-10 17:22:31.293777400 -0500 da39a3ee5e6b4b0d3255bfef95601890afd80709 /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Documents/random/New Text Document.txt 
2017-04-10 17:20:14.371134400 -0500 17b661485ec13d6a7664bbb04c7ce17426443eda /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Documents/CallToAction.7z 
2017-04-10 16:22:26.173000000 -0500 b37c46530bb0ea4289069f663aed368214b5fd21 /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Pictures/mobilepictures/Thumbs.db 
2017-04-10 16:19:06.000000000 -0500 65304bbd6c17d1793f9b34f507dec2c624bba31f /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Pictures/mobilepictures/Jo887K.png 
2017-04-08 14:08:22.854000000 -0500 34e163be8e43c5631d8b92e9c43ab0bf0fa62b9c /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Pictures/Thumbs.db 
2017-04-08 14:05:42.292154300 -0500 b84b7ed93e90995f49215cb66c9b88dc780852df /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Pictures/Cabin location for justic.jpg 
2017-04-08 14:01:26.286493900 -0500 da39a3ee5e6b4b0d3255bfef95601890afd80709 /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Desktop/Myloves/Ashleyloveletter.txt 
2017-04-08 13:52:28.014744900 -0500 6daa6ad76d4f1196bd41de42c05a76fce135899f /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Documents/Invoice _ 4541.docx 
2017-04-08 12:34:48.653810100 -0500 cfd05f64fa39fbb3c267360a3b30e5c5687d920c /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Documents/futurevacationplans.txt 
2017-04-08 12:32:48.538797000 -0500 da39a3ee5e6b4b0d3255bfef95601890afd80709 /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/passwords.txt 
2017-04-05 13:25:36.000000000 -0500 6da45c897c997c778ecba028491f6d9e8279306d /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Desktop/Myloves/LoveLetterToRebecca.docx 
2017-04-05 13:23:50.000000000 -0500 8fddeabb0229736c8432e61970bbcd6fa8f91c06 /mnt/snapshot2/Users/Default.WIN-SUBQGETTI1M/Desktop/Myloves/LoveLetterToLucy.docx 
...snip...

We get a list of relevant files that we can then copy over and examine. Furthermore, hashes are included also, these help when reporting about findings. It also helps to find things again if you record the hash instead of the directory. There are times when it helps to have both.

Supplmenting Directory Analysis with FTK

FTK by itself may be too overkill when doing some directory analysis, however it's really useful for tallying down items, reporting, and analyzing files in depth. A good workflow for synergy between FTK and bash is to find things using the dirAnalysis.sh script and then handing over a list of findings to the person using FTK. A list of relevant hashes would do and then the FTK could further examine everything. You can also analyze individual files using a bash utilities, however this may be a troublesome task that is best suited for FTK. It's better to use bash to create a list of relevant items quickly and then hand it over.