Getting Started with Timesketch and Docker

I’m sure I’m not doing everything the easiest/fastest/best way, but hopefully this is useful to others. Feedback welcome.

Contents

1. Why Docker

2. Docker Installation

3. Skadi Setup

4. Running Docker

5. Why Timesketch

6. Using Timesketch
   

Why Docker?

• Docker runs on Windows, Mac, or Linux.

• Docker builds images in layers. If you need to change the image, only the affected layers will need to be rebuilt.

• Repeatable process.

• If I break the container, no harm. If I break a server, I need to submit a ticket to restore from backup.

• Containers can be automatically removed upon process completion, leaving your hard disks nice and tidy.

• Great for tools that might take more effort to set up such as Bulk_Extractor-rec or NSRL Lookup. I’ll post them below since they were a pain for me to get working. Alan Orlikoski set up the Skadi environment with ~21 containers. My effort to set up Timesketch is now trivial compared to his previous work.

• Check https://hub.docker.com/ for tons of premade images like SIFT, Autopsy,

Docker Installation

1. Lots of documentation on Docker, but I don’t always find it the easiest to understand. When I started learning about Docker, these links were helpful:

   • What is Docker?

   • Docker terminology

   • Docker containers, images, and registries

   • Youtube: Learn Docker in 12 Minutes

2. Create a Docker account and download Docker for Windows.

3. Installation will have you use Windows or Linux containers, log out, enable Hyper-V, and reboot.

   • This will prevent use of VMs in VMware and VirtualBox. To switch use “Turn Windows Features On or Off” to uncheck Hyper-V. Requires reboot.

4. To use both VMware/VirtualBox and Docker

   • Install Docker on a Linux VM.

   • Use a Mac to run them side-by-side.

   • It may be possible to install Docker on a Windows VM, but requires nested virtualization and seems subject to problems. Documentation here.

   • Docker will not work inside WSL as the daemon doesn’t run directly there. Looks like this changes with WSL 2.0 since Microsoft is implementing a full Linux kernel.

5. Edit Docker Settings

   • Right-click on the Docker icon in the system tray and select Settings.

     • Advanced | CPU, Memory, Swap, Disk Image Max Size

   • Share Drives if you are processing files stored on your Windows’ disks.

     • Docker will prompt you to do this if later you point to an non-shared location

   • If you are running out disk space, your Elastic indices will be set to read only. Symptoms in Timesketch include inability to star events. Free or add disk space, then run the following command.

     • curl -XPUT "http://192.168.XXX.XXX:9200/_all/_settings" -H "Content-Type: application/json" -d "{\"index.blocks.read_only_allow_delete\": null}"

   • Troubleshooting Timesketch

     • docker-compose logs -f nginx

     • docker-compose logs -f timesketch

       • I’ve had trouble with unicode characters when attempting to export events: á (xE1) and ó (xF3), for example. The above command makes the issue clear.

6. Lock down your containers. See Alan’s tweet below.

https://twitter.com/AlanOrlikoski/status/1095448770270150656

Skadi Setup

1. Download and decompress Skadi-master.zip

2. Edit Heap Size in Skadi/Docker/.env.

   • Set to no more than half of your physical RAM. Threshold is approximately 32 GB. Guidance from Elastic.

3. Edit Skadi/Docker/timesketch_default.conf. This needs to be stored in /etc/ as timesketch.conf. Alan’s docker-compose file does this for you when you run start_skadi.ps1 later

4. You can point to a Timesketch hosted externally by editing the configuration file: EXTERNAL_HOST_URL = 'https://localhost'.

   • There is a script to use secure networking. I haven’t explored this yet. It is written in bash, so I’m not sure of the procedure for Windows.

     • Skadi/scripts/secure_network.sh

   • I’ve gotten this to work across a VPN. I’m not sure of the best way to do this, and this procedure did not work for all of my colleagues previously.

     • In the Docker Daemon tab, switch to Advanced and add the following: "bip": "192.168.XXX.XXX/24",

       • More information here.

   • In this VPN scenario I created a Docker image based on the CDQR Dockerfile to run my commands locally and then upload to Timesketch. I like having more dockerfile commands than not for when I need to add or change lines. With more layers in the image there are fewer operations per layer resulting in smaller layers. Smaller layers = faster rebuilds.

5. Customize your dockerfile with tools like Volatility. A great thing about the dockerfile is it’s self-documenting.

   • If you’re not using Skadi locally, copy the timesketch configuration file

     • COPY timesketch/timesketch_default.conf /etc

   • Bulk_Extractor-rec # this will add a bit of time to the build

     • RUN apt-get -y install flex libz-dev autoconf libssl-dev libxml2-dev tre-agrep libtre5 libtre-dev libtool libewf-dev afflib-tools sqlite3 libsqlite3-dev

     • RUN apt-get -y install openjdk-8-jdk # since Java PPA is discontinued

     • RUN apt-get update

     • RUN git clone --recursive https://github.com/4n6ist/bulk_extractor-rec.git /usr/share/bulk_extractor-rec

     • RUN cd /usr/share/bulk_extractor-rec/ && sh bootstrap.sh && ./configure && make && make install

   • NSRL Lookup # this will add a lot of time to the build

     • RUN cd /usr/share/ && wget https://dl.bintray.com/boostorg/release/1.71.0/source/boost_1_71_0.tar.bz2 && tar --bzip2 -xf boost_1_71_0.tar.bz2

     • RUN cd /usr/share/boost_1_71_0 && ./bootstrap.sh && ./b2 install

     • RUN cd /usr/share/ && wget https://github.com/Kitware/CMake/releases/download/v3.15.5/cmake-3.15.5.tar.gz

     • RUN tar -xzvf /usr/share/cmake-3.15.5.tar.gz && cd cmake-3.15.5 && ./bootstrap && make && make install

     • RUN cd /usr/share/ && git clone https://github.com/rjhansen/nsrllookup.git

     • RUN cd /usr/share/nsrllookup && cmake -D CMAKE_BUILD_TYPE=Release . && make && make install

   • To build an image locally, run this command in the directory where your dockerfile is located:

     • docker build . -t image_name/optional_tag

6. Run Skadi/Docker/start_skadi.ps1

Running Docker

1. Docker uses the basic format for execution in containers:

   • docker run [options] [image_name|id] [command to run in the container]

     • -it runs the container interactively as opposed to -dt for detached execution

     • --name labels the running container with a known value

     • --rm deletes the container after completing

     • -v maps a host directory to a container directory

2. Creating the plaso file with CDQR

   • docker run --name hostname_project --rm -v "C:\Users\xxxx\Projects\XXXX\":/Source -it image_name cdqr.py /Source/image.E01 /Destination --nohash --max_cpu

   • To simplify this, use Alan’s helper script.

3. I’ve not had much success using the GUI to upload a plaso file other than the processed XP Image from Lance Mueller. Uploading an existing plaso file via CLI using your image:

   • docker run --name hostname_project --rm -v "C:\Users\xxxx\Projects\XXXX\":/Source -it image_name cdqr.py /Source/timeline.plaso /Destination --max_cpu --es_ts hostname_project --plaso_db

   • If hosting Timesketch locally, use “--network host” when running your docker container.

   • I like naming my Elastic indices starting with the host name for readability in Timesketch.

4. I keep a text file of commands I use the most. I edit the commands there and paste them into my terminal.

5. Useful Docker commands

   • docker image ls # list available docker images

   • docker container ls # list running containers

   • docker exec -it container_name|id bash # enter bash of a running container

   • docker checkpoint create container_name|id checkpoint_name # create a “snapshot”

   • docker container prune # remove stopped containers

   • docker image prune # remove dangling images

   • docker kill container_name|id

   • docker [start | stop | restart] container_name|id

   • docker push image_name/tag

   • docker pull image_name/tag

6. Next steps

   • For your team host your Docker images in a private registry like GitLab

Why Timesketch?

• Multi-user

• Multi-timeline

• Bookmark events

• Comment on events

• Share direct link to events

Using Timesketch

1. In a browser navigate to localhost/timesketch

2. The default credentials are skadi/skadi

3. Create a sketch. Think of sketches like projects.

4. Add timelines to your sketch.

5. Click Share to give access to the sketch to specific users or groups.

   • See Timesketch User Guide to create users and groups.

6. Start searching in the Explore Tab. You can search on any field name. This blog post is really helpful.

   • parser:"winreg/bagmru/shell_items"

   • event_identifier:4624 AND "/LogonType\"\>10/"

   • A tip I got from Joachim Metz is that to uniquely identify events, you need to specify Event ID, Qualifier, and Provider.

     • You can get a list of local event publishers via the command line with “wevtutil ep”. I’m not sure if publisher == provider.

7. Save your searches as templates for you and your team to use later on the same project or new projects. Example templates. You can import and export templates.