Develop local with VSCode and Minikube

Introduction

This document will describe how to setup a local development environment using VSCode as the IDE, Minikube as the k8s cluster to run Turbinia and Skaffold to handle the development cycle.

This setup will provide run time debugging with breakpoints/watches in VSCode as well as hot-reloading of code changes to the live Turbinia deployment without having to rebuild the containers or restart the deployment.

NOTE: This setup has been tested by the Turbinia developers in the following configurations.

  • MacAir M3 8GB with Docker Desktop

  • GCE (e2-standard-4) Debian 12/bookworm 16GB with Docker Engine (Running on GCP using the VSCode Remote SSH extension)

    • Setting up the VSCode Remote-SSH extension is outside of scope for this documentation but more information can be found here.

    • NOTE: The GCE machine is used as a local machine running in GCP where you will install minikube and the other dependencies on.

  • If you want to develop with a full VSCode web interface you can use the GCP Cloud Shell Editor

    • This setup has all the dependencies and VSCode extensions needed for Turbinia development pre-installed and configured. You can continue the setup at the section here.

    • Take note of the limitations here

    • If the limitations are an issue for you, have a look into the (paid) version called Google Cloud Workstation.

Components

Minikube

As the main supported way to run Turbinia is on k8s it makes sense to also develop as close to a production setup as possible. Minikube is local Kubernetes cluster so you can run the complete Turbinia stack from your laptop.

Skaffold

Skaffold is a development workflow manager that takes care of building and deploying and monitoring for code changes so you can focus on writing code instead of manually building or restarting containers after every code change.

VSCode + Gemini Code Assist/Google Cloud Code extension

VSCode is an IDE that combined with the Google Cloud Code extension will manage the dependencies needed for the development workflow and provide an interface to debug through breakpoints and watches. The extension will manage minikube, skaffold, kubectl and their setup and configuration in a separate folder as not to interfere with your system configuration.

NOTE: Gemini Code Assist is disabled by default and can be enabled through the extension setting Cloudcode -> Duet AI -> Enable. This will give you LLM supported code advice. See here for further details about Gemini Code Assist.

Setup

Requirements

Please install the following requirements on your system.

  • Docker, you can install Docker Desktop or only the engine.

    • Make sure docker can be execute as non-root (docker run hello-world). See here for instructions.

    • Some operating systems can run Minikube (faster) with other hypervisors. If you choose so expect some extra custom setup to be done. We advice the use of Docker for Minikube.

  • Helm, to manage the deployment of Turbinia.

  • VSCode, our IDE of choice for this setup.

Configure VSCode

Start VSCode and install the following extensions:

Restart VSCode. Once VSCode has restarted let’s install the development dependencies (minikube, skaffold and kubectl). We will let the Google Cloud Code extension manage the dependencies and they will be installed in a separate folder, not in your system folders. In VSCode:

  • Control-Shift-P (or Command-Shift-P on Macs) to open the command menu

  • Choose Cloud Code:  Control Minikube

  • Choose Continue with Managed dependencies

This will have VSCode install the dependencies (minikube, skaffold and kubectl) and it can take a while.

Cloud Code Dependencies Install

Start Minkube cluster

We will now start the minikube k8s cluster

  • Control-Shift-P (or Command-Shift-P on Macs) to open the command menu

  • Choose Cloud Code:  Control Minikube

  • Choose Start

Minikube Start

Minikube Start Output

NOTE: If you want to change the default cluster CPU and Memory usage, you can set those before starting the cluster

minikube config set cpus 4
minikube config set memory 16384

Turbinia source and deployment code

Now we have VSCode setup we are going to get a copy of the Turbinia source and deployment code. Clone the Turbinia repository by forking the Turbinia repository into your own Github account and clone it locally from there.

git clone ssh://git@github.com:[YOUR_GITHUB_ACCOUNT]/turbinia.git

Let’s get the helm charts for the Turbinia deployment. In your cloned turbinia repository

mkdir charts && cd charts
helm repo add osdfir-charts https://google.github.io/osdfir-infrastructure
helm pull osdfir-charts/turbinia --untar && cd ..

Open up the turbinia folder in VSCode.

Prepare Cluster

Open a terminal (inside VSCode is the easiest, but any terminal will do) and let’s configure skaffold, the local cluster and the additional helm repository for Redis.

skaffold config set --global local-cluster true
eval $(minikube -p minikube docker-env)
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add kube-prometheus-stack https://prometheus-community.github.io/helm-charts

Verify Setup

Execute a build with skaffold (from the root of the cloned Turbinia Github repository)

skaffold build

This will build a Turbinia Server container image successfully if skaffold has been correctlty setup and configured as described above.

Skaffold Build

Install the Turbinia Client

We will install the Turbinia client into a Python virtual environment to be able to control Turbinia during our development workflow.

python -m venv .venv
source .venv/bin/activate
pip install poetry
cd turbinia/api/cli
poetry install

Create the Turbinia Client configuration file in $HOME/.turbinia_api_config.json using the base configuration from here.

Turbinia Client

Run

Now we are ready to run the development cluster of Turbinia. This will startup the Turbinia API server, the worker and the server in the local minikube k8s cluster.

skaffold dev

NOTE: if one of the services fails to deploy, try again. Sometimes a time-out occurs due to Redis starting too slow.

The Turbinia WebUI should now be available on http://localhost:8000.

Turbinia WebUI

Verify debugging and hot-reloading

Debugging

The Turbinia containers are started with the python debugger enabled and listening on port 10000 (worker) and 20000 (server). VSCode launch debug configurations have been provided.

While the cluster is running in development mode with skaffold you can use those launch configuration to connect to the worker and server (yes, at the same time if you want that!), set breakpoints and inspect watches.

Turbinia VSCode Launch Configurations

Hot-reloading

Hot reloading has been enabled through the python jurigged module and will monitor for any change on the /home/turbinia folder in the containers. When you change a file in VSCode, skaffold will sync that file into the container and jurigged will hot reload that file into the running process.

Turbinia Hot Reloading

NOTE: As python hot reloading of code into an already running process is tricky it may not work in all cases (see Caveats section here). If that happens let skaffold rebuild the container by touching the skaffold.yamlfile.

Keep in mind that hot-reloading:

  • will load a changed Python source code file as Python byte code in memory of a running process

  • will not re-execute any change code automatically

  • means that you need to execute something to trigger the code path and code you changed

Test Run

Let’s test the whole setup by executing a request with a disk image located at test_data/artifact_disk.dd.

Upload the disk to the shared /mnt/turbiniavolume folder using the evidence upload command.

turbinia-client evidence upload -p test_data/artifact_disk.dd  12345

# /home/user/turbinia/test_data/artifact_disk.dd:
* Original Name: artifact_disk.dd
* File Name: artifact_disk_2024-08-01T06:46:51.610412Z.dd
* File Path: /mnt/turbiniavolume/upload/12345/artifact_disk_2024-08-01T06:46:51.610412Z.dd
* Size: 20971520

Start a Turbinia rawdisk request.

turbinia-client submit rawdisk --source_path /mnt/turbiniavolume/upload/12345/artifact_disk_2024-08-01T06:46:51.610412Z.dd

Sending request: {'evidence': {'type': 'RawDisk', 'source_path': '/mnt/turbiniavolume/upload/12345/artifact_disk_2024-08-01T06:46:51.610412Z.dd'}, 'request_options': {}} 
Received response: {'request_id': '4d76df84849c484a835d37fbc7668122'}

You can check the Turbinia WebUI at http://localhost:8000 or use the turbinia-client to verify the status of the request.

turbinia-client status request 4d76df84849c484a835d37fbc7668122

Next

Try our Turbinia minikube development 101 Codelab here

Cleanup development environment

  • Remove all minikube clusters minikube delete --all --purge

  • Remove all images/containers/volumes from Docker docker system prune -a

  • Remove the Google Cloud Code Extension from VSCode

Troubleshooting and Tips

Debugging API Server

You can enable debugging by uncommenting the API Server build and setValueTemplates sections in the skaffold.yaml file.

Developing and debugging the WebUI

The WebUI is developed with Vue and build using node/npm. This means there is no easy way to debug this in the container. We suggest developing with a local running WebUI using npm run dev in the web/ folder. This will spinup a development web server for the WebUI that will talk to the API server managed by skaffold.

Google Cloud Code tools not found (minikube, skaffold, kubectl)

The extension will add the PATH automatically to your config. But if you have a different or custom shell configuration this may fail. Add the path manually to your PATH.

  • Linux: $HOME/.cache/cloud-code/installer/google-cloud-sdk/bin

  • Mac: $HOME/Library/Application Support/cloud-code/installer/google-cloud-sdk/bin

Installing K9s

Install k9s to easily manage your k8s cluster (eg logs and shells into pods)

skaffold commands

  • Enable more debugging info with skaffold dev -v debug

minikube commands

  • Status minikube status

  • Stop minikube stop

  • Purge clusters to start over minikube delete --all --purge

helm commands

  • Remove stale Turbinia cluster helm list && helm uninstall dev-release

kubectl commands

Advice is to install and use k9s(see above) but kubectl can be used as well.

  • kubectl get pods

  • kubctl logs [podname]

  • kubectl exec -it [podname] -- /bin/bash