docs: add development guide and tilt support (#744)

* chore: add tilt-based development environment

closes: #41
closes: #135

Introduce a comprehensive Tilt setup for streamlined local development
of the Kubeflow Notebooks workspace components (controller, backend,
and frontend).

Key features:

* Orchestrated development workflow: Tiltfile manages the entire
  development lifecycle including cluster setup, dependency installation,
  Docker builds, and Kubernetes deployments

* Automated infrastructure setup:
  - Kind cluster creation and verification via setup-kind.sh
  - Cert-manager installation via setup-cert-manager.sh
  - CRD installation with proper dependency ordering

* Component integration:
  - Controller: Docker builds with code generation, CRD installation,
    and Kubernetes deployment with dev-specific overrides (USE_ISTIO=false)
  - Backend: Docker builds and Kubernetes deployment with port forwarding
  - Frontend: Docker builds with webpack dev server supporting hot reloading
    and Kubernetes deployment with dev-specific overrides
    - more memory/cpu to handle webpack compilation
    - startup probe to allow for webpack compilation

* Frontend Tilt support:
  - New start:tilt npm script for Tilt-specific development
  - Enables standalone frontend mode with proper backend proxy configuration

* Developer experience improvements:
  - File watching with intelligent exclusions to prevent rebuild loops
  - Resource dependency management ensuring proper startup order
  - Optional frontend via ENABLE_FRONTEND environment variable
  - Comprehensive documentation in DEVELOPMENT.md

* Hot reloading and live updates:
  - Controller/backend/frontend changes trigger automatic Docker rebuilds and
    redeployments
  - Port forwards configured for easy local access

This setup enables developers to iterate quickly on all components with minimal manual setup and configuration.

Note: Paulo provided change to frontend logic to ensure paths are normalized to avoid routing issues when
navigating in the browser.

Signed-off-by: Andy Stoneberg <[email protected]>
Signed-off-by: Paulo Rego <[email protected]>

* chore: move frontend dockerfile

Signed-off-by: Andy Stoneberg <[email protected]>

* mathew: 1

Signed-off-by: Mathew Wicks <[email protected]>

---------

Signed-off-by: Andy Stoneberg <[email protected]>
Signed-off-by: Paulo Rego <[email protected]>
Signed-off-by: Mathew Wicks <[email protected]>
Co-authored-by: Mathew Wicks <[email protected]>

Author: andyatmiami

CONTRIBUTING.md

@@ -0,0 +1,5 @@
+# Contributing Guide - Kubeflow Notebooks v2
+
+Please see [`CONTRIBUTING.md`](https://github.com/kubeflow/notebooks/blob/main/CONTRIBUTING.md) in the `main` branch for the current contributing guidelines.
+
+For information on developing Kubeflow Notebooks v2, please see [`DEVELOPMENT_GUIDE.md`](DEVELOPMENT_GUIDE.md).

DEVELOPMENT_GUIDE.md

@@ -0,0 +1,227 @@
+# Development Guide - Kubeflow Notebooks v2
+
+Thanks for your interest in developing Kubeflow Notebooks v2! 
+
+This document provides instructions to help you set up your development environment and get started with contributing to the project.
+
+Please refer to the [`CONTRIBUTING.md`](CONTRIBUTING.md) for general guidelines on contributing to the project.
+
+## Project Structure
+
+Kubeflow Notebooks v2 is organized into several key components:
+
+- [`workspaces/controller`](workspaces/controller): A Kubernetes controller and webook written in Go.
+- [`workspaces/backend`](workspaces/backend): A backend API server written in Go _(note: also uses controller-runtime)_
+- [`workspaces/frontend`](workspaces/frontend): A React-based frontend application.
+
+## Development Workflow
+
+### STEP 1 - Find areas to contribute
+
+- Check the [issue tracker](https://github.com/kubeflow/notebooks/issues?q=is%3Aissue%20state%3Aopen%20label%3Aarea%2Fv2) for open issues
+   - __NOTE:__ please make a comment if you intend to work on an issue to avoid duplication of effort.
+- Introduce yourself on [`#kubeflow-notebooks` channel](https://www.kubeflow.org/docs/about/community/#kubeflow-slack-channels) and share your background/interests
+- Attend the [Kubeflow Notebooks WG Calls](https://www.kubeflow.org/docs/about/community/#kubeflow-community-meetings) and ask for guidance
+
+### STEP 2 - Make code changes
+
+- Keep changes focused and small
+- This guide covers two common approaches to set up your development environment:
+   - [Developing with Tilt (Recommended)](#developing-with-tilt-recommended)
+   - [Developing Locally](#developing-locally)
+- Ensure you add appropriate tests (unit and e2e) for new features or bug fixes
+- You may want to use interactive tools like [`k9s`](https://k9scli.io/) to help inspect your Kubernetes cluster during development
+
+### STEP 3 - Run tests and linting
+
+- Overall e2e Tests:
+   - __TBA:__ instructions for running e2e tests will be added here once available.
+- Controller:
+   - Unit Tests: `make test`
+   - Integration Tests: `make test-e2e`
+   - Linting: `make lint`
+- Backend:
+   - Unit Tests: `make test`
+   - Linting: `make lint`
+- Frontend:
+   - Unit Tests + Linting: `npm run test`
+
+### STEP 4 - Submit a pull request
+
+- Follow the [contributing guidelines](CONTRIBUTING.md)
+- Remember to raise your PR against the `notebooks-v2` branch
+- Ensure your PR includes:
+  - A clear description of the changes
+  - Relevant issue references
+  - Tests for new features or bug fixes
+  - Passing tests and linting checks
+- Git Tips:
+   - If you are brand new to git, see GitHub's [Git Guide](https://github.com/git-guides)
+   - For more advanced git usage, see the [Git Book](https://git-scm.com/book/en/v2)
+   - Unless you are confident it's not appropriate, please [__SQUASH__ your commits](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History) to a __single commit__ before requesting a PR review.
+      - `git rebase -i HEAD~n` where `n` is the number of commits to squash.
+      - Mark all but the first commit as `squash` or `s`.
+      - Edit the commit message as needed.
+
+## Developing with Tilt (recommended)
+
+[Tilt](https://github.com/tilt-dev/tilt) makes it much easier to develop Kubernetes controllers (especially with webhooks) locally by providing a live-reload development environment.
+
+### Tilt - Prerequisites
+
+Before using Tilt, ensure you have the following installed:
+
+- [Tilt](https://docs.tilt.dev/install.html) - v0.33.0 or later
+- [Docker](https://docs.docker.com/get-docker/) - for building and running containers
+- [Kubernetes cluster](https://kubernetes.io/docs/setup/) - a local cluster (e.g., [Kind](https://kind.sigs.k8s.io/))
+- [kubectl](https://kubernetes.io/docs/tasks/tools/) - configured to connect to your cluster
+
+### Tilt - Quick Start
+
+First, verify that you have the required tools installed and check their versions:
+
+```bash
+tilt version
+docker --version
+kubectl version --client
+```
+
+You can now start developing with Tilt by following these steps:
+
+```bash
+# (from the root of the repository)
+cd developing
+make tilt-up
+```
+
+What this does:
+
+- Set up the Kind cluster (if it doesn't exist)
+- Install cert-manager
+- Open the Tilt UI in your browser (usually http://localhost:10350)
+- Build and deploy the controller, backend, and frontend (if enabled) to your Kubernetes cluster
+- Set up port forwards for easy access
+- Enable live updates when you make code changes
+
+> [!IMPORTANT]
+> 
+> Always use `make tilt-up` instead of running `tilt up` directly. 
+> 
+> The Makefile ensures:
+> - The Kind cluster exists and is properly configured
+> - The Kubernetes context is switched to `kind-tilt`
+> - Cert-manager is installed (required for webhooks)
+> - All prerequisites are met before Tilt starts
+
+> [!TIP]
+>
+> To run Tilt without the frontend (useful for backend/controller-only development):
+>
+> ```bash
+> ENABLE_FRONTEND=false make tilt-up
+> ```
+
+Wait until all resources show green/healthy status. 
+The frontend may take a couple of minutes on first start as webpack compiles the bundle.
+
+Access the components:
+
+- Controller health: `http://localhost:8081/healthz`
+- Backend API: [Swagger UI](http://localhost:4000/api/v1/swagger/)
+- Frontend UI: `http://localhost:9000` (if enabled)
+
+You can now make changes to the codebase, and Tilt will automatically rebuild and redeploy the affected components.
+
+### Tilt - Clean Up
+
+When you are done developing with Tilt, you can stop it and clean up the resources.
+
+```bash
+# In the terminal where Tilt is running, press Ctrl+C
+
+# or in another terminal
+# (from the root of the repository)
+cd developing
+make tilt-down
+```
+
+What this does:
+
+- Stop all Tilt-managed resources
+- Clean up deployments (but not the namespace)
+
+If you want to completely remove the Kind cluster, you can do so with:
+
+```bash
+kind delete cluster --name tilt
+```
+
+## Tilt - Troubleshooting
+
+It tilt fails to start, check the logs in the Tilt UI for specific error messages.
+
+Alternatively, you can try building the components manually to identify issues:
+
+```bash
+# test controller build
+# (from the root of the repository)
+cd workspaces/controller
+make build
+
+# test backend build
+# (from the root of the repository)
+cd workspaces/backend
+make build
+
+# test frontend build
+# (from the root of the repository)
+cd workspaces/frontend
+npm ci
+npm run build:prod
+```
+
+> [!TIP]
+>
+> If you see the following error while `Tilt` is trying to build an image:
+> 
+> ```text
+> Build Failed: failed to dial gRPC: unable to upgrade to h2c, received 404
+> ```
+> 
+> Try disabling Docker BuildKit support in the terminal where you are running `make tilt-up`:
+> ```bash
+> export DOCKER_BUILDKIT=0
+> ```
+
+> [!TIP]
+> 
+> If you encounter other strange issues.
+> 
+> Try deleting the Kind cluster and starting fresh:
+> 
+> ```bash
+> kind delete cluster --name tilt
+> ```
+
+## Developing Locally
+
+If you prefer to develop without Tilt, you can set up your environment to build and run the Kubeflow Notebooks v2 controller locally.
+
+> [!WARNING]
+> 
+> When developing locally, it can be challenging to configure the webhooks correctly as they will be running on your local machine rather than inside the cluster.
+> As webhooks are critical to the behavior of Kubeflow Notebooks v2, make sure you are aware of this limitation.
+
+### Local - Prerequisites
+
+Before developing locally, ensure you have the following installed:
+
+- [Go](https://golang.org/doc/install) - v1.21.0 or later (for controller and backend)
+- [Node.js](https://nodejs.org/) - v20.0.0 or later (for frontend)
+
+### Local - Quick Start
+
+Each component has a `Makefile` or `package.json` to help with building, testing, and running the component locally.
+You should start by reviewing the these files in each component directory.
+
+We welcome contributions to improve the local development experience, please open an issue or PR!

developing/.gitignore

@@ -0,0 +1,7 @@
+# Binaries for programs and plugins
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+bin/*

developing/Makefile

@@ -0,0 +1,77 @@
+.PHONY: check-tilt tilt-up tilt-down setup-kind setup-cert-manager kustomize
+
+# Variables
+CLUSTER_NAME := tilt
+KIND_CONTEXT := kind-$(CLUSTER_NAME)
+
+# Location to install dependencies to
+LOCALBIN ?= $(shell pwd)/bin
+$(LOCALBIN):
+	mkdir -p $(LOCALBIN)
+
+# Tool Binaries
+KUSTOMIZE ?= $(LOCALBIN)/kustomize
+
+# Tool Versions
+KUSTOMIZE_VERSION ?= v5.5.0
+
+# Export KIND_EXPERIMENTAL_PROVIDER to honor it if set in user's environment
+# (e.g., KIND_EXPERIMENTAL_PROVIDER=podman for podman support)
+export KIND_EXPERIMENTAL_PROVIDER
+
+# Check if tilt is installed
+.PHONY: check-tilt
+check-tilt:
+	@if ! command -v tilt >/dev/null 2>&1; then \
+		echo "ERROR: tilt is not installed. Please install tilt first:"; \
+		echo "  curl -fsSL https://raw.githubusercontent.com/tilt-dev/tilt/master/scripts/install.sh | bash"; \
+		echo "  or visit: https://docs.tilt.dev/install.html"; \
+		exit 1; \
+	fi
+
+# Ensure kind cluster exists and context is set before running tilt
+setup-kind:
+	@echo "Setting up Kind cluster..."
+	@if [ -n "$$KIND_EXPERIMENTAL_PROVIDER" ]; then \
+		echo "Using KIND_EXPERIMENTAL_PROVIDER=$$KIND_EXPERIMENTAL_PROVIDER"; \
+	fi
+	@./scripts/setup-kind.sh
+
+# Install cert-manager (depends on kind cluster being set up)
+setup-cert-manager: setup-kind
+	@echo "Setting up cert-manager..."
+	@./scripts/setup-cert-manager.sh
+
+
+# Run tilt up with kind cluster and cert-manager setup
+tilt-up: check-tilt setup-cert-manager kustomize
+	@echo "Starting Tilt..."
+	@tilt up
+
+
+# Stop Tilt
+tilt-down: check-tilt kustomize
+	@echo "Stopping Tilt..."
+	@tilt down
+
+# Install kustomize
+kustomize: $(KUSTOMIZE)
+$(KUSTOMIZE): $(LOCALBIN)
+	$(call go-install-tool,$(KUSTOMIZE),sigs.k8s.io/kustomize/kustomize/v5,$(KUSTOMIZE_VERSION))
+
+# go-install-tool will 'go install' any package with custom target and name of binary, if it doesn't exist
+# $1 - target path with name of binary
+# $2 - package url which can be installed
+# $3 - specific version of package
+define go-install-tool
+@[ -f "$(1)-$(3)" ] || { \
+set -e; \
+package=$(2)@$(3) ;\
+echo "Downloading $${package}" ;\
+rm -f $(1) || true ;\
+GOBIN=$(LOCALBIN) go install $${package} ;\
+mv $(1) $(1)-$(3) ;\
+} ;\
+ln -sf $(1)-$(3) $(1)
+endef
+

developing/OWNERS

@@ -0,0 +1,5 @@
+labels:
+  - area/ci
+  - area/v2
+approvers:
+  - andyatmiami