[WJ-3000] Add dev docker-compose

docs/development.md

@@ -123,6 +123,15 @@ $ docker exec -it [container id] sh
 
 One reason you may need to enter the container is to manually adjust the Wikijump config. For example, if you use a port other than 80 for your Docker container, you will need to edit `site.custom_domain` to add the port number (e.g. "`www.wikijump.localhost:8080`"). Alternatively, use curl to set the domain directly (e.g. "`-H 'www.wikijump.localhost'`")
 
+## Unhealthy Service
+
+Sometimes when starting a container locally, it will report being "unhealthy" and cause your service to be nonfunctional. There are a few reasons to check for when troubleshooting:
+
+* If the container is not `deepwell`, it may be that `deepwell` is unhealthy and a dependent container is unable to connect to it.
+* Since builds are done in runtime, the _initial_ build a container does (before it can do faster incremental builds) can take a long time. In some cases, docker will decide that the service has taken too long and is unhealthy. Try restarting it to allow the build to finish.
+* If the build fails (say due to a compiler error), the service will not be healthy. If the failure happened a while ago, the logs will not be visible at the bottom of your screen; check `./deploy logs [container]` to see what the issue may be.
+* The service may be trying to access a file which is unavailable. Check that the file hasn't been removed locally, or that if you are not using local binding (i.e. `--no-dev`), that the files have been manually installed into the container.
+
 ## Making Web Requests
 
 Once you have a local instance of Wikijump running, you may wish to make `curl` requests against it to test various pieces of its functionality. However there are a few considerations to be had given the deployment situation:

install/dev/README.md

@@ -0,0 +1,25 @@
+## DEV docker-compose
+
+For a permanent dev deployment, you can run docker-compose to start the provided containers. Feel free to modify this configuration to suit your purposes.
+
+The main infrastructural requirements are a Postgres database and two S3 buckets. Information about how to access external resources must be passed as environment variables.
+Presently, Valkey (Redis) is exposed as Docker images. If you wish, you can replace it with an external service by updating the appropriate environment variables.
+
+Note that the database is intended to be recreated on dev re-deploy (at least for the time being), which can be implemented by deleting the container before the next `up`.
+
+Referring to S3 is a bit unusual, because which variables are required differ depending on the provider. For any container which requires S3 access, all of the required S3 variables must be provided so the system knows how to connect to the S3 service.
+
+| Environment Variable    | Required              | Description |
+|-------------------------|-----------------------|-------------|
+| `S3_FILES_BUCKET`       | Depends on container. | The name of the bucket where uploaded files and avatars are kept. |
+| `S3_TEXT_BLOCKS_BUCKET` | Depends on container. | The name of the bucket where hosted text blocks are kept. |
+| `S3_AWS_REGION`         | If using AWS S3.      | The AWS region this bucket is in. |
+| `S3_REGION_NAME`        | If not using AWS S3.  | The region this bucket is in. |
+| `S3_CUSTOM_ENDPOINT`    | If not using AWS S3.  | The S3 endpoint to connect to. |
+| `S3_PATH_STYLE`         | Always.               | Boolean. Reflects whether this S3 service expects requests to be [path-style](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access) (true) or [virtual-host-style](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#virtual-hosted-style-access) (false). Some services accept both. |
+| `S3_ACCESS_KEY_ID`      | If not using an AWS profile file. | S3 credentials. |
+| `S3_SECRET_ACCESS_KEY`  | If not using an AWS profile file. | S3 credentials. |
+| `AWS_PROFILE_NAME`      | If providing credentials via AWS profile file. | The name of the AWS profile to read credentials from. |
+
+* Container `deepwell` requires Postgres, Redis, and both S3 buckets.
+* Container `wws` requires Redis and both S3 buckets.

install/dev/deepwell/Dockerfile

@@ -20,7 +20,7 @@ WORKDIR /src/deepwell
 RUN patch /src/deepwell/seeder/sites.json /src/deepwell/seeder/sites.dev.patch
 
 # Cache rust dependencies
-RUN cargo vendor
+RUN mkdir .cargo && cargo vendor > .cargo/config.toml
 
 # Build deepwell server
 RUN cargo build --release

install/dev/deepwell/deepwell-start

@@ -4,10 +4,6 @@ set -e
 # Wait for database to be ready
 pg_isready -d "$DATABASE_URL"
 
-# Wipe database for fresh schema changes
-# LOCAL and DEV only
-psql -d "$DATABASE_URL" -c 'DROP OWNED BY wikijump'
-
 # Run migrations
 cd /opt/database
 /usr/local/cargo/bin/sqlx migrate run

install/dev/docker-compose.yaml

@@ -0,0 +1,101 @@
+services:
+  cache:
+    build: valkey
+    restart: always
+    # You can add a volume for /data if you wish
+    healthcheck:
+      test: ["CMD", "valkey-cli", "ping"]
+      interval: 10s
+      timeout: 5s
+      retries: 3
+    ports:
+      - "6379:6379"
+
+  deepwell:
+    build:
+      context: ../..
+      dockerfile: install/dev/deepwell/Dockerfile
+    ports:
+      - "2747:2747"
+    links:
+      - cache
+      - database
+    environment:
+      - "REDIS_URL=redis://cache"
+    restart: always
+    healthcheck:
+      test: ["CMD", "wikijump-health-check"]
+      interval: 120s
+      timeout: 2s
+      retries: 3
+    depends_on:
+      cache:
+        condition: service_healthy
+      database:
+        condition: service_healthy
+
+  framerail:
+    build:
+      context: ../..
+      dockerfile: install/dev/framerail/Dockerfile
+    ports:
+      - "3000:3000"
+    links:
+      - deepwell
+    environment:
+      - "DEEPWELL_HOST=deepwell"
+    restart: always
+    healthcheck:
+      test: ["CMD", "wikijump-health-check"]
+      interval: 120s
+      timeout: 2s
+      retries: 3
+    depends_on:
+      deepwell:
+        condition: service_healthy
+
+  wws:
+    build:
+      context: ../..
+      dockerfile: install/dev/wws/Dockerfile
+    ports:
+      - "7000:7000"
+    links:
+      - deepwell
+      - cache
+    environment:
+      - "ADDRESS=[::]:7000"
+      - "DEEPWELL_URL=http://deepwell:2747"
+      - "REDIS_URL=redis://cache"
+    restart: always
+    healthcheck:
+      test: ["CMD", "wikijump-health-check"]
+      interval: 120s
+      timeout: 2s
+      retries: 3
+    depends_on:
+      deepwell:
+        condition: service_healthy
+      cache:
+        condition: service_healthy
+
+  caddy:
+    build:
+      context: ../..
+      dockerfile: install/dev/caddy/Dockerfile
+    ports:
+      - "80:80"
+      - "443:443"
+      - "443:443/udp"
+    links:
+      - framerail
+      - wws
+    restart: always
+    healthcheck:
+      test: ["CMD", "wikijump-health-check"]
+      interval: 30s
+      timeout: 2s
+      retries: 3
+    depends_on:
+      deepwell:
+        condition: service_healthy

install/dev/wws/Dockerfile

@@ -4,9 +4,6 @@
 
 FROM rust:latest AS rust
 
-# Install helpers
-RUN cargo install cargo-watch sqlx-cli
-
 # Install files
 COPY ./install/local/wws/wws-start /usr/local/bin/wikijump-wws-start
 COPY ./install/local/wws/health-check.sh /usr/local/bin/wikijump-health-check
@@ -17,7 +14,7 @@ COPY ./wws /src/wws
 WORKDIR /src/wws
 
 # Cache rust dependencies
-RUN cargo vendor
+RUN mkdir .cargo && cargo vendor > .cargo/config.toml
 
 # Build wws
 RUN cargo build --release

install/local/README.md

@@ -0,0 +1,9 @@
+## LOCAL docker-compose
+
+This `docker-compose.yaml` (and corresponding `docker-compose.dev.yaml`) file are used in standing up local instances of Wikijump. The convenience script `./deploy.py` is provided to make management easier, providing options for common variations.
+
+There are two important things to note about the local tier:
+1. It runs its containers in "watch mode". This means that building the service takes place after container start, not at container build time, and that if you modify local watched files, the service will rebuild and restart.
+2. Which is related to the fact that several directories are instead *mapped* into the container rather than copied into it. This way, any local changes are reflected in the container.
+
+See `docs/development.md` for more information on local deployments.

install/local/deepwell/Dockerfile

@@ -13,7 +13,8 @@ RUN apt update
 RUN apt install -y libmagic-dev
 
 # Install helpers
-RUN cargo install cargo-watch sqlx-cli
+RUN cargo install cargo-watch && \
+    cargo install sqlx-cli --no-default-features --features rustls,postgres
 
 # Install files
 COPY ./install/local/deepwell/deepwell.toml /etc/deepwell.toml

install/local/postgres/Dockerfile

@@ -1,11 +1,7 @@
-FROM postgres:16-alpine
+FROM postgres:17-alpine
 
 EXPOSE 5432
 
-# Build variables
-ARG ENVIRONMENT="local"
-ARG FILES_DOMAIN="wjfiles.test"
-
 # Create system user
 RUN adduser -S wikijump
 

install/local/wws/Dockerfile

@@ -9,7 +9,7 @@
 FROM rust:latest AS rust
 
 # Install helpers
-RUN cargo install cargo-watch sqlx-cli
+RUN cargo install cargo-watch
 
 # Install files
 COPY ./install/local/wws/wws-start /usr/local/bin/wikijump-wws-start

install/prod/deepwell/Dockerfile

@@ -14,7 +14,7 @@ COPY ./deepwell /src/deepwell
 WORKDIR /src/deepwell
 
 # Cache rust dependencies
-RUN cargo vendor
+RUN mkdir .cargo && cargo vendor > .cargo/config.toml
 
 # Build deepwell server
 RUN cargo build --release

install/prod/wws/Dockerfile

@@ -17,7 +17,7 @@ COPY ./wws /src/wws
 WORKDIR /src/wws
 
 # Cache rust dependencies
-RUN cargo vendor
+RUN mkdir .cargo && cargo vendor > .cargo/config.toml
 
 # Build wws
 RUN cargo build --release