DjangoBench V2: enhance inbox endpoint

benchpress/config/jobs.yml

@@ -251,7 +251,7 @@
           - 'benchmarks/django_workload/django-workload/django-workload/log_load_balancer'
           - 'benchmarks/django_workload/django-workload/client/*.log'
           - 'benchmarks/django_workload/*.log'
-          - '/tmp/siege_out_*'
+          - '/tmp/wrk_out_*'
           - 'benchmarks/django_workload/breakdown.csv'
 
 - benchmark: django_workload
@@ -332,7 +332,7 @@
           - 'benchmarks/django_workload/django-workload/django-workload/log_load_balancer'
           - 'benchmarks/django_workload/django-workload/client/*.log'
           - 'benchmarks/django_workload/*.log'
-          - '/tmp/siege_out_*'
+          - '/tmp/wrk_out_*'
           - 'benchmarks/django_workload/breakdown.csv'
 
 - benchmark: django_workload
@@ -372,7 +372,7 @@
           - 'benchmarks/django_workload/django-workload/django-workload/log_load_balancer'
           - 'benchmarks/django_workload/django-workload/client/*.log'
           - 'benchmarks/django_workload/*.log'
-          - '/tmp/siege_out_*'
+          - '/tmp/wrk_out_*'
           - 'benchmarks/django_workload/breakdown.csv'
 
 - benchmark: django_workload
@@ -412,7 +412,7 @@
           - 'benchmarks/django_workload/django-workload/django-workload/log_load_balancer'
           - 'benchmarks/django_workload/django-workload/client/*.log'
           - 'benchmarks/django_workload/*.log'
-          - '/tmp/siege_out_*'
+          - '/tmp/wrk_out_*'
           - 'benchmarks/django_workload/breakdown.csv'
 
 
@@ -497,7 +497,7 @@
           - 'benchmarks/django_workload/django-workload/django-workload/log_load_balancer'
           - 'benchmarks/django_workload/django-workload/client/*.log'
           - 'benchmarks/django_workload/*.log'
-          - '/tmp/siege_out_*'
+          - '/tmp/wrk_out_*'
           - 'benchmarks/django_workload/breakdown.csv'
 
 - name: feedsim_default

packages/django_workload/README.md

@@ -180,9 +180,9 @@ For `django_workload_default` and `django_workload_arm` jobs:
   * `duration` \- Duration of each iteration of test, default `5M` (5 minutes)
   * `iterations` \- Number of iterations to run, default 7
   * `reps`: Number of requests (per client worker) that the load generator will send in each iteration.
-    This will override `duration` and is useful to workaround the hanging problem of Siege (the load generator).
-    Note the total number of requests that Siege will send will be `reps * client_workers`, where
-    `client_workers = 1.2 * NPROC`.
+    This is useful when you want to run the benchmark with a fixed number of requests rather than
+    a fixed duration.  Unlike the behavior of Siege, now the total number of requests that wrk will send
+    will be `reps`, _not_ `reps * iterations`.
   * `interpreter` \- Which python interpreter to use: choose between `cpython` or `cinder`.
     Defaults to `cpython`.
   * `use_async` \- If this is set to 1, DjangoBench will use this new asynchronous server stack;
@@ -276,9 +276,9 @@ in these paths (based on the DCPerf repo's root folder):
   * `benchmarks/django_workload/django-workload/django-workload/log_load_balancer/*.log`
 - Traditional uWSGI synchronous server log:
   * `benchmarks/django_workload/django-workload/django-workload/django-uwsgi.log`
-- Siege log:
+- Wrk log:
   * `benchmarks/django_workload/django-workload/client/*.log`
-  * `/tmp/siege_out_*`
+  * `/tmp/wrk_out_*`
 - Cassandra and memcached log:
   * `benchmarks/django_workload/cassandra.log`
   * `benchmarks/django_workload/memcached.log`
@@ -326,17 +326,14 @@ In this case, please start Cassandra DB by running the following command:
 Where `<host-ip>` is the IP address that Cassandra is supposed to bind and the
 benchmarking machine can connect to.
 
-### Siege hanging
+### Load generator options
 
-Django benchmark should finish in around 35 minutes. If you see it not finishing
-for long time and the CPU utilization is very low, it's probably because the
-load tester Siege run into deadlock and hang. This a known issue being discussed
-in [Siege's repo](https://github.com/JoeDog/siege/issues/4) and it may happen more
-frequently on newer platforms.
+DjangoBench uses [wrk](https://github.com/wg/wrk) as the load generator, which is a modern
+HTTP benchmarking tool capable of generating significant load with a single multi-threaded
+process.
 
-As a workaround, we provide an option to run the benchmark with fixed number of
-requests instead of fixed amount of time. The benchmarking command will be the
-following:
+By default, the benchmark runs for a fixed duration (5 minutes per iteration). However,
+you can also run the benchmark with a fixed number of requests instead:
 
 ```
 ./benchpress_cli.py run django_workload_default -r clientserver -i '{"db_addr": "<db-server-ip>", "reps": <REPS>, "iterations": <ITER>}'
@@ -351,14 +348,6 @@ If you do not wish to change the number of iterations, then run the following:
 ./benchpress_cli.py run django_workload_default -r clientserver -i '{"db_addr": "<db-server-ip>", "reps": <REPS>}'
 ```
 
-#### How to choose the number of reps?
-
-We recommend the REPS to be somewhere between 3000 and 8000. The runtime will
-depend on the computation power of your CPU.
-If you have already run the default time-based Django benchmark once, you can
-make REPS to be `wc -l /tmp/siege_out_1` divided by the number of your logical
-CPU cores. That way the runtime of each iteration will be close to 5 minutes.
-
 ### Cinder-specific issues
 
 **Note**: Cinder currently does not work on ARM platforms when JIT is enabled

packages/django_workload/install_django_workload.sh

@@ -174,5 +174,15 @@ pip install -e . --no-index --find-links file://"$OUT/django-workload/django-wor
 
 deactivate
 
-cd "${BENCHPRESS_ROOT}/packages/django_workload" || exit 1
-bash -x install_siege.sh
+# 6. Install wrk
+WRK_VERSION="4.2.0"
+pushd "${DJANGO_WORKLOAD_ROOT}" || exit 1
+if ! [ -d wrk ]; then
+  git clone --branch "${WRK_VERSION}" https://github.com/wg/wrk
+  pushd wrk || exit 1
+  git apply --check "${DJANGO_PKG_ROOT}/templates/wrk.diff" && \
+    git apply "${DJANGO_PKG_ROOT}/templates/wrk.diff"
+  make
+  popd # wrk
+fi
+popd # "${DJANGO_WORKLOAD_ROOT}"

packages/django_workload/install_django_workload_aarch64.sh

@@ -313,12 +313,17 @@ popd  # ${DJANGO_SERVER_ROOT}
 
 echo "Python dependencies installation completed"
 
-# Install siege
-pushd "${DJANGO_PKG_ROOT}" || exit 1
-bash -x install_siege.sh
-popd
-
-echo "Siege installed successfully"
+WRK_VERSION="4.2.0"
+pushd "${DJANGO_WORKLOAD_ROOT}" || exit 1
+if ! [ -d wrk ]; then
+  git clone --branch "${WRK_VERSION}" https://github.com/wg/wrk
+  pushd wrk || exit 1
+  git apply --check "${DJANGO_PKG_ROOT}/templates/wrk.diff" && \
+    git apply "${DJANGO_PKG_ROOT}/templates/wrk.diff"
+  make && echo "Wrk built successfully"
+  popd # wrk
+fi
+popd # "${DJANGO_WORKLOAD_ROOT}"
 
 # =====================================================================
 # Step 7: Build and Install Proxygen (for DjangoBench V2)
@@ -415,6 +420,9 @@ DATASET_DIR="${DJANGO_SERVER_ROOT}/django_workload/feed_flow/dataset"
 mkdir -p "${DATASET_DIR}/text"
 mkdir -p "${DATASET_DIR}/binary"
 
+DATASET_DIR2="${DJANGO_SERVER_ROOT}/django_workload/feed_flow/dataset"
+ln -s "${DATASET_DIR}" "${DATASET_DIR2}"
+
 # Download Silesia Corpus if not already present
 if [ ! -f "${DJANGO_WORKLOAD_ROOT}/silesia.zip" ]; then
     echo "Downloading Silesia Corpus dataset..."

packages/django_workload/install_django_workload_aarch64_ubuntu22.sh

@@ -301,12 +301,17 @@ popd  # ${DJANGO_SERVER_ROOT}
 
 echo "Python dependencies installation completed"
 
-# Install siege
-pushd "${DJANGO_PKG_ROOT}" || exit 1
-bash -x install_siege.sh
-popd
-
-echo "Siege installed successfully"
+WRK_VERSION="4.2.0"
+pushd "${DJANGO_WORKLOAD_ROOT}" || exit 1
+if ! [ -d wrk ]; then
+  git clone --branch "${WRK_VERSION}" https://github.com/wg/wrk
+  pushd wrk || exit 1
+  git apply --check "${DJANGO_PKG_ROOT}/templates/wrk.diff" && \
+    git apply "${DJANGO_PKG_ROOT}/templates/wrk.diff"
+  make && echo "Wrk built successfully"
+  popd # wrk
+fi
+popd # "${DJANGO_WORKLOAD_ROOT}"
 
 # =====================================================================
 # Step 7: Build and Install Proxygen (for DjangoBench V2)
@@ -403,6 +408,9 @@ DATASET_DIR="${DJANGO_SERVER_ROOT}/django_workload/feed_flow/dataset"
 mkdir -p "${DATASET_DIR}/text"
 mkdir -p "${DATASET_DIR}/binary"
 
+DATASET_DIR2="${DJANGO_SERVER_ROOT}/django_workload/feed_flow/dataset"
+ln -s "${DATASET_DIR}" "${DATASET_DIR2}"
+
 # Download Silesia Corpus if not already present
 if [ ! -f "${DJANGO_WORKLOAD_ROOT}/silesia.zip" ]; then
     echo "Downloading Silesia Corpus dataset..."

packages/django_workload/install_django_workload_x86_64_centos9.sh

@@ -310,12 +310,17 @@ popd  # ${DJANGO_SERVER_ROOT}
 
 echo "Python dependencies installation completed"
 
-# Install siege
-pushd "${DJANGO_PKG_ROOT}" || exit 1
-bash -x install_siege.sh
-popd
-
-echo "Siege installed successfully"
+WRK_VERSION="4.2.0"
+pushd "${DJANGO_WORKLOAD_ROOT}" || exit 1
+if ! [ -d wrk ]; then
+  git clone --branch "${WRK_VERSION}" https://github.com/wg/wrk
+  pushd wrk || exit 1
+  git apply --check "${DJANGO_PKG_ROOT}/templates/wrk.diff" && \
+    git apply "${DJANGO_PKG_ROOT}/templates/wrk.diff"
+  make && echo "Wrk built successfully"
+  popd # wrk
+fi
+popd # "${DJANGO_WORKLOAD_ROOT}"
 
 # =====================================================================
 # Step 7: Build and Install Proxygen (for DjangoBench V2)
@@ -412,6 +417,9 @@ DATASET_DIR="${DJANGO_SERVER_ROOT}/django_workload/feed_flow/dataset"
 mkdir -p "${DATASET_DIR}/text"
 mkdir -p "${DATASET_DIR}/binary"
 
+DATASET_DIR2="${DJANGO_SERVER_ROOT}/django_workload/feed_flow/dataset"
+ln -s "${DATASET_DIR}" "${DATASET_DIR2}"
+
 # Download Silesia Corpus if not already present
 if [ ! -f "${DJANGO_WORKLOAD_ROOT}/silesia.zip" ]; then
     echo "Downloading Silesia Corpus dataset..."

packages/django_workload/install_django_workload_x86_64_ubuntu22.sh

@@ -298,12 +298,17 @@ popd  # ${DJANGO_SERVER_ROOT}
 
 echo "Python dependencies installation completed"
 
-# Install siege
-pushd "${DJANGO_PKG_ROOT}" || exit 1
-bash -x install_siege.sh
-popd
-
-echo "Siege installed successfully"
+WRK_VERSION="4.2.0"
+pushd "${DJANGO_WORKLOAD_ROOT}" || exit 1
+if ! [ -d wrk ]; then
+  git clone --branch "${WRK_VERSION}" https://github.com/wg/wrk
+  pushd wrk || exit 1
+  git apply --check "${DJANGO_PKG_ROOT}/templates/wrk.diff" && \
+    git apply "${DJANGO_PKG_ROOT}/templates/wrk.diff"
+  make && echo "Wrk built successfully"
+  popd # wrk
+fi
+popd # "${DJANGO_WORKLOAD_ROOT}"
 
 # =====================================================================
 # Step 7: Build and Install Proxygen (for DjangoBench V2)
@@ -400,6 +405,9 @@ DATASET_DIR="${DJANGO_SERVER_ROOT}/django_workload/feed_flow/dataset"
 mkdir -p "${DATASET_DIR}/text"
 mkdir -p "${DATASET_DIR}/binary"
 
+DATASET_DIR2="${DJANGO_SERVER_ROOT}/django_workload/feed_flow/dataset"
+ln -s "${DATASET_DIR}" "${DATASET_DIR2}"
+
 # Download Silesia Corpus if not already present
 if [ ! -f "${DJANGO_WORKLOAD_ROOT}/silesia.zip" ]; then
     echo "Downloading Silesia Corpus dataset..."

packages/django_workload/srcs/bin/run.sh

@@ -156,9 +156,9 @@ cleanup() {
     rm -f cassandra.pid
   fi
 
-  # Kill Siege
-  SIEGE_PID="$(pgrep siege)"
-  [ -n "$SIEGE_PID" ] && { echo "Killing siege"; kill -9 "$SIEGE_PID" 2>/dev/null || true; }
+  # Kill wrk
+  WRK_PID="$(pgrep wrk)"
+  [ -n "$WRK_PID" ] && { echo "Killing wrk"; kill -9 "$WRK_PID" 2>/dev/null || true; }
 
   echo "Done"
   if [ "$CLEANUP_REQS" -gt 0 ]; then
@@ -308,7 +308,8 @@ run_benchmark() {
   DURATION="$_duration" \
   LOG="$_siege_logs_path" \
   SOURCE="$_urls_path" \
-  python3 ./run-siege -i "${iterations}" -r "${reps}" -R "${BENCHPRESS_ROOT}/packages/django_workload/templates/siege.conf"
+  BASE_URL="http://localhost:8000" \
+  python3 ./run-wrk -i "${iterations}" -r "${reps}"
 }
 
 load_snapshot(){