Skip to content

Commit 7d82b64

Browse files
hhzhang16daiyaanarfeen
authored andcommitted
feat: remove scripts for manipulating pvcs (#4206)
Signed-off-by: Hannah Zhang <[email protected]> Signed-off-by: Daiyaan <[email protected]>
1 parent 7484c2f commit 7d82b64

File tree

6 files changed

+58
-426
lines changed

6 files changed

+58
-426
lines changed

deploy/utils/README.md

Lines changed: 37 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,7 @@ This includes:
1919
- `manifests/`
2020
- `pvc.yaml` — PVC `dynamo-pvc` for storing profiler results and configurations
2121
- `pvc-access-pod.yaml` — short‑lived pod for copying profiler results from the PVC
22-
- `kubernetes.py` — helper used by tooling to apply/read resources (e.g., access pod for PVC downloads)
23-
- `inject_manifest.py` — utility for injecting deployment configurations into the PVC for profiling
24-
- `download_pvc_results.py` — utility for downloading benchmark/profiling results from the PVC
22+
- `kubernetes.py` — helper used by tooling to apply/read resources (e.g., access pod for PVC access)
2523
- `dynamo_deployment.py` — utilities for working with DynamoGraphDeployment resources
2624
- `requirements.txt` — Python dependencies for benchmarking utilities
2725

@@ -70,64 +68,72 @@ After running the setup script, verify the resources by checking:
7068
kubectl get pvc dynamo-pvc -n $NAMESPACE
7169
```
7270

73-
### PVC Manipulation Scripts
71+
### Working with the PVC
7472

75-
These scripts interact with the Persistent Volume Claim (PVC) that stores configuration files and benchmark/profiling results. They're essential for the Dynamo benchmarking and profiling workflows.
73+
The Persistent Volume Claim (PVC) stores configuration files and benchmark/profiling results. Use `kubectl cp` to copy files to and from the PVC.
7674

77-
#### Why These Scripts Are Needed
75+
#### Setting Up PVC Access
7876

79-
1. **For Pre-Deployment Profiling**: The profiling job needs access to your Dynamo deployment configurations (DGD manifests) to test different parallelization strategies
80-
2. **For Retrieving Results**: Both benchmarking and profiling jobs write their results to the PVC, which you need to download for analysis
77+
First, create a temporary access pod to interact with the PVC:
8178

82-
#### Script Usage
79+
```bash
80+
# Create access pod
81+
kubectl apply -f deploy/utils/manifests/pvc-access-pod.yaml -n $NAMESPACE
82+
83+
# Wait for pod to be ready
84+
kubectl wait --for=condition=Ready pod/pvc-access-pod -n $NAMESPACE --timeout=60s
85+
```
8386

84-
**Inject deployment configurations for profiling:**
87+
#### Copying Files to the PVC
88+
89+
**Copy deployment configurations for profiling:**
8590

8691
```bash
87-
# The profiling job reads your DGD config from the PVC
88-
# IMPORTANT: All paths must start with /data/ for security reasons
89-
python3 -m deploy.utils.inject_manifest \
90-
--namespace $NAMESPACE \
91-
--src ./my-disagg.yaml \
92-
--dest /data/configs/disagg.yaml
92+
# Copy a single file
93+
kubectl cp ./my-disagg.yaml $NAMESPACE/pvc-access-pod:/data/configs/disagg.yaml
94+
95+
# Copy an entire directory
96+
kubectl cp ./configs/ $NAMESPACE/pvc-access-pod:/data/configs/
9397
```
9498

99+
#### Downloading Files from the PVC
100+
95101
**Download benchmark results:**
96102

97103
```bash
98-
# After benchmarking completes, download results
99-
python3 -m deploy.utils.download_pvc_results \
100-
--namespace $NAMESPACE \
101-
--output-dir ./benchmarks/results \
102-
--folder /data/results \
103-
--no-config # optional: skip *.yaml/*.yml in the download
104+
# Download entire results directory
105+
kubectl cp $NAMESPACE/pvc-access-pod:/data/results ./benchmarks/results
106+
107+
# Download a specific subdirectory
108+
kubectl cp $NAMESPACE/pvc-access-pod:/data/results/benchmark-name ./benchmarks/results/benchmark-name
104109
```
105110

106111
**Download profiling results (optional, for local inspection):**
107112

108113
```bash
109114
# Optional: Download profiling data for local analysis
110115
# The planner reads directly from the PVC, so this is only needed for inspection
111-
python3 -m deploy.utils.download_pvc_results \
112-
--namespace $NAMESPACE \
113-
--output-dir ./profiling_data \
114-
--folder /data
116+
kubectl cp $NAMESPACE/pvc-access-pod:/data ./profiling_data
115117
```
116118

117119
> **Note on Profiling Results**: When using DGDR (DynamoGraphDeploymentRequest) for SLA-driven profiling, profiling data is stored in `/data/` on the PVC. The planner component reads this data directly from the PVC, so downloading is **optional** - only needed if you want to inspect the profiling results locally (e.g., view performance plots, check configurations).
118120
119-
#### Path Requirements
121+
#### Cleanup Access Pod
122+
123+
When finished, delete the access pod:
124+
125+
```bash
126+
kubectl delete pod pvc-access-pod -n $NAMESPACE
127+
```
120128

121-
**Important**: The PVC is mounted at `/data` in the access pod for security reasons. All destination paths must start with `/data/`.
129+
#### Path Structure
122130

123-
**Common path patterns:**
131+
**Common path patterns in the PVC:**
124132
- `/data/configs/` - Configuration files (DGD manifests)
125133
- `/data/results/` - Benchmark results (for download after benchmarking jobs)
126134
- `/data/` - Profiling data (used directly by planner, typically not downloaded)
127135
- `/data/benchmarking/` - Benchmarking artifacts
128136

129-
**User-friendly error messages**: If you forget the `/data/` prefix, the script will show a helpful error message with the correct path and example commands.
130-
131137
#### Next Steps
132138

133139
For complete benchmarking and profiling workflows:

deploy/utils/download_pvc_results.py

Lines changed: 0 additions & 219 deletions
This file was deleted.

0 commit comments

Comments
 (0)