Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
200 changes: 200 additions & 0 deletions contrib/fetch_checkpoints.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
#!/bin/sh
# Fetch checkpoint data from Dash network
#
# This script fetches block information at specific heights to create checkpoints
# for the rust-dashcore SPV client.
#
# Usage:
# # Fetch specific heights using block explorer
# ./fetch_checkpoints.sh mainnet 2400000 2450000
#
# # Fetch using RPC (requires running Dash Core node)
# RPC_URL=http://localhost:9998 RPC_USER=user RPC_PASS=pass ./fetch_checkpoints.sh mainnet 2400000
Comment on lines +2 to +12
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it works; this may be sufficient, however, I would prefer our helper scripting be done in python or some other non-bash language


set -e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

set -e makes per-height error handling ineffective; failures will abort the script instead of continuing

With set -e enabled, a non‑zero exit from get_block_rpc/get_block_explorer inside command substitution:

block_data=$(get_block_rpc "$height")

will cause the whole script to exit before the subsequent $? check, so the // Error: Failed to fetch block...; continue path is never reached. This can happen e.g. when the explorer/RPC returns invalid JSON or no blockHash.

Consider restructuring the loop so failures are handled inside an if/! guard (where set -e is suppressed) instead of relying on $? after command substitution, for example:

-    if [ -n "$RPC_URL" ] && [ -n "$RPC_USER" ] && [ -n "$RPC_PASS" ]; then
-        block_data=$(get_block_rpc "$height")
-    else
-        block_data=$(get_block_explorer "$height")
-    fi
-
-    if [ $? -ne 0 ] || [ -z "$block_data" ]; then
-        echo "// Error: Failed to fetch block at height $height"
-        continue
-    fi
+    if [ -n "$RPC_URL" ] && [ -n "$RPC_USER" ] && [ -n "$RPC_PASS" ]; then
+        if ! block_data=$(get_block_rpc "$height"); then
+            echo "// Error: Failed to fetch block at height $height"
+            continue
+        fi
+    else
+        if ! block_data=$(get_block_explorer "$height"); then
+            echo "// Error: Failed to fetch block at height $height"
+            continue
+        fi
+    fi
+
+    if [ -z "$block_data" ]; then
+        echo "// Error: Empty block response at height $height"
+        continue
+    fi

This preserves fail‑fast behavior for unrecoverable cases while allowing per‑height failures to be skipped as intended.

Also applies to: 173-196


I'll verify the review comment by examining the actual file and testing the shell behavior claims.


🏁 Script executed:

#!/bin/bash
# View the contrib/fetch_checkpoints.sh file to see the actual code
cat contrib/fetch_checkpoints.sh

Length of output: 6138


🏁 Script executed:

#!/bin/bash
# Test set -e behavior with command substitution and $? check
# This will help verify the review comment's claim

cat > /tmp/test_set_e.sh << 'EOF'
#!/bin/sh
set -e

# Test 1: command substitution + $? check under set -e
echo "=== Test 1: set -e with command substitution ==="
output=$(false 2>/dev/null)
result=$?
echo "After 'output=\$(false)', \$? = $result"
echo "Script continued (this line would not print if set -e caused exit)"

# Test 2: In a loop with continue
echo ""
echo "=== Test 2: Loop with command substitution and continue ==="
for i in 1 2 3; do
    output=$([ $i -eq 2 ] && false || echo "value_$i")
    if [ $? -ne 0 ] || [ -z "$output" ]; then
        echo "Skipping iteration $i"
        continue
    fi
    echo "Processed: $output"
done
echo "Loop completed"
EOF

sh /tmp/test_set_e.sh 2>&1 || echo "[Script exited with code: $?]"

Length of output: 77


The review comment is accurate and well-founded.

The shell test confirms the core claim: under set -e, when output=$(false) fails inside command substitution, the script exits immediately rather than allowing the subsequent $? check. Your test script never reached the lines after the failing command substitution, exiting with code 1 instead.

In the actual code at lines 186–189, both get_block_rpc() and get_block_explorer() return exit code 1 on errors (e.g., when block hash is null or missing). Under set -e, these failures inside command substitution will abort the entire script, never reaching the continue at line 193. This defeats the intended per-height error recovery.

The proposed restructuring using if ! block_data=$(...) is the correct approach, as it suppresses set -e within the guard and allows proper per-height error handling.

🤖 Prompt for AI Agents
In contrib/fetch_checkpoints.sh around lines 186-189 (affecting behavior up to
line 193), failures from get_block_rpc() and get_block_explorer() inside command
substitutions are causing the script to exit due to set -e, preventing the
intended per-height continue; change each risky substitution to a guarded form
like: use if ! var=$(command); then log the error and continue; fi so the
command failure is handled locally and the loop proceeds. Ensure you apply this
pattern to both places where command substitution can fail so the continue at
line 193 is reachable.


NETWORK=${1:-mainnet}
shift || true

# Configuration
if [ "$NETWORK" = "mainnet" ]; then
EXPLORER_URL="https://insight.dash.org/insight-api"
elif [ "$NETWORK" = "testnet" ]; then
EXPLORER_URL="https://testnet-insight.dashevo.org/insight-api"
else
echo "Error: Invalid network. Use 'mainnet' or 'testnet'"
exit 1
fi

# Check for required tools
if ! command -v curl >/dev/null 2>&1; then
echo "Error: curl is required but not installed"
exit 1
fi

if ! command -v jq >/dev/null 2>&1; then
echo "Error: jq is required but not installed. Install with: sudo apt install jq"
exit 1
fi

# Function to get current blockchain height
get_current_height() {
if [ -n "$RPC_URL" ] && [ -n "$RPC_USER" ] && [ -n "$RPC_PASS" ]; then
# Use RPC
curl -s -u "$RPC_USER:$RPC_PASS" \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","id":"checkpoint","method":"getblockchaininfo","params":[]}' \
"$RPC_URL" | jq -r '.result.blocks'
else
# Use explorer
curl -s "$EXPLORER_URL/status?q=getInfo" | jq -r '.info.blocks'
fi
}

# Function to get block by height using explorer
get_block_explorer() {
height=$1

# Get block hash at height
block_hash=$(curl -s "$EXPLORER_URL/block-index/$height" | jq -r '.blockHash')

if [ -z "$block_hash" ] || [ "$block_hash" = "null" ]; then
echo "Error: Failed to get block hash for height $height" >&2
return 1
fi

# Get full block data
curl -s "$EXPLORER_URL/block/$block_hash"
}

# Function to get block by height using RPC
get_block_rpc() {
height=$1

# Get block hash
block_hash=$(curl -s -u "$RPC_USER:$RPC_PASS" \
-H "Content-Type: application/json" \
-d "{\"jsonrpc\":\"2.0\",\"id\":\"checkpoint\",\"method\":\"getblockhash\",\"params\":[$height]}" \
"$RPC_URL" | jq -r '.result')

if [ -z "$block_hash" ] || [ "$block_hash" = "null" ]; then
echo "Error: Failed to get block hash for height $height" >&2
return 1
fi

# Get block data
curl -s -u "$RPC_USER:$RPC_PASS" \
-H "Content-Type: application/json" \
-d "{\"jsonrpc\":\"2.0\",\"id\":\"checkpoint\",\"method\":\"getblock\",\"params\":[\"$block_hash\",2]}" \
"$RPC_URL" | jq -r '.result'
}

# Function to format checkpoint as Rust code
format_checkpoint() {
block_json=$1

height=$(echo "$block_json" | jq -r '.height')
hash=$(echo "$block_json" | jq -r '.hash')
prev_hash=$(echo "$block_json" | jq -r '.previousblockhash // "0000000000000000000000000000000000000000000000000000000000000000"')
timestamp=$(echo "$block_json" | jq -r '.time')
bits=$(echo "$block_json" | jq -r '.bits')
merkle_root=$(echo "$block_json" | jq -r '.merkleroot')
nonce=$(echo "$block_json" | jq -r '.nonce')
chain_work=$(echo "$block_json" | jq -r '.chainwork // "0x0000000000000000000000000000000000000000000000000000000000000000"')

# Ensure chainwork has 0x prefix
if ! echo "$chain_work" | grep -q "^0x"; then
chain_work="0x$chain_work"
fi

# Convert bits to hex format
if echo "$bits" | grep -q "^[0-9a-fA-F]\+$"; then
bits_hex="0x$bits"
else
bits_hex="$bits"
fi

# Format as Rust code
cat <<EOF
// Block $height ($hash)
create_checkpoint(
$height,
"$hash",
"$prev_hash",
$timestamp,
$bits_hex,
"$chain_work",
"$merkle_root",
$nonce,
None, // TODO: Add masternode list if available (format: Some("ML${height}__<protocol_version>"))
),
EOF
}

# Main script
echo "Fetching checkpoints for $NETWORK..." >&2
echo "" >&2

# Get current height
current_height=$(get_current_height)
if [ -z "$current_height" ]; then
echo "Error: Failed to get current blockchain height" >&2
exit 1
fi
echo "Current blockchain height: $current_height" >&2
echo "" >&2

# If no heights specified, show usage
if [ $# -eq 0 ]; then
echo "Usage: $0 <network> <height1> [height2] [height3] ..." >&2
echo "" >&2
echo "Current height: $current_height" >&2
echo "Suggested checkpoint heights:" >&2
if [ "$NETWORK" = "mainnet" ]; then
echo " 2400000 - Block 2.4M" >&2
echo " 2450000 - Block 2.45M" >&2
echo " 2500000 - Block 2.5M (if available)" >&2
else
echo " 1200000 - Block 1.2M" >&2
echo " 1300000 - Block 1.3M" >&2
fi
echo "" >&2
echo "Examples:" >&2
echo " $0 mainnet 2400000" >&2
echo " RPC_URL=http://localhost:9998 RPC_USER=user RPC_PASS=pass $0 mainnet 2400000" >&2
exit 1
fi

# Output header
echo "// Generated checkpoints for $NETWORK"
echo "// Add these to dash-spv/src/chain/checkpoints.rs"
echo ""

# Fetch each requested height
for height in "$@"; do
if [ "$height" -gt "$current_height" ]; then
echo "// Skipping height $height (exceeds current height $current_height)"
continue
fi

echo "Fetching block at height $height..." >&2

# Get block data
if [ -n "$RPC_URL" ] && [ -n "$RPC_USER" ] && [ -n "$RPC_PASS" ]; then
block_data=$(get_block_rpc "$height")
else
block_data=$(get_block_explorer "$height")
fi

if [ $? -ne 0 ] || [ -z "$block_data" ]; then
echo "// Error: Failed to fetch block at height $height"
continue
fi

# Format and output
format_checkpoint "$block_data"
done

echo ""
echo "// NOTE: Update masternode_list_name values if the blocks have DML (Deterministic Masternode List)"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there something missing in the script?

echo "// Check coinbase transaction for masternode list information"
12 changes: 12 additions & 0 deletions dash-spv/src/chain/checkpoints.rs
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,18 @@ pub fn mainnet_checkpoints() -> Vec<Checkpoint> {
972444458,
Some("ML2300000__70232"),
),
// Block 2350000 (2025) - additional recent checkpoint
create_checkpoint(
2350000,
"00000000000000258216a62e8c7170be1207335474ddfa667092a71e3d4162d2",
"000000000000002702e07f9f3402026e4cd5707961ce54af22c873331a329708",
1759648416,
0x192f2bfb,
"0x00000000000000000000000000000000000000000000ada31a5dd056c969c842",
"3e77d2ed0c24aab79096d700cdc4fe3ea9502fcec38ee114c81c9063842a6725",
3635235496,
Some("ML2350000__70232"),
),
]
}

Expand Down
Loading
Loading