-
Notifications
You must be signed in to change notification settings - Fork 8
feat(spv): add checkpoints and start sync from last checkpoint (#165,… #186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: v0.41-dev
Are you sure you want to change the base?
Changes from all commits
7505274
d70c1ff
70abe7f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,200 @@ | ||
| #!/bin/sh | ||
| # Fetch checkpoint data from Dash network | ||
| # | ||
| # This script fetches block information at specific heights to create checkpoints | ||
| # for the rust-dashcore SPV client. | ||
| # | ||
| # Usage: | ||
| # # Fetch specific heights using block explorer | ||
| # ./fetch_checkpoints.sh mainnet 2400000 2450000 | ||
| # | ||
| # # Fetch using RPC (requires running Dash Core node) | ||
| # RPC_URL=http://localhost:9998 RPC_USER=user RPC_PASS=pass ./fetch_checkpoints.sh mainnet 2400000 | ||
|
|
||
| set -e | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chainset -e makes per-height error handling ineffective; failures will abort the script instead of continuing With block_data=$(get_block_rpc "$height")will cause the whole script to exit before the subsequent Consider restructuring the loop so failures are handled inside an - if [ -n "$RPC_URL" ] && [ -n "$RPC_USER" ] && [ -n "$RPC_PASS" ]; then
- block_data=$(get_block_rpc "$height")
- else
- block_data=$(get_block_explorer "$height")
- fi
-
- if [ $? -ne 0 ] || [ -z "$block_data" ]; then
- echo "// Error: Failed to fetch block at height $height"
- continue
- fi
+ if [ -n "$RPC_URL" ] && [ -n "$RPC_USER" ] && [ -n "$RPC_PASS" ]; then
+ if ! block_data=$(get_block_rpc "$height"); then
+ echo "// Error: Failed to fetch block at height $height"
+ continue
+ fi
+ else
+ if ! block_data=$(get_block_explorer "$height"); then
+ echo "// Error: Failed to fetch block at height $height"
+ continue
+ fi
+ fi
+
+ if [ -z "$block_data" ]; then
+ echo "// Error: Empty block response at height $height"
+ continue
+ fiThis preserves fail‑fast behavior for unrecoverable cases while allowing per‑height failures to be skipped as intended. Also applies to: 173-196 I'll verify the review comment by examining the actual file and testing the shell behavior claims. 🏁 Script executed: #!/bin/bash
# View the contrib/fetch_checkpoints.sh file to see the actual code
cat contrib/fetch_checkpoints.shLength of output: 6138 🏁 Script executed: #!/bin/bash
# Test set -e behavior with command substitution and $? check
# This will help verify the review comment's claim
cat > /tmp/test_set_e.sh << 'EOF'
#!/bin/sh
set -e
# Test 1: command substitution + $? check under set -e
echo "=== Test 1: set -e with command substitution ==="
output=$(false 2>/dev/null)
result=$?
echo "After 'output=\$(false)', \$? = $result"
echo "Script continued (this line would not print if set -e caused exit)"
# Test 2: In a loop with continue
echo ""
echo "=== Test 2: Loop with command substitution and continue ==="
for i in 1 2 3; do
output=$([ $i -eq 2 ] && false || echo "value_$i")
if [ $? -ne 0 ] || [ -z "$output" ]; then
echo "Skipping iteration $i"
continue
fi
echo "Processed: $output"
done
echo "Loop completed"
EOF
sh /tmp/test_set_e.sh 2>&1 || echo "[Script exited with code: $?]"Length of output: 77 The review comment is accurate and well-founded. The shell test confirms the core claim: under In the actual code at lines 186–189, both The proposed restructuring using 🤖 Prompt for AI Agents |
||
|
|
||
| NETWORK=${1:-mainnet} | ||
| shift || true | ||
|
|
||
| # Configuration | ||
| if [ "$NETWORK" = "mainnet" ]; then | ||
| EXPLORER_URL="https://insight.dash.org/insight-api" | ||
| elif [ "$NETWORK" = "testnet" ]; then | ||
| EXPLORER_URL="https://testnet-insight.dashevo.org/insight-api" | ||
| else | ||
| echo "Error: Invalid network. Use 'mainnet' or 'testnet'" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Check for required tools | ||
| if ! command -v curl >/dev/null 2>&1; then | ||
| echo "Error: curl is required but not installed" | ||
| exit 1 | ||
| fi | ||
|
|
||
| if ! command -v jq >/dev/null 2>&1; then | ||
| echo "Error: jq is required but not installed. Install with: sudo apt install jq" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Function to get current blockchain height | ||
| get_current_height() { | ||
| if [ -n "$RPC_URL" ] && [ -n "$RPC_USER" ] && [ -n "$RPC_PASS" ]; then | ||
| # Use RPC | ||
| curl -s -u "$RPC_USER:$RPC_PASS" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"jsonrpc":"2.0","id":"checkpoint","method":"getblockchaininfo","params":[]}' \ | ||
| "$RPC_URL" | jq -r '.result.blocks' | ||
| else | ||
| # Use explorer | ||
| curl -s "$EXPLORER_URL/status?q=getInfo" | jq -r '.info.blocks' | ||
| fi | ||
| } | ||
|
|
||
| # Function to get block by height using explorer | ||
| get_block_explorer() { | ||
| height=$1 | ||
|
|
||
| # Get block hash at height | ||
| block_hash=$(curl -s "$EXPLORER_URL/block-index/$height" | jq -r '.blockHash') | ||
|
|
||
| if [ -z "$block_hash" ] || [ "$block_hash" = "null" ]; then | ||
| echo "Error: Failed to get block hash for height $height" >&2 | ||
| return 1 | ||
| fi | ||
|
|
||
| # Get full block data | ||
| curl -s "$EXPLORER_URL/block/$block_hash" | ||
| } | ||
|
|
||
| # Function to get block by height using RPC | ||
| get_block_rpc() { | ||
| height=$1 | ||
|
|
||
| # Get block hash | ||
| block_hash=$(curl -s -u "$RPC_USER:$RPC_PASS" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d "{\"jsonrpc\":\"2.0\",\"id\":\"checkpoint\",\"method\":\"getblockhash\",\"params\":[$height]}" \ | ||
| "$RPC_URL" | jq -r '.result') | ||
|
|
||
| if [ -z "$block_hash" ] || [ "$block_hash" = "null" ]; then | ||
| echo "Error: Failed to get block hash for height $height" >&2 | ||
| return 1 | ||
| fi | ||
|
|
||
| # Get block data | ||
| curl -s -u "$RPC_USER:$RPC_PASS" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d "{\"jsonrpc\":\"2.0\",\"id\":\"checkpoint\",\"method\":\"getblock\",\"params\":[\"$block_hash\",2]}" \ | ||
| "$RPC_URL" | jq -r '.result' | ||
| } | ||
|
|
||
| # Function to format checkpoint as Rust code | ||
| format_checkpoint() { | ||
| block_json=$1 | ||
|
|
||
| height=$(echo "$block_json" | jq -r '.height') | ||
| hash=$(echo "$block_json" | jq -r '.hash') | ||
| prev_hash=$(echo "$block_json" | jq -r '.previousblockhash // "0000000000000000000000000000000000000000000000000000000000000000"') | ||
| timestamp=$(echo "$block_json" | jq -r '.time') | ||
| bits=$(echo "$block_json" | jq -r '.bits') | ||
| merkle_root=$(echo "$block_json" | jq -r '.merkleroot') | ||
| nonce=$(echo "$block_json" | jq -r '.nonce') | ||
| chain_work=$(echo "$block_json" | jq -r '.chainwork // "0x0000000000000000000000000000000000000000000000000000000000000000"') | ||
|
|
||
| # Ensure chainwork has 0x prefix | ||
| if ! echo "$chain_work" | grep -q "^0x"; then | ||
| chain_work="0x$chain_work" | ||
| fi | ||
|
|
||
| # Convert bits to hex format | ||
| if echo "$bits" | grep -q "^[0-9a-fA-F]\+$"; then | ||
| bits_hex="0x$bits" | ||
| else | ||
| bits_hex="$bits" | ||
| fi | ||
|
|
||
| # Format as Rust code | ||
| cat <<EOF | ||
| // Block $height ($hash) | ||
| create_checkpoint( | ||
| $height, | ||
| "$hash", | ||
| "$prev_hash", | ||
| $timestamp, | ||
| $bits_hex, | ||
| "$chain_work", | ||
| "$merkle_root", | ||
| $nonce, | ||
| None, // TODO: Add masternode list if available (format: Some("ML${height}__<protocol_version>")) | ||
| ), | ||
| EOF | ||
| } | ||
|
|
||
| # Main script | ||
| echo "Fetching checkpoints for $NETWORK..." >&2 | ||
| echo "" >&2 | ||
|
|
||
| # Get current height | ||
| current_height=$(get_current_height) | ||
| if [ -z "$current_height" ]; then | ||
| echo "Error: Failed to get current blockchain height" >&2 | ||
| exit 1 | ||
| fi | ||
| echo "Current blockchain height: $current_height" >&2 | ||
| echo "" >&2 | ||
|
|
||
| # If no heights specified, show usage | ||
| if [ $# -eq 0 ]; then | ||
| echo "Usage: $0 <network> <height1> [height2] [height3] ..." >&2 | ||
| echo "" >&2 | ||
| echo "Current height: $current_height" >&2 | ||
| echo "Suggested checkpoint heights:" >&2 | ||
| if [ "$NETWORK" = "mainnet" ]; then | ||
| echo " 2400000 - Block 2.4M" >&2 | ||
| echo " 2450000 - Block 2.45M" >&2 | ||
| echo " 2500000 - Block 2.5M (if available)" >&2 | ||
| else | ||
| echo " 1200000 - Block 1.2M" >&2 | ||
| echo " 1300000 - Block 1.3M" >&2 | ||
| fi | ||
| echo "" >&2 | ||
| echo "Examples:" >&2 | ||
| echo " $0 mainnet 2400000" >&2 | ||
| echo " RPC_URL=http://localhost:9998 RPC_USER=user RPC_PASS=pass $0 mainnet 2400000" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Output header | ||
| echo "// Generated checkpoints for $NETWORK" | ||
| echo "// Add these to dash-spv/src/chain/checkpoints.rs" | ||
| echo "" | ||
|
|
||
| # Fetch each requested height | ||
| for height in "$@"; do | ||
| if [ "$height" -gt "$current_height" ]; then | ||
| echo "// Skipping height $height (exceeds current height $current_height)" | ||
| continue | ||
| fi | ||
|
|
||
| echo "Fetching block at height $height..." >&2 | ||
|
|
||
| # Get block data | ||
| if [ -n "$RPC_URL" ] && [ -n "$RPC_USER" ] && [ -n "$RPC_PASS" ]; then | ||
| block_data=$(get_block_rpc "$height") | ||
| else | ||
| block_data=$(get_block_explorer "$height") | ||
| fi | ||
|
|
||
| if [ $? -ne 0 ] || [ -z "$block_data" ]; then | ||
| echo "// Error: Failed to fetch block at height $height" | ||
| continue | ||
| fi | ||
|
|
||
| # Format and output | ||
| format_checkpoint "$block_data" | ||
| done | ||
|
|
||
| echo "" | ||
| echo "// NOTE: Update masternode_list_name values if the blocks have DML (Deterministic Masternode List)" | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there something missing in the script? |
||
| echo "// Check coinbase transaction for masternode list information" | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it works; this may be sufficient, however, I would prefer our helper scripting be done in python or some other non-bash language