On-Demand Nightly PyPI Publishing (#216)

Summary:
This PR implements on-demand nightly publishing for the tritonparse package. Instead of publishing a new nightly version every day regardless of changes, the workflow now checks if there are new commits since the last nightly release and skips publishing if there are none.

## Changes

### New File: `.github/scripts/check_new_commits.sh`

A standalone script that:
1. Queries PyPI API to get the latest nightly version (e.g., `0.3.2.dev20251208071617`)
2. Extracts the timestamp from the version string
3. Uses `git log --since` to check for new commits after that timestamp
4. Outputs `should_publish=true/false` to `GITHUB_OUTPUT`

Key features:
- **Network retry mechanism**: 3 retries with 5s delay between attempts
- **Fail-open design**: If PyPI API is unreachable, proceeds with publish (never blocks releases due to network issues)
- **Edge case handling**: Handles first nightly release, unparseable versions, etc.

### Modified: `.github/workflows/nightly-pypi.yml`

- Added a new step to run `check_new_commits.sh` before computing the nightly version
- All subsequent steps (build, check, publish) are conditioned on `should_publish != 'false'`
- Tag-based releases bypass the check entirely

## Workflow Logic

```
Trigger (schedule/workflow_dispatch/tag)
         │
         ▼
    Is tag push? ───Yes──► Build and Publish directly
         │
         No
         ▼
    Run check_new_commits.sh
         │
         ▼
    Has new commits? ───No──► Skip (exit early)
         │
         Yes
         ▼
    Compute version → Build → Publish
```

## Benefits

- **Reduces noise**: No more daily releases with identical content
- **Saves resources**: Skips unnecessary CI builds
- **Cleaner PyPI history**: Only releases with actual changes
- **Robust**: Network failures don't block legitimate releases

## Testing

Run locally with:
```bash
cd tritonparse_oss
GITHUB_OUTPUT=/tmp/gh_output.txt bash .github/scripts/check_new_commits.sh
cat /tmp/gh_output.txt
```

Pull Request resolved: #216

Reviewed By: xuzhao9

Differential Revision: D88696841

Pulled By: FindHao

fbshipit-source-id: 04e2ff74fc7cfa1fda2769e5a0fdaa975988c964

Author: FindHao

.github/scripts/check_new_commits.sh

@@ -0,0 +1,115 @@
+#!/bin/bash
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# Check if there are new commits since the last nightly PyPI release.
+# This script is used by nightly-pypi.yml to implement on-demand nightly publishing.
+#
+# Exit code:
+#   Always exits with 0 (success). The publish decision is communicated via GITHUB_OUTPUT.
+#
+# Output:
+#   Sets GITHUB_OUTPUT variable: should_publish=true/false
+#     - true: New commits found or check skipped due to errors (fail-open behavior)
+#     - false: No new commits since last nightly release
+
+set -o pipefail
+# Note: set -e is intentionally not used to allow explicit error handling.
+# The script implements fail-open behavior where errors should not block publishing.
+
+# Configuration
+MAX_RETRIES=3
+RETRY_DELAY=5
+CURL_TIMEOUT=10
+PACKAGE_NAME="${PACKAGE_NAME:-tritonparse}"
+
+# Dependencies: This script requires 'jq' and 'curl' to be installed.
+# These are pre-installed on ubuntu-latest GitHub Actions runners.
+
+# Function: Fetch latest nightly version from PyPI with retry
+fetch_latest_nightly() {
+    local retries=0
+
+    while [ $retries -lt $MAX_RETRIES ]; do
+        local response
+        response=$(curl -s --max-time $CURL_TIMEOUT \
+            "https://pypi.org/pypi/${PACKAGE_NAME}/json" 2>/dev/null)
+        local curl_exit=$?
+
+        if [ $curl_exit -eq 0 ] && [ -n "$response" ]; then
+            # Try to parse and extract latest dev version
+            local latest
+            latest=$(echo "$response" | \
+                jq -r '.releases | keys[] | select(contains(".dev"))' 2>/dev/null | \
+                sort -V | tail -1)
+            local jq_exit=$?
+
+            if [ $jq_exit -eq 0 ] && [ -n "$latest" ]; then
+                echo "$latest"
+                return 0
+            fi
+        fi
+
+        retries=$((retries + 1))
+        echo "::warning::PyPI API request failed (attempt $retries/$MAX_RETRIES)"
+
+        if [ $retries -lt $MAX_RETRIES ]; then
+            echo "Retrying in ${RETRY_DELAY}s..."
+            sleep $RETRY_DELAY
+        fi
+    done
+
+    echo "::warning::All $MAX_RETRIES attempts failed"
+    return 1
+}
+
+main() {
+    echo "Checking for new commits since last nightly release..."
+
+    # Step 1: Fetch latest nightly version (with retry)
+    LATEST_NIGHTLY=$(fetch_latest_nightly)
+    FETCH_STATUS=$?
+
+    # Step 2: Network request failed -> skip check, proceed with publish
+    if [ $FETCH_STATUS -ne 0 ]; then
+        echo "::warning::Failed to fetch PyPI data after $MAX_RETRIES attempts"
+        echo "::warning::Skipping commit check, proceeding with publish"
+        echo "should_publish=true" >> "$GITHUB_OUTPUT"
+        exit 0
+    fi
+
+    # Step 3: No nightly version exists -> first nightly release
+    if [ -z "$LATEST_NIGHTLY" ]; then
+        echo "No existing nightly version found, proceeding with publish"
+        echo "should_publish=true" >> "$GITHUB_OUTPUT"
+        exit 0
+    fi
+
+    echo "Latest nightly on PyPI: $LATEST_NIGHTLY"
+
+    # Step 4: Extract timestamp from version (format: X.Y.Z.devYYYYMMDDHHMMSS)
+    TIMESTAMP=$(echo "$LATEST_NIGHTLY" | sed -n 's/.*\.dev\([0-9]\{14\}\).*/\1/p')
+
+    if [ -z "$TIMESTAMP" ]; then
+        echo "::warning::Cannot parse timestamp from version, proceeding with publish"
+        echo "should_publish=true" >> "$GITHUB_OUTPUT"
+        exit 0
+    fi
+
+    # Step 5: Convert to date format for git log
+    SINCE_DATE="${TIMESTAMP:0:4}-${TIMESTAMP:4:2}-${TIMESTAMP:6:2} ${TIMESTAMP:8:2}:${TIMESTAMP:10:2}:${TIMESTAMP:12:2} UTC"
+    echo "Last nightly published at: $SINCE_DATE"
+
+    # Step 6: Check for new commits since last nightly
+    COMMITS_SINCE=$(git log --since="$SINCE_DATE" --oneline | wc -l)
+
+    if [ "$COMMITS_SINCE" -eq 0 ]; then
+        echo "No new commits since last nightly, skipping publish"
+        echo "should_publish=false" >> "$GITHUB_OUTPUT"
+        exit 0
+    else
+        echo "Found $COMMITS_SINCE new commit(s) since last nightly"
+        echo "should_publish=true" >> "$GITHUB_OUTPUT"
+        exit 0
+    fi
+}
+
+main

.github/workflows/nightly-pypi.yml

@@ -23,9 +23,15 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: "3.11"
+
+      - name: Check for new commits since last nightly
+        id: check
+        if: github.ref_type != 'tag'
+        run: bash .github/scripts/check_new_commits.sh
+
       - name: Compute nightly version from latest tag (next patch + timestamp)
         id: ver
-        if: github.ref_type != 'tag'
+        if: github.ref_type != 'tag' && steps.check.outputs.should_publish != 'false'
         run: |
           # Get latest tag; allow 'v' prefix; fail if none
           if ! TAG=$(git describe --tags --abbrev=0 2>/dev/null); then
@@ -46,6 +52,7 @@ jobs:
           echo "Computed nightly version: ${NEXT}.dev${DATE}"
 
       - name: Build sdist/wheel
+        if: github.ref_type == 'tag' || steps.check.outputs.should_publish != 'false'
         run: |
           python -m pip install --upgrade pip
           pip install build setuptools-scm
@@ -55,12 +62,13 @@ jobs:
           python -m build
 
       - name: Check metadata
+        if: github.ref_type == 'tag' || steps.check.outputs.should_publish != 'false'
         run: |
           pip install twine
           twine check dist/*
 
       - name: Publish to PyPI (API token)
-        if: github.event_name == 'schedule' || github.ref_type == 'tag'
+        if: (github.event_name == 'schedule' || github.ref_type == 'tag') && steps.check.outputs.should_publish != 'false'
         uses: pypa/gh-action-pypi-publish@release/v1
         with:
           user: __token__