Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
ffa1d88
add deploy-sandbox.yml
jtherrmann Dec 2, 2025
49a3eab
rename hyp3-sandbox to hyp3-edc-sandbox
jtherrmann Dec 2, 2025
dff2aa6
deploy-edc-sandbox perms
jtherrmann Dec 2, 2025
3d29a01
Update OPERA_DIST_S1.yml
cmarshak Dec 11, 2025
4bd5059
Update CHANGELOG.md
cmarshak Dec 11, 2025
3e31034
Update CHANGELOG.md
cmarshak Dec 11, 2025
674d9c1
Update OPERA_DIST_S1.yml
cmarshak Dec 11, 2025
2a01337
Update CHANGELOG.md
cmarshak Dec 11, 2025
0c0ef2c
Update CHANGELOG.md
jhkennedy Dec 11, 2025
feeeb7a
Merge pull request #2991 from ACCESS-Cloud-Based-InSAR/Expose-context…
jhkennedy Dec 11, 2025
420693c
Fix OPERA_DIST_S1
jhkennedy Dec 11, 2025
695c818
Merge pull request #2992 from ASFHyP3/jhkennedy-patch-1
jhkennedy Dec 11, 2025
0c2ff1f
distribution_url
jtherrmann Dec 16, 2025
0599eef
delete jth-sandbox
jtherrmann Dec 16, 2025
4f6e5d7
edc-sandbox cost profile
jtherrmann Dec 16, 2025
131669d
start updating deployment docs
jtherrmann Dec 19, 2025
94f3f8f
update ssl cert docs for edc and jpl
jtherrmann Dec 19, 2025
d715f9f
clarify region
jtherrmann Dec 19, 2025
e5ad0ad
update gh secrets in deploy docs
jtherrmann Dec 19, 2025
6d26374
wording
jtherrmann Dec 19, 2025
a801e85
finish updating ssl cert docs
jtherrmann Dec 19, 2025
ac5adaf
wording
jtherrmann Dec 19, 2025
88f1fd6
revise
jtherrmann Dec 19, 2025
c2c06b6
Merge branch 'develop' into hyp3-edc-sandbox
jtherrmann Dec 19, 2025
5608e7d
Merge pull request #2985 from ASFHyP3/hyp3-edc-sandbox
jtherrmann Dec 19, 2025
2c6638f
fix: Run the checks workflow on a push to any PR to main or develop
kalebvonburris Jan 12, 2026
6c2dde5
adding hyp3-slimsar-test to deploy yml to deploy test hyp3 for slimsa…
ZachHoppinen Jan 12, 2026
f0a022c
add changelog entry for new deployment
ZachHoppinen Jan 12, 2026
d975076
combine changelog added entries
ZachHoppinen Jan 12, 2026
7699876
Merge pull request #2998 from ASFHyP3/hyp3-slimsar-test
ZachHoppinen Jan 12, 2026
c85bb5c
Merge branch 'develop' into kb/fix/1476-checks-from-forks-dont-run
kalebvonburris Jan 21, 2026
ac22307
Keep push event trigger for tests workflow
jhkennedy Jan 22, 2026
33fde8d
Update static analysis workflow to include pull requests
jhkennedy Jan 22, 2026
f5239b4
Merge pull request #2997 from ASFHyP3/kb/fix/1476-checks-from-forks-d…
jhkennedy Jan 22, 2026
bc67b2d
Bump the github-actions-deps group with 3 updates
dependabot[bot] Jan 22, 2026
79a526d
Merge pull request #2982 from ASFHyP3/dependabot/github_actions/githu…
jhkennedy Jan 22, 2026
6072b78
Update ARIA_AUTORIFT.yml to support custom chip-size and search-range
cmspeed Jan 23, 2026
17909c0
Update CHANGELOG.md
cmspeed Jan 23, 2026
3f0b2a9
fix missing newline at end of file
jhkennedy Jan 23, 2026
8dbf7ea
Merge pull request #3001 from cmspeed/update-autorift-yml
jhkennedy Jan 23, 2026
2fe2883
Add condition to run Snyk action only for non-forked PRs
jhkennedy Jan 23, 2026
904b8dc
Apply suggestion from @jhkennedy
jhkennedy Jan 23, 2026
f8f368f
Remove premature version entry in CHANGELOG
jhkennedy Jan 23, 2026
9248eb8
Merge pull request #3002 from ASFHyP3/jhkennedy-patch-1
jhkennedy Jan 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/deploy-custom-prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -310,7 +310,7 @@ jobs:
url: https://${{ matrix.domain }}

steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
with:
fetch-depth: 0

Expand Down
19 changes: 18 additions & 1 deletion .github/workflows/deploy-custom-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,13 +127,30 @@ jobs:
required_surplus: 0
security_environment: ASF
ami_id: /aws/service/ecs/optimized-ami/amazon-linux-2023/recommended/image_id

- environment: hyp3-slimsar-test
domain: hyp3-slimsar-test.asf.alaska.edu
template_bucket: cf-templates-1ce4dbsaugtx1-us-west-2
image_tag: test
product_lifetime_in_days: 14
default_credits_per_user: 0
default_application_status: APPROVED
cost_profile: DEFAULT
job_files: >-
job_spec/INSAR_ISCE_BURST.yml
instance_types: r6id.xlarge,r6id.2xlarge,r6id.4xlarge,r6id.8xlarge,r6idn.xlarge,r6idn.2xlarge,r6idn.4xlarge,r6idn.8xlarge
default_max_vcpus: 640
expanded_max_vcpus: 640
required_surplus: 0
security_environment: ASF
ami_id: /aws/service/ecs/optimized-ami/amazon-linux-2023/recommended/image_id

environment:
name: ${{ matrix.environment }}
url: https://${{ matrix.domain }}

steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
with:
fetch-depth: 0

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/deploy-daac-prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
name: ${{ matrix.environment }}

steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
with:
fetch-depth: 0

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/deploy-daac-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
name: ${{ matrix.environment }}

steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
with:
fetch-depth: 0

Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
name: Deploy jth sandbox to AWS
name: Deploy HyP3 EDC Sandbox to AWS

permissions:
contents: read

on:
push:
branches:
- hyp3-jth-sandbox
- hyp3-edc-sandbox

concurrency: ${{ github.workflow }}-${{ github.ref }}

Expand All @@ -14,13 +17,13 @@ jobs:
fail-fast: false
matrix:
include:
- environment: hyp3-jth-sandbox
- environment: hyp3-edc-sandbox
template_bucket: cf-templates-bywc0durdnqy-us-west-2
image_tag: test
product_lifetime_in_days: 14
default_credits_per_user: 0
default_application_status: APPROVED
cost_profile: DEFAULT
cost_profile: EDC
opera_rtc_s1_end_date: Default
job_files: >-
job_spec/AUTORIFT.yml
Expand All @@ -36,24 +39,24 @@ jobs:
required_surplus: 0
security_environment: EDC
ami_id: /ngap/amis/image_id_ecs_al2023_x86
distribution_url: ''
distribution_url: 'https://d3bvvghf83wjqc.cloudfront.net'

environment:
name: ${{ matrix.environment }}

steps:
- uses: actions/checkout@v4.2.2
- uses: actions/checkout@v6.0.1
with:
fetch-depth: 0

- uses: aws-actions/configure-aws-credentials@v4
- uses: aws-actions/configure-aws-credentials@v5
with:
aws-access-key-id: ${{ secrets.V2_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.V2_AWS_SECRET_ACCESS_KEY }}
aws-session-token: ${{ secrets.V2_AWS_SESSION_TOKEN }}
aws-region: ${{ secrets.AWS_REGION }}

- uses: actions/setup-python@v5
- uses: actions/setup-python@v6
with:
python-version: 3.13

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/deploy-plus-prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
url: https://${{ matrix.domain }}

steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
with:
fetch-depth: 0

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/deploy-plus-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
url: https://${{ matrix.domain }}

steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
with:
fetch-depth: 0

Expand Down
17 changes: 12 additions & 5 deletions .github/workflows/static-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,13 @@ name: Static code analysis
permissions:
contents: read

on: push
on:
push:
pull_request:
branches:
- main
- develop


env:
SETUPTOOLS_SCM_PRETEND_VERSION_FOR_HYP3: v0.0.0
Expand All @@ -23,7 +29,7 @@ jobs:
matrix:
security_environment: [ASF, EDC, JPL, JPL-public]
steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
- uses: actions/setup-python@v6
with:
python-version: 3.13
Expand All @@ -37,7 +43,7 @@ jobs:
openapi-spec-validator:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
- uses: actions/setup-python@v6
with:
python-version: 3.13
Expand All @@ -50,7 +56,7 @@ jobs:
statelint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
- uses: ruby/setup-ruby@v1
with:
ruby-version: 2.7
Expand All @@ -69,8 +75,9 @@ jobs:

snyk:
runs-on: ubuntu-latest
if: ${{ ! github.event.pull_request.head.repo.fork }}
steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1
- uses: snyk/actions/setup@v1.0.0
- uses: actions/setup-python@v6
with:
Expand Down
9 changes: 7 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
name: Run tests

on: push
on:
push:
pull_request:
branches:
- main
- develop

env:
SETUPTOOLS_SCM_PRETEND_VERSION_FOR_HYP3: v0.0.0
Expand All @@ -10,7 +15,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6.0.0
- uses: actions/checkout@v6.0.1

- uses: actions/setup-python@v6
with:
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [10.12.2]

### Added
- Added optional `chip_size` and `search_range` parameters to the `ARIA_AUTORIFT.yml` job specification to enable user-defined `chip-size` and `search-range`.
- Added a `model_context_length` parameter to the `OPERA_DIST_S1` job specification.
- Added the custom hyp3-slimsar-test deployment

### Changed
- Increased the maximum `stride_for_norm_param_estimation` to 32 from 16 for the `OPERA_DIST_S1` job specification to handle models with a larger input size.

## [10.12.1]

### Fixed
Expand Down
79 changes: 18 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,10 @@ also referred to as "security environments" throughout our code and docs
- JPL
- JPL-public

For EDC, you will also need to refer to our
[Deploy HyP3 to Earthdata Cloud](https://github.com/ASFHyP3/.github-private/blob/main/docs/Deploy-HyP3-to-Earthdata-Cloud.md)
internal docs article (only accessible to members of ASF).

> [!IMPORTANT]
> JPL deployments _must_ start with the JPL security environment, but can be migrated to `JPL-public`
> after they are fully deployed and approved to have a public bucket.
Expand All @@ -67,19 +71,6 @@ For JPL, these deployment docs assume that:
- the JPL account was set up in the "default" manner by the JPL cloud team
- the developer deploying the account is able to log in with the `power_user` role

For a new EDC deployment, you need the following items (not necessarily a comprehensive list):
- SSL certificate in AWS Certificate Manager for custom CloudFront domain name
- ID of the CloudFront Origin Access Identity used to access data in S3

EDC UAT/prod deployment steps are not fully documented here.
When deploying HyP3 to a new EDC account for the first time, you should also refer to the
[SOP for deploying HyP3 to EDC](https://asfdaac.atlassian.net/wiki/spaces/ST/pages/2290319361/SOP-ASF-DAAC-EDC-011).
You should then be able to deploy additional copies of HyP3 to an EDC Sandbox account
by following this README alone.

After deploying HyP3 to an EDC Sandbox account, you'll need to follow our documentation on
[Accessing Private API Gateways in Earthdata Cloud](https://github.com/ASFHyP3/.github-private/blob/main/docs/Accessing-Private-API-Gateways-in-Earthdata-Cloud.md).

> [!TIP]
> You can expand and collapse details specific to a security environment as you go through this README.
> Make sure you're looking at the details for the security environment you're deploying into!
Expand Down Expand Up @@ -240,52 +231,21 @@ Go to AWS console -> Secrets Manager, then:
1. Click the orange "Next" button (we won't configure rotation)
1. Click the orange "Store" button to save the Secret

#### Upload SSL cert

> [!WARNING]
> This step must be done by an ASF employee.

To allow HTTPS connections, HyP3 needs an SSL certificate that is valid for its deployment domain name (URL).

If HyP3 is being deployed to an ASF-managed AWS account, we can use the master certificate that covers all
`*.asf.alaska.edu` domains. Otherwise, we'll need a deployment specific certificate.

*Important: Skip this step for EDC Sandbox deployments.*

<details>
<summary>ASF-managed AWS account: Upload the ASF master SSL certificate</summary>
<br />

Upload the `*.asf.alaska.edu` SSL certificate to AWS Certificate Manager (ACM):
#### Request SSL cert

1. AWS console -> Certificate Manager (ACM) -> import certificate
1. Open https://gitlab.asf.alaska.edu/operations/puppet/-/tree/production/modules/certificates/files
1. The contents of the `asf.alaska.edu.cer` file go in "Certificate body"
1. The contents of the `asf.alaska.edu.key` file go in "Certificate private key"
1. The contents of the `intermediates.pem` file go in "Certificate chain"
</details>
*Note: For EDC accounts, you should create the cert in the `us-east-1` region
for use with the CloudFront distribution that you will create later,
even if you're deploying HyP3 to `us-west-2`.*

<details>
<summary>Externally-managed AWS account (e.g., JPL, EDC, CloudBank): Request and upload deployment specific SSL certificate</summary>
<br />
To allow HTTPS connections, HyP3 needs an SSL certificate that is valid for its deployment domain name (URL):

Submit a Platform request in ASF JIRA for a new certificate, including the domain name
(e.g. `hyp3-foobar.asf.alaska.edu`).
AWS console -> AWS Certificate Manager -> Request a public certificate:
- The domain name should be something like `hyp3-foobar.asf.alaska.edu`
- Choose "DNS validation"

Once you receive the certificate's private key and links to download the certificate in various formats,
download these files:
1. Certificate Body (the "as Certificate Only, PEM encoded" link from the email)
2. Certificate Private Key (from the Platform team; typically in MatterMost)
3. Certificate Chain (the "as Root/Intermediate(s) only, PEM encoded" link from the email)

and then upload them to AWS Certificate Manager (ACM):

1. AWS console -> Certificate Manager (ACM) -> import certificate
1. The contents of (1) above goes in Certificate body
1. The contents of (2) above goes in Certificate private key
1. The contents of (3) above goes in Certificate chain

</details>
Then add the validation record to
https://gitlab.asf.alaska.edu/operations/puppet/-/edit/production/modules/legacy_dns/files/asf.alaska.edu.db
(see previous records for examples).

### Create the GitHub environment

Expand All @@ -301,8 +261,7 @@ and then upload them to AWS Certificate Manager (ACM):
(This is typically `main` for prod deployments, `develop` for test deployments, or a feature branch name for sandbox deployments.)
4. Add the following environment secrets:
- `AWS_REGION` - e.g. `us-west-2`
- `BUCKET_READ_PRINCIPALS` (EDC only) - List of AWS IAM principals granted read access to data in S3 for Earthdata Cloud deployments. For EDC Sandbox deployments, if you don't know what to put here, you can simply set it to `arn:aws:iam::<edc-sandbox-account-id>:root`, where `<edc-sandbox-account-id>` is the AWS account ID for the EDC Sandbox account.
- `CERTIFICATE_ARN` (ASF and JPL only) - ARN of the AWS Certificate Manager certificate that you imported manually (aws console -> certificate manager -> list certificates, e.g. `arn:aws:acm:us-west-2:xxxxxxxxxxxx:certificate/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`)
- `CERTIFICATE_ARN` (ASF and JPL only) - ARN of the AWS Certificate Manager certificate that you created manually
- `CLOUDFORMATION_ROLE_ARN` (ASF only) - part of the `hyp3-ci` stack that you deployed, e.g. `arn:aws:iam::xxxxxxxxxxxx:role/hyp3-ci-CloudformationDeploymentRole-XXXXXXXXXXXXX`
- `SECRET_ARN` - ARN for the AWS Secrets Manager Secret that you created manually
- `V2_AWS_ACCESS_KEY_ID` - AWS access key ID:
Expand All @@ -317,7 +276,7 @@ and then upload them to AWS Certificate Manager (ACM):

You will need to add the deployment to the matrix in an existing GitHub Actions `deploy-*.yml` workflow or create
a new one for the deployment. If you need to create a new one, we recommend copying one of the
`deploy-*-sandbox.yml` workflows, and then updating all of the fields (`environment`, `domain`, `template_bucket`, etc.)
existing workflows, and then updating all of the fields
as appropriate for your deployment. Also make sure to update the top-level `name` of the workflow and the name
of the branch to deploy from. (This is typically `main` for prod deployments, `develop` for test deployments, or a feature branch name for sandbox deployments.)

Expand All @@ -336,12 +295,10 @@ Once HyP3 is deployed, there are a few follow on tasks you may need to do for a
> [!WARNING]
> This step must be done by an ASF employee.

*Important: Skip this step for EDC Sandbox deployments.*

Open a PR adding a line to https://gitlab.asf.alaska.edu/operations/puppet/-/blob/production/modules/legacy_dns/files/asf.alaska.edu.db
for the new custom domain name (AWS console -> api gateway -> custom domain names -> "API Gateway domain name").

Ask the Platform team in the `~development-support` channel in Mattermost to review/merge the PR.
Ask someone from ASF support to review/merge the PR.

Changes should take effect within 15-60 minutes after merging.
Confirm that a Swagger UI is available at your chosen API URL.
Expand Down
18 changes: 18 additions & 0 deletions job_spec/ARIA_AUTORIFT.yml
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,20 @@ AUTORIFT:
description: Shapefile for determining the correct search parameters by geographic location. Path to shapefile must be understood by GDAL.
type: string
default: '/vsicurl/https://its-live-data.s3.amazonaws.com/autorift_parameters/v001/autorift_solidearth_0120m.shp'
chip_size:
api_schema:
description: Custom chip size in pixels for AutoRIFT (overrides parameter_file if provided)
type: integer
nullable: true
default: null
example: 32
search_range:
api_schema:
description: Custom search range in pixels for AutoRIFT (overrides parameter_file if provided)
type: integer
nullable: true
default: null
example: 64
cost_profiles:
DEFAULT:
cost: 1.0
Expand All @@ -129,6 +143,10 @@ AUTORIFT:
- Ref::reference
- --secondary
- Ref::secondary
- --chip-size
- Ref::chip_size
- --search-range
- Ref::search_range
timeout: 10800
compute_environment: AriaAutorift
vcpu: 1
Expand Down
Loading