chore: Add integration tests for Android #1146

tustanivsky · 2025-11-12T10:02:44Z

This PR adds Android integration tests for the Sentry Unreal SDK using PowerShell's Pester framework. The tests validate end-to-end functionality by running the SentryPlayground sample application on mobile devices and verifying that events are properly captured and sent to Sentry.

It includes the necessary CI workflows and test scripts for both local (adb) and cloud (SauceLabs) execution.

The motivation for using SauceLabs Real Device Cloud to run tests in CI pipeline is the current limitation of Android emulators which do not support the rendering backends required by Unreal applications.

Changes

Added integration-test-android.yml to run Android integration tests on SauceLabs.
Updated ci.yml to include the new Android test workflow.
Added Test Scripts:
- Integration.Tests.Android.SauceLabs.ps1 for running tests on SauceLabs devices.
- Integration.Tests.Android.Adb.ps1 for running tests locally via adb.
Expanded README.md with comprehensive instructions for running Android tests locally and via SauceLabs, including required environment variables and prerequisites.
Added SAUCE_USERNAME and SAUCE_ACCESS_KEY secrets to the repo.

Related to:

Closes #964

#skip-changelog

…ing)

Fix artifact download Shell Fix Fix Fix Fix Set config Fix syntax Sauce skip run Update appium Test test test apk name Test Test Test Test Test Test Test Test Test Test Test tets Test Test Poll session status Test Test Test 2 runs Test Test Try fix redirect Test Test fix id Test app status poll Test Clean up

cursor · 2025-11-21T14:33:39Z

sample/Source/SentryPlayground/SentryPlaygroundGameInstance.cpp

-	const TCHAR* CommandLine = FCommandLine::Get();
+	FString CommandLine = FCommandLine::Get();
+
+	UE_LOG(LogSentrySample, Display, TEXT("Startin app with commandline: %s\n"), *CommandLine);


Bug: Typo in logging statement: "Startin" misspelled

The log message contains a typo: "Startin app with commandline" should be "Starting app with commandline". While this doesn't affect functionality, it's a spelling error that should be corrected before merging.

vaind · 2025-11-21T14:23:43Z

.github/workflows/integration-test-android.yml

+      - name: Install Pester
+        shell: pwsh
+        run: Install-Module -Name Pester -Force -SkipPublisherCheck


Pester: 5.7.1 is already installed in GH, have you had issues without this step?

https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md#powershell-modules

vaind · 2025-11-21T14:30:02Z

.github/workflows/integration-test-android.yml

+      - name: Run integration tests
+        id: run-integration-tests
+        shell: pwsh
+        env:
+          SENTRY_UNREAL_TEST_DSN: ${{ secrets.SENTRY_UNREAL_TEST_DSN }}
+          SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_API_TOKEN }}
+          SENTRY_UNREAL_TEST_APP_PATH: ${{ github.workspace }}/sample-build/SentryPlayground-arm64.apk
+          UNREAL_VERSION: ${{ inputs.unreal-version }}
+        run: |
+          cd integration-test
+          mkdir build
+          cmake -B build -S .
+          Invoke-Pester Integration.Tests.Android.SauceLabs.ps1 -CI


maybe a bit cleaner (move working dir to param and remove unnecessary mkdir)

Suggested change

- name: Run integration tests

id: run-integration-tests

shell: pwsh

env:

SENTRY_UNREAL_TEST_DSN: ${{ secrets.SENTRY_UNREAL_TEST_DSN }}

SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_API_TOKEN }}

SENTRY_UNREAL_TEST_APP_PATH: ${{ github.workspace }}/sample-build/SentryPlayground-arm64.apk

UNREAL_VERSION: ${{ inputs.unreal-version }}

run: |

cd integration-test

mkdir build

cmake -B build -S .

Invoke-Pester Integration.Tests.Android.SauceLabs.ps1 -CI

- name: Run integration tests

id: run-integration-tests

shell: pwsh

working-directory: integration-test

env:

SENTRY_UNREAL_TEST_DSN: ${{ secrets.SENTRY_UNREAL_TEST_DSN }}

SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_API_TOKEN }}

SENTRY_UNREAL_TEST_APP_PATH: ${{ github.workspace }}/sample-build/SentryPlayground-arm64.apk

UNREAL_VERSION: ${{ inputs.unreal-version }}

run: |

cmake -B build -S .

Invoke-Pester Integration.Tests.Android.SauceLabs.ps1 -CI

vaind · 2025-11-21T14:32:16Z

integration-test/Integration.Tests.Android.Adb.ps1

@@ -0,0 +1,418 @@
+# Integration tests for Sentry Unreal SDK on Android


I assume this would be moved to app-runner as a new provider, in which case both android integration test files would be merged to a single one, right?

vaind · 2025-11-21T14:33:54Z

.github/workflows/integration-test-android.yml

+          cd integration-test
+          mkdir build
+          cmake -B build -S .
+          Invoke-Pester Integration.Tests.Android.SauceLabs.ps1 -CI


Also, invoke-pester assumes files are named "*.Tests.ps1" so the filename should be Android.Adb.Tests.ps1 or similar. In this case it doesn't matter because you're specifying the exact path, but if you wanted to run all integration tests locally, calling invoke-pester integration-test would only pick up files ending with ".Tests.ps1"

vaind · 2025-11-21T14:41:31Z

integration-test/Integration.Tests.Android.SauceLabs.ps1

+    try {
+        if ($IsMultipart) {
+            # Use curl for multipart uploads (PowerShell's Invoke-WebRequest struggles with this)
+            $curlCmd = "curl -u `"$username`:$accessKey`" -X $Method `"$Uri`" -F `"payload=@$FilePath`" -F `"name=$(Split-Path $FilePath -Leaf)`""


This should ideally use Invoke-Webrequest to be cross-platform. I've noticed the comment but have you tried the Form param? https://stackoverflow.com/a/60323436

vaind · 2025-11-21T14:43:01Z

integration-test/Integration.Tests.Android.SauceLabs.ps1

+                $params['ContentType'] = $ContentType
+            }
+
+            $response = Invoke-RestMethod @params


Invoke-Restmethod should be avoided - it will just return plain string and swallow parsing error. see app-runner which uses invoke-webrequest and convertfrom-json

vaind · 2025-11-21T14:46:34Z

integration-test/Integration.Tests.Android.SauceLabs.ps1

+    }
+
+    Write-Host "Uploading APK to SauceLabs Storage..." -ForegroundColor Yellow
+    $uploadUri = "https://api.${Region}.saucelabs.com/v1/storage/upload"


Maybe worth checking if the file exists somehow, e.g. by using a hash-based filename. But I don't really know - if it's always rebuilt then not worth the trouble.

tustanivsky added 11 commits November 11, 2025 16:31

Update sample app to receive input args on Android (required for test…

7042674

…ing)

Add Android integration tests to CI

6e34b25

Comment crash capturing test

588aada

Fix test script

c82a55e

Fix minor issues

6b14d0a

Rename vars

a7bb5e3

Clean up

7b3ab68

Better comments

a47a7ea

Refactor

35c488b

Fix workflow

297cf36

Fix

d62218d

tustanivsky force-pushed the chore/android-tests branch from 3ba125b to d62218d Compare November 13, 2025 06:53

Simplify mechanism to pass input args to Android app

35862ce

tustanivsky force-pushed the chore/android-tests branch from a3d3766 to 35862ce Compare November 13, 2025 14:32

Clean up comments

3bc3f33

tustanivsky force-pushed the chore/android-tests branch 4 times, most recently from 41221d6 to 3bc3f33 Compare November 20, 2025 13:00

Merge branch 'main' into chore/android-tests

281af4a

tustanivsky force-pushed the chore/android-tests branch from 963fa85 to 3b14ec3 Compare November 21, 2025 06:46

tustanivsky force-pushed the chore/android-tests branch from d8ab932 to c9079bc Compare November 21, 2025 09:16

tustanivsky added 7 commits November 21, 2025 11:58

Add separate SauceLabs test script

957c826

Fix log retrieval

11d997a

Fix

f87da41

Rename script for running Android integration tests locally

4ea1b22

Update readme

d09d95e

Revert CI tweaks for faster iterations

89c6014

Clean up

b99f9a6

Fix

4c7547b

tustanivsky marked this pull request as ready for review November 21, 2025 14:31

cursor bot reviewed Nov 21, 2025

View reviewed changes

vaind reviewed Nov 21, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

chore: Add integration tests for Android #1146

chore: Add integration tests for Android #1146

Uh oh!

tustanivsky commented Nov 12, 2025 •

edited

Loading

Uh oh!

cursor bot Nov 21, 2025

Uh oh!

vaind Nov 21, 2025

Uh oh!

vaind Nov 21, 2025

Uh oh!

vaind Nov 21, 2025

Uh oh!

vaind Nov 21, 2025

Uh oh!

vaind Nov 21, 2025

Uh oh!

vaind Nov 21, 2025

Uh oh!

vaind Nov 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -0,0 +1,418 @@
		# Integration tests for Sentry Unreal SDK on Android

Uh oh!

chore: Add integration tests for Android #1146

Are you sure you want to change the base?

chore: Add integration tests for Android #1146

Uh oh!

Conversation

tustanivsky commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

cursor bot Nov 21, 2025

Choose a reason for hiding this comment

Bug: Typo in logging statement: "Startin" misspelled

Uh oh!

vaind Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

vaind Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

vaind Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

vaind Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

vaind Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

vaind Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

vaind Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tustanivsky commented Nov 12, 2025 •

edited

Loading