Skip to content

Create Pipeline For Running Agents with Device Capabilities #33147

@PureWeen

Description

@PureWeen

Gathering Data From the Agent Run

Overview

This specification defines how Azure DevOps (AzDO) pipelines should collect, organize, and publish test artifacts generated by .NET MAUI UI test runs. The goal is to provide comprehensive debugging information when tests fail and enable effective CI/CD integration.

  1. We should be able to send a prompt to the agent
  2. We should be able to specify a specific agent
  3. We should gather the logs (currently logs are all stored in a CustomAgentLogsTmp) directory
  4. post any screenshots taking from the running app to the github issue

Background: Current Local Testing Setup

The following is the directory that all logs are output to.
We could change this to artifacts, but the idea here would be to attach this to the pipeline run as artifacts and we could also look at include images with the github issue

Directory Structure

When tests run locally via the custom scripts, artifacts are organized in:

CustomAgentLogsTmp/
├── UITests/                    # NUnit UI tests (via BuildAndRunHostApp.ps1)
│   ├── test-output.log         # dotnet test console output
│   ├── android-device.log      # Android logcat for test app
│   └── ios-device.log          # iOS device/simulator logs
│
└── Sandbox/                    # Appium sandbox tests (via BuildAndRunSandbox.ps1)
    ├── RunWithAppiumTest.cs    # Generated test script
    ├── build-run-output.log    # Build and test execution log
    ├── appium.log              # Appium server logs
    ├── android-device.log      # Android logcat
    ├── ios-device.log          # iOS device logs
    └── *.png                   # Screenshots from test execution

Metadata

Metadata

Assignees

No one assigned

    Labels

    s/triagedIssue has been reviewed

    Type

    No type

    Projects

    Status

    Todo

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions