Skip to content

Conversation

@suzuki-shm
Copy link

Resolves #785


This pull request adds support for configuring and testing Large Language Model (LLM) diagnostics in the API and workspace diagnostic systems. The changes introduce new model and contract types to represent LLM settings, update test data generation, and refactor integration test helpers to handle these new structures.

Details

Large Language Model diagnostics support

  • Added new contract types (LargeLanguageModelSettings and LargeLanguageModelMessageSettings) to ApiDiagnostic.cs, Diagnostic.cs, and WorkspaceDiagnostic.cs for serializing LLM diagnostic options, including logs, requests, and responses.
  • Extended the main diagnostic models (ApiDiagnosticModel, DiagnosticModel) and their test generators to include an optional LargeLanguageModel property and generation logic for LLM settings.

Integration test refactoring

  • Refactored integration test helpers in ApiDiagnostic.cs and Diagnostic.cs to use a new MapToDto function, which maps the extended model to DTOs including LLM settings, replacing the previous getDto implementation.
  • Updated normalization logic in integration tests to handle LLM settings, ensuring consistent test output and comparison for diagnostics with LLM configuration.

Test data generation

  • Updated generators for diagnostic model updates and DTO overrides to include LLM settings, enabling richer test coverage for diagnostics involving LLMs.

Results

Here is the screenshot of extracted properties from my APIM resource.
image

@suzuki-shm
Copy link
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Extracting the new largeLanguageModel diagnostic setting not extracting

1 participant