Phase 2: Test Coverage - Fix All Failing Tests

[floriangrousset]

Phase 2: Test Coverage Enhancement

This PR implements comprehensive improvements to test quality and coverage as part of the DevOps foundation initiative.

Overview

3 commits addressing test reliability, code quality warnings, and coverage gaps:

1. ✅ Phase 2a: Fix failing tests (12 tests fixed)
2. ✅ Phase 2b: Reduce pytest warnings (56 → 5 warnings)
3. ✅ Phase 2c: Increase test coverage (37.17% → 39.79%)

Phase 2a: Test Fixes

Fixed 12 failing tests broken by Phase 3 security API changes:

Changes

• Updated tests for profile-based secure credential API
• Changed from explicit credentials to profile="default" pattern
• Added proper mocking for ConfigLoader.load() and get_profile_info()
• Fixed CLI test output checking (stdout vs output)
• Added getpass.getpass mocking for interactive tests

Files Modified

• tests/test_cli/test_setup.py: 4 tests fixed
• tests/test_domains/test_configuration.py: 6 tests fixed
• tests/test_integration.py: 1 test fixed
• tests/test_integration/test_secure_config.py: 1 test fixed

Result: 355 tests passing (was 343 passing, 12 failing)

Phase 2b: Warning Reduction

Reduced pytest warnings from 56 to 5 by addressing deprecations and incorrect markers:

Fixes Applied

1. Pydantic V2 Migration (src/opnsense_mcp/core/models.py)

   • Changed from class-based Config to model_config = ConfigDict
   • Eliminates deprecation warning for all tests using OPNsenseConfig

2. Pytest Asyncio Markers (6 test files)

   • Removed class-level @pytest.mark.asyncio from classes with synchronous methods
   • Added method-level decorators to individual async test methods
   • Fixed 51 warnings about non-async functions marked with asyncio

Files Modified

• src/opnsense_mcp/core/models.py
• tests/test_core/test_client_basic.py
• tests/test_core/test_config_loader.py
• tests/test_core/test_exceptions.py
• tests/test_core/test_models.py
• tests/test_core/test_connection.py
• tests/test_core/test_state.py

Result: 56 → 5 warnings (remaining are RuntimeWarnings in network_services tests)

Phase 2c: Coverage Enhancement

Improved overall coverage by 2.62 percentage points with 43 new comprehensive tests:

Coverage Improvements

Module	Before	After	Change	Tests Added
shared/error_sanitizer.py	14.63%	99.19%	+84.56pp	30
shared/error_handlers.py	17.86%	100%	+82.14pp	- (bonus)
cli/delete.py	9.52%	100%	+90.48pp	13
Overall	37.17%	39.79%	+2.62pp	43

New Test Files

tests/test_shared/test_error_sanitizer.py (30 tests)

Comprehensive testing of error message sanitization:

• ✅ All error type sanitization (AuthenticationError, AuthorizationError, etc.)
• ✅ Sensitive data redaction (passwords, API keys, tokens)
• ✅ Context sanitization (nested dicts, recursive processing)
• ✅ Case-insensitive pattern matching
• ✅ log_error_safely() function behavior
• ✅ httpx error handling with response details

tests/test_cli/test_delete.py (13 tests)

Complete coverage of profile deletion workflow:

• ✅ Profile not found error handling
• ✅ User confirmation flow (accept/cancel)
• ✅ Force flag (--force, -f) behavior
• ✅ Empty profile list handling
• ✅ Remaining profiles display
• ✅ ConfigurationError and unexpected error handling
• ✅ Profile info display and emoji output

Bug Fix

src/opnsense_mcp/cli/delete.py: Fixed typer.Exit() handling

• Added specific except typer.Exit block before generic Exception handler
• Prevents Exit(0) from being caught and converted to Exit(1)
• Preserves intended exit codes for user cancellation

Test Results

✅ 398 tests passing (was 355, +43 new tests)
⚠️  5 warnings remaining (RuntimeWarnings in network_services)
📊 Coverage: 37.17% → 39.79% (+2.62pp)

Impact

Quality Improvements

• ✅ All tests now passing and compatible with secure credential API
• ✅ Cleaner test output (95% reduction in warnings)
• ✅ Better test reliability and maintainability

Coverage Gains

• ✅ Critical security module (error_sanitizer) now 99% covered
• ✅ Error handling paths fully tested
• ✅ CLI delete command fully tested with edge cases

Future Work

To reach 80%+ overall coverage would require testing large domain modules:

• domains/certificates.py (425 lines, 21% coverage)
• domains/users.py (508 lines, 12% coverage)
• domains/dns_dhcp.py (430 lines, 14% coverage)
• domains/traffic_shaping.py (461 lines, 14% coverage)

These could be addressed in a follow-up phase with focused domain testing.

Checklist

• All tests passing (398/398)
• No regressions introduced
• Coverage increased (+2.62pp)
• Warnings reduced (56 → 5)
• Bug fix included (typer.Exit handling)
• Test documentation complete
• Ready for merge to develop

🤖 Generated with Claude Code