WIP: AppSec integration implementation #63
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Core AppSec functionality implemented: - Added AppSecClient struct for HTTP communication with CrowdSec AppSec engine - Implemented host-specific AppSec configuration with global fallbacks - Added AppSec validation in SPOA request handling - Created AppSec request/response handling with proper HTTP headers Configuration changes: - Added appsec_url to root config (uncommented in main config) - Updated HAProxy SPOE config to include src-ip for AppSec validation - Added AppSec acquisition config for CrowdSec container API and worker integration: - Added ValHostAppSec method to worker client - Implemented AppSec validation in API handlers - Added AppSecEnabled field to host responses - Context propagation for AppSec validation calls Testing infrastructure: - Created docker-compose.test.yaml for AppSec testing - Added test configuration files for host networking setup - Created run-appsec-test.sh script for test orchestration - Added unit tests for AppSec functionality Architecture notes: - Each host can override global AppSec settings - SPOA always attempts AppSec validation, API handles enabled/disabled logic - AppSec client uses proper CrowdSec AppSec protocol headers - Host networking approach for testing with cscli hubtest TODO: Complete testing setup and validate end-to-end AppSec flow
This commit adds comprehensive AppSec (Web Application Firewall) support to the HAProxy SPOA bouncer, enabling real-time threat detection and remediation. ## Core Features ### AppSec Client Implementation - Added AppSecClient struct for HTTP communication with CrowdSec AppSec engine - Implemented host-specific AppSec configuration with global fallbacks - Added proper HTTP header handling following CrowdSec AppSec protocol - Context-aware request validation with timeout handling ### Configuration Support - Added appsec_url to root configuration for global AppSec settings - Host-specific AppSec URL and API key override support - AppSec always_send configuration for bypassing IP-based decisions - Proper configuration validation and error handling ### SPOA Integration - Integrated AppSec validation into HTTP request processing pipeline - Added proper header extraction from HAProxy SPOE messages - Implemented HTTP version handling (1.0, 1.1, 2.0, 3.0) - Added source IP extraction for AppSec validation ### API and Worker Updates - Added ValHostAppSec method to worker client - Implemented AppSec validation in API handlers with context propagation - Added AppSecEnabled field to host responses for status reporting - Proper error handling and fallback to allow on AppSec failures ## Configuration Changes ### HAProxy Configuration - Added http-buffer-request option for proper request body buffering - Updated crowdsec.cfg to include src-ip for AppSec validation - Added version extraction from HAProxy SPOE messages ### CrowdSec Integration - Added AppSec acquisition configuration (appsec.yaml) - Updated docker-compose.yaml with AppSec collections - Proper AppSec engine configuration and port exposure ### Testing Infrastructure - Added comprehensive Vagrantfile for complete testing environment - Includes HAProxy 3.1, CrowdSec, SPOA bouncer, and Nginx backend - Automated service startup with proper dependency ordering - Support for AppSec testing with cscli hubtest and Nuclei ## Security Features ### Header Security - Prevents header injection attacks by copying original headers first - Trusted CrowdSec headers override any malicious x-crowdsec-appsec-* headers - Proper API key validation and authentication ### Error Handling - Graceful degradation on AppSec engine failures - Fallback to allow remediation to prevent blocking legitimate traffic - Comprehensive logging for debugging and monitoring ## Testing and Documentation - Added unit tests for AppSec functionality - Updated worker client tests with new signature - Added Vagrant-based testing environment documentation - Proper GOB type registration for http.Header serialization ## Breaking Changes - Updated AppSec.Init signature to accept global configuration - Modified ValHostAppSec method signature to include version parameter - Removed unnecessary referer and contentType parameters This implementation provides a robust, production-ready AppSec integration that seamlessly works with existing CrowdSec infrastructure while maintaining backward compatibility and security best practices.
Removed docker-compose.test.yaml and run-appsec-test.sh as these were replaced by the more comprehensive Vagrant-based testing environment.
- Fixed errcheck issues by properly handling error returns in AppSec tests - Replaced hardcoded HTTP status codes with http package constants - Used http.NoBody instead of nil for GET requests - Used http.MethodPost and http.MethodGet constants - Refactored handleHTTPRequest function to reduce cyclomatic complexity - Extracted helper methods for better code organization and maintainability All CI linting errors have been resolved while maintaining functionality.
Fixed TestAppSec_ValidateRequest_POST by using io.ReadAll() instead of r.Body.Read() to properly read the entire request body. The previous approach was causing EOF errors due to incomplete body reading. All tests now pass successfully.
- Removed unused handleCaptchaValidation function (will be implemented in separate branch) - Fixed testifylint issue by using assert.NoError instead of require.NoError in HTTP handler All golangci-lint checks now pass with 0 issues.
Resolved conflicts in: - config/haproxy.cfg - internal/api/admin_handlers.go - internal/api/worker_handlers.go - lua/crowdsec.lua - pkg/spoa/root.go All AppSec functionality preserved and integrated with latest main branch changes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Core AppSec functionality implemented:
Configuration changes:
API and worker integration:
Testing infrastructure:
Architecture notes:
TODO: