This DevOps project employs a comprehensive CI pipeline to automate the development and deployment process. The architecture emphasizes security, performance, and reliability, integrating industry-leading tools and practices.
-This project is using an example microservices demo (Google Cloud) with the link shown below.
My DevOps Scripting Examples - Re-usable examples Terraform, Bash scripts and other scripting - https://github.com/T-Py-T/devops-install-scripts
Online Boutique is a cloud-first microservices demo application consisting of 11 microservices applications. The application is a web-based e-commerce app where users can browse items, add them to the cart, and purchase them.
Each of the 11 microservices written in different languages that talk to each other over gRPC.
Find Protocol Buffers Descriptions at the ./protos directory.
| Service | Language | Description |
|---|---|---|
| frontend | Go | Exposes an HTTP server to serve the website. Does not require signup/login and generates session IDs for all users automatically. |
| cartservice | C# | Stores the items in the user's shopping cart in Redis and retrieves it. |
| productcatalogservice | Go | Provides the list of products from a JSON file and ability to search products and get individual products. |
| currencyservice | Node.js | Converts one money amount to another currency. Uses real values fetched from European Central Bank. It's the highest QPS service. |
| paymentservice | Node.js | Charges the given credit card info (mock) with the given amount and returns a transaction ID. |
| shippingservice | Go | Gives shipping cost estimates based on the shopping cart. Ships items to the given address (mock) |
| emailservice | Python | Sends users an order confirmation email (mock). |
| checkoutservice | Go | Retrieves user cart, prepares order and orchestrates the payment, shipping and the email notification. |
| recommendationservice | Python | Recommends other products based on what's given in the cart. |
| adservice | Java | Provides text ads based on given context words. |
| loadgenerator | Python/Locust | Continuously sends requests imitating realistic user shopping flows to the frontend. |
| Home Page | Checkout Screen |
|---|---|
 |
 |
- Kubernetes/AKS: The app is designed to run on Kubernetes (both locally on "Docker for Desktop", as well as on the cloud with AKS).
- gRPC: Microservices use a high volume of gRPC calls to communicate to each other.
- Synthetic Load Generation: The application demo comes with a background job that creates realistic usage patterns on the website using Locust load generator.
- Automation: The build, test, and deployment process is automated, reducing the risk of human error and speeding up the cycle times. Automation ensures that every code change is tested and validated before deployment.
- Security First: Integrating Aqua Trivy ensures that security vulnerabilities and code quality issues are detected and addressed early in the pipeline, fostering a secure development lifecycle.
- Scalability: Kubernetes provides a scalable infrastructure that can handle fluctuating loads, ensuring consistent performance during peak traffic.
- Version Control and Code Review: GitHub serves as the foundation for collaboration and quality control, ensuring that only well-reviewed, high-quality code reaches production.
- Secrets Management: Docker credentials are stored securely using Azure Devops variables.
- Static Analysis: Trivy is used for static analysis.
- Build and Push Images: Docker images are built and pushed to Docker Hub.
- Image Scanning: Docker images are pulled and scanned for vulnerabilities using Trivy.
- Pull and Test Images: Docker images are pulled and tested.
The CI/CD pipeline is depicted in the diagram below, which mirrors the "as-built" system, showcasing the tools and workflows utilized.
- GitHub:
- Serves as the backbone of version control, ensuring seamless collaboration among team members.
- Pull requests and branch strategies help enforce coding standards and encourage peer reviews.
- Integrated with Jenkins to trigger automated builds and tests upon code commits, ensuring continuous integration.
- Setup with the Azure Devops Project with connection to the git repository
Azure DevOps:
- Orchestrates the CI/CD pipeline, ensuring that builds, tests, and deployments are fully automated.
- Integrates with tools like GitHub and Docker to create a streamlined process from code commit to deployment.
- Provides real-time feedback to developers about build status and test results.
- After successful build the pipeline will update the deployment-services.yaml
Azure Pipelines
CI Pipelines
Updates to YAML from Pipelines
Release Pipelines
Dev AKS Deployment
Prod AKS Deployment
Azure DevOps - TODO: - Regional vCPU - Constraints in allocations requried me to keep requesting additional resources. - This is cost prohibitive, so 1 pod for each service is left - Update to Terraform Apply - Currently elies on external Terraform setup for environment to work - When cluster is built API changes - Permissions for Jenkins need to be created with cluster (kubectl) - Separate GitOps Repo - Move deployment code to its own repo instead of a branch of this repo
-
Aqua Trivy:
- Scans Docker docs/img and source code for vulnerabilities, ensuring that potential security issues are caught before deployment.
- Generates detailed reports that can be used to address vulnerabilities promptly.
Trivy File Scan of /src/
Trivy Image Scan results
- Docker:
- Packages the Java application into lightweight, portable containers, ensuring consistent environments across development, testing, and production stages.
- Simplifies deployment by abstracting underlying infrastructure differences.
- Azure Kubernetes Service (AKS):
- Manages the deployment and scaling of containerized applications in a highly available environment.
- Ensures zero downtime by automatically scaling and redistributing workloads as needed.
- Namespace configurations (e.g.,
webapps) isolate different parts of the system for better organization and security.