|
| 1 | +# datadog-service |
| 2 | + |
| 3 | +[](https://goreportcard.com/report/github.com/keptn-sandbox/datadog-service) |
| 4 | + |
| 5 | +This implements the `datadog-service` that integrates the [Datadog](https://en.wikipedia.org/wiki/Datadog) observability platform with Keptn. This enables you to use Datadog as the source for the Service Level Indicators ([SLIs](https://keptn.sh/docs/0.19.x/reference/files/sli/)) that are used for Keptn [Quality Gates](https://keptn.sh/docs/concepts/quality_gates/). |
| 6 | +If you want to learn more about Keptn visit us on [keptn.sh](https://keptn.sh) |
| 7 | + |
| 8 | +Check the issue on the main repo for more info: https://github.com/keptn/keptn/issues/2652 |
| 9 | + |
| 10 | + |
| 11 | +## Quickstart |
| 12 | +If you are on Mac or Linux, you can use [examples/kup.sh](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/examples/kup.sh) to set up a local Keptn installation that uses Datadog. This script creates a local minikube cluster, installs Keptn, Istio, Datadog and the Datadog integration for Keptn (check the script for pre-requisites). |
| 13 | + |
| 14 | +To use the script, |
| 15 | +```bash |
| 16 | +export DD_API_KEY="<your-datadog-api-key>" DD_APP_KEY="<your-datadog-app-key>" DD_SITE="datadoghq.com" |
| 17 | +examples/kup.sh |
| 18 | +``` |
| 19 | +Check [the official docs](https://docs.datadoghq.com/account_management/api-app-keys/) for how to create the Datadog API key and Application key |
| 20 | + |
| 21 | +Note: Application keys get the same permissions as you. You might want to narrow down the permissions (datadog-service only reads metrics from the API. Check the official docs linked above for more information). |
| 22 | + |
| 23 | +## If you already have a Keptn cluster running |
| 24 | +1. Install datadog |
| 25 | +```bash |
| 26 | +export DD_API_KEY="<your-datadog-api-key>" DD_APP_KEY="<your-datadog-app-key>" DD_SITE="datadoghq.com" |
| 27 | +helm install datadog --set datadog.apiKey=${DD_API_KEY} datadog/datadog --set datadog.appKey=${DD_APP_KEY} --set datadog.site=${DD_SITE} --set clusterAgent.enabled=true --set clusterAgent.metricsProvider.enabled=true --set clusterAgent.createPodDisruptionBudget=true --set clusterAgent.replicas=2 |
| 28 | + |
| 29 | +``` |
| 30 | +2. Install Keptn datadog-service to integrate Datadog with Keptn |
| 31 | +```bash |
| 32 | +export DD_API_KEY="<your-datadog-api-key>" DD_APP_KEY="<your-datadog-app-key>" DD_SITE="datadoghq.com" |
| 33 | +# cd datadog-service |
| 34 | +helm install datadog-service ./helm --set datadogservice.ddApikey=${DD_API_KEY} --set datadogservice.ddAppKey=${DD_APP_KEY} --set datadogservice.ddSite=${DD_SITE} |
| 35 | +``` |
| 36 | + |
| 37 | +3. Add SLI and SLO |
| 38 | +```bash |
| 39 | +keptn add-resource --project="<your-project>" --stage="<stage-name>" --service="<service-name>" --resource=/path-to/your/sli-file.yaml --resourceUri=datadog/sli.yaml |
| 40 | +keptn add-resource --project="<your-project>" --stage="<stage-name>" --service="<service-name>" --resource=/path-to/your/slo-file.yaml --resourceUri=slo.yaml |
| 41 | +``` |
| 42 | +Example: |
| 43 | +```bash |
| 44 | +keptn add-resource --project="podtatohead" --stage="hardening" --service="helloservice" --resource=./quickstart/sli.yaml --resourceUri=datadog/sli.yaml |
| 45 | +keptn add-resource --project="podtatohead" --stage="hardening" --service="helloservice" --resource=./quickstart/slo.yaml --resourceUri=slo.yaml |
| 46 | +``` |
| 47 | +Check [./quickstart/sli.yaml](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/examples/quickstart/sli.yaml) and [./quickstart/slo.yaml](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/examples/quickstart/slo.yaml) for example SLI and SLO. |
| 48 | + |
| 49 | +4. Configure Keptn to use datadog SLI provider |
| 50 | +Use keptn CLI version [0.15.0](https://github.com/keptn/keptn/releases/tag/0.15.0) or later. |
| 51 | +```bash |
| 52 | +keptn configure monitoring datadog --project <project-name> --service <service-name> |
| 53 | +``` |
| 54 | + |
| 55 | +5. Trigger delivery |
| 56 | +```bash |
| 57 | +keptn trigger delivery --project=<project-name> --service=<service-name> --image=<image> --tag=<tag> |
| 58 | +``` |
| 59 | +Example: |
| 60 | +```bash |
| 61 | +keptn trigger delivery --project=podtatohead --service=helloservice --image=docker.io/jetzlstorfer/helloserver --tag=0.1.1 |
| 62 | +``` |
| 63 | +Observe the results in the [Keptn Bridge](https://keptn.sh/docs/0.19.x/bridge/) |
| 64 | +## Compatibility Matrix |
| 65 | + |
| 66 | +*Please fill in your versions accordingly* |
| 67 | + |
| 68 | +| Keptn Version | [datadog-service Docker Image](https://github.com/keptn-sandbox/datadog-service/pkgs/container/datadog-service) | |
| 69 | +|:----------------:|:----------------------------------------:| |
| 70 | +| 0.11.4 | ghcr.io/keptn-sandbox/datadog-service:0.1.0 | |
| 71 | +| 0.11.4 | ghcr.io/keptn-sandbox/datadog-service:0.2.0 | |
| 72 | +| 0.15.0 | ghcr.io/keptn-sandbox/datadog-service:0.15.0 | |
| 73 | +| 0.19.0 | ghcr.io/keptn-sandbox/datadog-service:0.19.0 | |
| 74 | + |
| 75 | +datadog-service version will match Keptn version starting from 0.15.0 version of Keptn e.g., datadog-service 0.15.x is compatible with Keptn 0.15.x |
| 76 | + |
| 77 | +## Installation |
| 78 | + |
| 79 | +```bash |
| 80 | +export DD_API_KEY="<your-datadog-api-key>" DD_APP_KEY="<your-datadog-app-key>" DD_SITE="datadoghq.com" |
| 81 | +# cd datadog-service |
| 82 | +helm install datadog-service ./helm --set datadogservice.ddApikey=${DD_API_KEY} --set datadogservice.ddAppKey=${DD_APP_KEY} --set datadogservice.ddSite=${DD_SITE} |
| 83 | +``` |
| 84 | +Tell Keptn to use datadog as SLI provider for your project/service |
| 85 | +```bash |
| 86 | +keptn configure monitoring datadog --project <project-name> --service <service-name> |
| 87 | +``` |
| 88 | + |
| 89 | +This should install the `datadog-service` together with a Keptn `distributor` into the `keptn` namespace, which you can verify using |
| 90 | + |
| 91 | +```console |
| 92 | +kubectl -n keptn get deployment datadog-service -o wide |
| 93 | +kubectl -n keptn get pods -l run=datadog-service |
| 94 | +``` |
| 95 | +### Up- or Downgrading |
| 96 | + |
| 97 | +Adapt and use the following command in case you want to up- or downgrade your installed version (specified by the `$VERSION` placeholder): |
| 98 | + |
| 99 | +```bash |
| 100 | +helm upgrade datadog-service ./helm --set datadogservice.ddApikey=${DD_API_KEY} --set datadogservice.ddAppKey=${DD_APP_KEY} --set datadogservice.ddSite=${DD_SITE} |
| 101 | +``` |
| 102 | + |
| 103 | +### Uninstall |
| 104 | + |
| 105 | +To delete a deployed *datadog-service* helm chart: |
| 106 | + |
| 107 | +```bash |
| 108 | +helm uninstall datadog-service |
| 109 | +``` |
| 110 | + |
| 111 | +## Development |
| 112 | + |
| 113 | +Development can be conducted using any GoLang compatible IDE/editor (e.g., Jetbrains GoLand, VSCode with Go plugins). |
| 114 | + |
| 115 | +It is recommended to make use of branches as follows: |
| 116 | + |
| 117 | +* `master` contains the latest potentially unstable version |
| 118 | +* `release-*` contains a stable version of the service (e.g., `release-0.1.0` contains version 0.1.0) |
| 119 | +* create a new branch for any changes that you are working on, e.g., `feature/my-cool-stuff` or `bug/overflow` |
| 120 | +* once ready, create a pull request from that branch back to the `master` branch |
| 121 | + |
| 122 | +When writing code, it is recommended to follow the coding style suggested by the [Golang community](https://github.com/golang/go/wiki/CodeReviewComments). |
| 123 | + |
| 124 | +### Where to start |
| 125 | + |
| 126 | +If you don't care about the details, your first entrypoint is [https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/eventhandlers.go](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/eventhandlers.go). Within this file |
| 127 | + you can add implementation for pre-defined Keptn Cloud events. |
| 128 | + |
| 129 | +To better understand all variants of Keptn CloudEvents, please look at the [Keptn Spec](https://github.com/keptn/spec). |
| 130 | + |
| 131 | +If you want to get more insights into processing those CloudEvents or even defining your own CloudEvents in code, please |
| 132 | + look into [https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/main.go](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/main.go) (specifically `processKeptnCloudEvent`), [https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/helm/templates](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/helm/templates), |
| 133 | + consult the [Keptn docs](https://keptn.sh/docs/) as well as existing [Keptn Core](https://github.com/keptn/keptn) and |
| 134 | + [Keptn Contrib](https://github.com/keptn-contrib/) services. |
| 135 | + |
| 136 | +### Common tasks |
| 137 | + |
| 138 | +* Build the binary: `go build -ldflags '-linkmode=external' -v -o datadog-service` |
| 139 | +* Run tests: `go test -race -v ./...` |
| 140 | +* Build the docker image: `docker build . -t ghcr.io/keptn-sandbox/datadog-service:latest` |
| 141 | +* Run the docker image locally: `docker run --rm -it -p 8080:8080 ghcr.io/keptn-sandbox/datadog-service:latest` |
| 142 | +* Push the docker image to DockerHub: `docker push ghcr.io/keptn-sandbox/datadog-service:latest` |
| 143 | +* Watch the deployment using `kubectl`: `kubectl -n keptn get deployment datadog-service -o wide` |
| 144 | +* Get logs using `kubectl`: `kubectl -n keptn logs deployment/datadog-service -f` |
| 145 | +* Watch the deployed pods using `kubectl`: `kubectl -n keptn get pods -l run=datadog-service` |
| 146 | + |
| 147 | + |
| 148 | +### Testing Cloud Events |
| 149 | + |
| 150 | +We have dummy cloud-events in the form of [RFC 2616](https://ietf.org/rfc/rfc2616.txt) requests in the [https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/test-events/](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/test-events/) directory. These can be easily executed using third party plugins such as the [Huachao Mao REST Client in VS Code](https://marketplace.visualstudio.com/items?itemName=humao.rest-client). |
| 151 | + |
| 152 | +## Automation |
| 153 | + |
| 154 | +### GitHub Actions: Automated Pull Request Review |
| 155 | + |
| 156 | +This repo uses [reviewdog](https://github.com/reviewdog/reviewdog) for automated reviews of Pull Requests. |
| 157 | + |
| 158 | +You can find the details in [https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/.github/workflows/reviewdog.yml](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/.github/workflows/reviewdog.yml). |
| 159 | + |
| 160 | +### GitHub Actions: Unit Tests |
| 161 | + |
| 162 | +This repo has automated unit tests for pull requests. |
| 163 | + |
| 164 | +You can find the details in [https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/.github/workflows/tests.yml](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/.github/workflows/tests.yml). |
| 165 | + |
| 166 | +## How to release a new version of this service |
| 167 | + |
| 168 | +It is assumed that the current development takes place in the master branch (either via Pull Requests or directly). |
| 169 | + |
| 170 | +To make use of the built-in automation using GH Actions for releasing a new version of this service, you should |
| 171 | + |
| 172 | +* branch away from master to a branch called `release-x.y.z` (where `x.y.z` is your version), |
| 173 | +* write release notes in the [https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/releasenotes/](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/releasenotes/) folder, |
| 174 | +* check the output of GH Actions builds for the release branch, |
| 175 | +* verify that your image was built and pushed to GHCR with the right tags, |
| 176 | +* update the image tags in [deploy/service.yaml], and |
| 177 | +* test your service against a working Keptn installation. |
| 178 | + |
| 179 | +If any problems occur, fix them in the release branch and test them again. |
| 180 | + |
| 181 | +Once you have confirmed that everything works and your version is ready to go, you should |
| 182 | + |
| 183 | +* create a new release on the release branch using the [GitHub releases page](https://github.com/keptn-sandbox/datadog-service/releases), and |
| 184 | +* merge any changes from the release branch back to the master branch. |
| 185 | + |
| 186 | +## Known problems |
| 187 | +1. If the evaluation window of the query is too short, the api might return an empty result which datadog-service treats as 0 and fails the evaluation. [Issue](https://github.com/keptn-sandbox/datadog-service/issues/10) |
| 188 | +2. There is an on-purpose 60s delay before the datadog metrics API is called. This is because, calling the metrics API earlier leads to incorrect data. [Issue](https://github.com/keptn-sandbox/datadog-service/issues/8) |
| 189 | +3. Does not support default queries for throughput, error rate, request latency etc., i.e., you have to enter the entire query. [Issue](https://github.com/keptn-sandbox/datadog-service/issues/9) |
| 190 | + |
| 191 | +## License |
| 192 | + |
| 193 | +Please find more information in the [https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/LICENSE](https://raw.githubusercontent.com/keptn-sandbox/datadog-service/release-0.19.0/LICENSE) file. |
0 commit comments