Skip to content

Expose failed API requrests through event log #2177

@Rohlik

Description

@Rohlik

Describe the bug
When provisioning invalid GrafanaDatasource, you will encounter an error like the one below, which isn't helpful because it doesn't give a reason. I spent quite a bit of time debugging it and figured out what was wrong 😥.
In the end, I was able to find out that the root cause was missing access: proxy parameter within the definition, even though official documentation doesn't explicitly mention it in the "minimal working example": https://grafana.com/docs/plugins/yesoreyeram-infinity-datasource/latest/setup/provisioning/#basic-provisioning.

Moreover, this plugin can generate its provisioning definition within its settings, and the generated config doesn't even include an access property 😕:
Image

It's only partly related, but there is also a JSON validation scheme for this GrafanaDatasource CR at https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/grafana.integreatly.org/grafanadatasource_v1beta1.json#', but even there, access is not set as required.

Error log:

{"level":"error","ts":"2025-09-05T07:31:06Z","msg":"Reconciler error","controller":"grafanadatasource","controllerGroup":"grafana.integreatly.org","controllerKind":"GrafanaDatasource","GrafanaDatasource":{"name":"infinity-datasource","namespace":"monitoring"},"namespace":"monitoring","name":"infinity-datasource","reconcileID":"0b292f41-360e-4c9d-bdb0-194e4f5ea6e4","error":"failed to apply to all instances: map[monitoring/grafana:[POST /datasources] addDataSource (status 400): {}]","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:353\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:300\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.1\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:202"}

Version
v5.19.4

To Reproduce
Steps to reproduce the behavior:

  1. Apply this CR in your K8s
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDatasource
metadata:
  name: infinity-datasource
spec:
  resyncPeriod: 30s
  instanceSelector:
    matchLabels:
      grafana_dashboard: "true"
  datasource:
    name: Infinity-Datasource
    type: yesoreyeram-infinity-datasource
    uid: INFINITY
  1. Datasource is not provisioned
  2. Check the logs of grafana-operator to see the error

Expected behavior
Provide a clear explanation within the logs of the issue to help resolve the error more quickly. Something like:

Datasource needs to have access property defined.

Suspect component/Location where the bug might be occurring
unknown

Runtime (please complete the following information):

  • OS: Linux
  • Grafana Operator Version v5.19.4
  • Environment: EKS
  • Deployment type: Helm

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage/acceptedIndicates an issue or PR is ready to be actively worked on.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions