BeginnerLesson 3 of 9

Data Sources (Prometheus)

Connect Grafana with Prometheus, validate queries, and troubleshoot common datasource issues.

Simple Explanation (ELI5)

Grafana needs a data source to draw charts. Prometheus is the most common one for metrics. Connect once, then all panels can query it.

Technical Explanation

A Grafana Prometheus datasource points to the Prometheus base URL. Grafana sends PromQL queries to that endpoint and renders results. In Kubernetes, this URL is often a service DNS name in the monitoring namespace.

Visual Section

Grafana
Prometheus API
Metrics Results

Hands-on Commands

yaml
apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus-server.monitoring.svc.cluster.local:9090
    isDefault: true
bash
# Prometheus health check
curl http://prometheus-server.monitoring.svc.cluster.local:9090/-/ready

# Grafana health check
curl http://localhost:3000/api/health

# Kubernetes port-forward test
kubectl -n monitoring port-forward svc/prometheus-server 9090:9090

Debugging Scenarios

Real-world Use Case

A team connects Grafana to HA Prometheus and uses one default datasource across all service dashboards for consistent metrics access.

Interview Questions

Beginner

What is a Grafana datasource?

A configured backend Grafana queries for visualization and alerts.

Why connect Grafana to Prometheus?

Prometheus provides time-series metrics that Grafana visualizes.

What protocol is typically used?

HTTP/HTTPS to the Prometheus API endpoint.

What does Test & Save do?

Validates datasource connectivity and basic query capability.

Can one dashboard use multiple data sources?

Yes, panels can point to different data sources.

Intermediate

How do you configure Grafana datasource as code?

Use provisioning files under Grafana provisioning/datasources.

Proxy mode vs direct mode?

Proxy mode routes queries via Grafana server; direct mode uses browser-to-datasource path.

Why datasource permissions matter?

To prevent unauthorized query access and accidental exposure.

How handle Prometheus URL in Kubernetes?

Use service DNS names and namespace-aware endpoints.

How validate datasource latency impact?

Measure query response times and panel render durations.

Scenario-based

Grafana says datasource down, but Prometheus UI works from your laptop. Why?

Grafana server network path differs from your laptop path.

Query returns data in Prometheus UI but no data in panel. What next?

Check panel datasource selection, time range, and variable filters.

TLS-enabled Prometheus fails in Grafana. Common fix?

Configure proper CA cert/auth settings and endpoint URL.

How would you switch datasource between staging and prod safely?

Use datasource variables and environment-specific provisioning.

Datasource works but dashboards are slow. Why?

Heavy PromQL queries, large time windows, and frequent refresh.

Summary

Prometheus integration is the foundation of practical Grafana usage. If datasource setup is solid, dashboard development becomes reliable and fast.