EKS Grafana Deployment for ServiceNow Workloads: How to Implement

The terraform-eks-grafana repo provisions a production-ready EKS cluster (via terraform-eks-cluster), then deploys Grafana, Prometheus, and Loki. Optional RDS (PostgreSQL) can be used as Grafana's backend. Use it to visualize metrics, logs, and traces for ServiceNow and mid-server workloads.

Prerequisites

• Terraform >= 1.5.0, < 2.0.0
• AWS credentials (e.g. AWS_PROFILE or env vars)
• kubectl and helm (optional, for local use)

Quick start

1. Clone and enter the repo

git clone https://github.com/timabiok/terraform-eks-grafana.git
cd terraform-eks-grafana

2. Copy example variables and set cluster name (and optionally RDS)

cp terraform.auto.tfvars.example terraform.auto.tfvars
# Edit: cluster_name, region, env; set grafana_use_rds = true to use RDS for Grafana

3. Initialize and apply (first apply creates EKS; you may need to run apply twice so Helm can connect)

terraform init
terraform plan
terraform apply

If Helm releases fail on first apply (cluster not ready), run:

aws eks update-kubeconfig --region <region> --name <cluster_name>
terraform apply

4. Configure kubectl and access Grafana

$(terraform output -raw configure_kubectl)
kubectl port-forward -n observability svc/kube-prometheus-stack-grafana 3000:80

Open http://localhost:3000 — default user/password from grafana_admin_user / grafana_admin_password (or see tfvars).

Optional: Grafana with RDS

Set grafana_use_rds = true in tfvars. Terraform creates an RDS PostgreSQL instance in the same VPC, a security group allowing EKS nodes → RDS on port 5432, and a Kubernetes secret with GF_DATABASE_* so Grafana uses RDS as its database.

Inputs schema (from repo)

Variable	Description	Default
cluster_name	EKS cluster name	(required)
create_vpc	Create VPC (true) or use existing (false)	true
grafana_admin_user / grafana_admin_password	Grafana admin credentials	admin / admin
grafana_use_rds	Use RDS PostgreSQL for Grafana	false
observability_namespace	Namespace for Grafana, Prometheus, Loki	observability
kube_prometheus_stack_version	kube-prometheus-stack Helm version	55.5.0
loki_chart_version	Loki Helm chart version	5.41.0

Production checklist

From the repo: use a remote backend (S3 + DynamoDB); prefer an existing K8s secret for Grafana admin password; set grafana_use_rds = true; enable Ingress + TLS; use grafana_replicas >= 2 and multi-AZ node groups for HA. See terraform.auto.tfvars.production.example and the README.

Data sources and ServiceNow

Grafana is preconfigured with Prometheus and Loki datasources. Add CloudWatch or ServiceNow REST APIs for CMDB and incident data. Build dashboards for mid-server queue depth, discovery runs, and integration errors.

Takeaways

• Use terraform-eks-grafana for EKS + Grafana + Prometheus + Loki in one apply; optional RDS for Grafana persistence.
• Run terraform apply twice if Helm fails on first run; then port-forward or configure Ingress to access Grafana.
• Point Grafana at Prometheus, Loki, CloudWatch, or ServiceNow for a unified observability layer.