Skip to content

Setup Amazon Managed Service for Prometheus (AMP) on AWS EKS

Avatar photo

Hi everyone! Here i am setting up Amazon Managed Service for Prometheus (AMP) to monitor my AWS EKS. Sharing the journey to and steps to provision AMP below:

Amazon Managed Service for Prometheus (AMP) is an AWS fully-managed monitoring and alerting service which is 100% compatible with Prometheus that makes it easy to monitor containerized applications and infrastructure at scale.

AMP supports the same metrics, the same PromQL queries, and can also make use of the 150+ Prometheus exporters. AMP runs across multiple Availability Zones for high availability, and is powered by CNCF Cortex for horizontal scalability. AMP will easily scale to ingest, store, and query millions of time series metrics.

No alt text provided for this image

The unique architecture for Amazon Managed Prometheus (AMP) is that you will still require a Prometheus server (deployment pod) which retrieve metrics/data from AWS EKS API and pushes those data to Amazon Managed Prometheus (AMP).

Deployment of Amazon Managed Prometheus (AMP)


  • Provision VPC, Subnets, Route Tables and Security Groups
  • Existing or new AWS EKS Cluster
  • Visual Studio Code IDE (Optional)

Step1: Configure Permission for Prometheus & Grafana

Enable service accounts to access AWS resources in three steps

1. Create an IAM OIDC provider for your cluster – You only need to do this once for a cluster.

2. Create an IAM role and attach an IAM policy to it with the permissions that your service accounts need – We recommend creating separate roles for each unique collection of permissions that pods need.

3. Associate an IAM role with a service account – Complete this task for each Kubernetes service account that needs access to AWS resources.

From the steps above, what you are performing is to create an AWS IAM role with attached IAM policy to be associated to a Kubernetes service account.

Step2:  Create an AWS Prometheus Workspace 

AWS Console > AWS AMP > Create a AWS Prometheus workspace (eg. Prometheus-Workspace )

No alt text provided for this image

Step3:  Deploy Prometheus server

Amazon Managed Service for Prometheus does not directly scrape operational metrics from containerized workloads in a Kubernetes cluster. It requires users to deploy and manage a standard Prometheus server.

Set the following environment variables before deploying the helm chart :


WORKSPACE_ID=<AWS Prometheus Workspace ID>

AWS_REGION=<your AWS account region>
  • IAM_PROXY_PROMETHEUS_ROLE_ARN‘ – is the IAM role name you have created in Step1
  • WorkSPACE_ID– is the Prometheus Workspace ID which you have created a Prometheus Workspace in Step2
  • AWS_Region – is the region in where your AWS Prometheus is deployed, i have input ‘ap-southeast-1‘ which is the Singapore region.

Now create a file called ‘amp_ingest_override_values.yaml‘ with the following content in it:

iamproxy-service-account is the newly created Kubernetes service account

The following code below is a set of default values for prometheus server helm chart which enable remoteWrite to AMP:

  ## Disable alert manager roles
        name: "iamproxy-service-account"
    create: false

  ## Disable pushgateway
    create: false

        max_samples_per_send: 1000
        max_shards: 200
        capacity: 2500

  ## Use a statefulset instead of a deployment for resiliency
    enabled: true

  ## Store blocks locally for short time period only
  retention: 1h
## Disable alert manager
  enabled: false

## Disable pushgateway
  enabled: false

Deploy the helm chart:

kubectl apply -f amp_ingest_override_values.yaml

Now we can deploy the Prometheus server (Pod) after you have set the configuration:

helm repo add prometheus-community
kubectl create ns prometheus

helm install prometheus-for-amp prometheus-community/prometheus -n prometheus -f ./amp_ingest_override_values.yaml \
--set serviceAccounts.server.annotations."eks\.amazonaws\.com/role-arn"="${IAM_PROXY_PROMETHEUS_ROLE_ARN}" \
--set server.remoteWrite[0].url="https://aps-workspaces.${AWS_REGION}${WORKSPACE_ID}/api/v1/remote_write" \
--set server.remoteWrite[0].sigv4.region=${AWS_REGION}

Now check the Service account in Kubernetes (AWS EKS):

kubectl get ServiceAccount -A

kubectl get pods --all-namespaces

Step4: Deploy Grafana

Enter the following code to deploy Grafana pods in your AWS EKS cluster :

#Add Grafana helm chart rep
helm repo add grafana

#Create a Grafana namespace
kubectl create ns grafana

#Install Grafana for AMP
helm install grafana-for-amp grafana/grafana -n grafana

#Check the Grafana pods
kubectl get pods --all-namespaces
  • Now Update your Grafana server to use the AWS signing proxy.
  • Create a new file and name it ‘amp_query_override_values.yaml‘. (This file will be used to update your Grafana deployment to enable the Sigv4 protocol which the AWS signing proxy uses to authenticate)
    name: "iamproxy-service-account"
    annotations: "${IAM_PROXY_PROMETHEUS_ROLE_ARN}"
    sigv4_auth_enabled: true:
  • Now execute the following command to update yourGrafana environment
helm upgrade --install grafana-for-amp grafana/grafana -n grafana -f ./amp_query_override_values.yaml

Port forward Grafana deployment so we can access the UI from our browser:

#Check Grafana pods
kubectl get pods --all-namespaces

#Port forward Grafana so we can access the dashboard UI from our browser port 5001
kubectl port-forward -n grafana pods/<grafana pod name> 5001:3000

Get the default Grafana admin password:

kubectl get secrets grafana-for-amp -n grafana -o jsonpath='{.data.admin-password}'|base64 --decode
  • Now access via your URL browser: http://localhost:5001
  • Username: admin
  • Password: <What you generated secrets from above>

Step5: Configure Grafana with Amazon Managed Service for Prometheus

  • Before we can visualise the metrics in Grafana, its has to be configured with 1 or more data sources. Lets specify the workspace within Amazon Managed Service for Prometheus (AMP).
  1. Grafana Configurations > Data Sources > Select Prometheus as data source

2. Input the AWS AMP Workspace query URL under HTTP/URL

3. Turn on SigV4 auth

4. Change default region of the AMP and AWS EKS cluster resides in

5. Lastly, Save & Test the data source. (Ensure the tick and Data source is working)

Step6: Deploy dashboard to monitor our workload in AWS EKS Cluster

  1. On the main page of Grafana login, lets add in some default dashboard by selecting Import:
  • Input ‘3119’ and press Load [ Kubernetes cluster monitoring (via Prometheus) ]
  • Then select Prometheus > Import

Done! You have completed deploying Amazon Managed Service for Prometheus (AMP) together with Grafana into your AWS EKS cluster


  • Cloudonaut ( )


The views expressed and the content shared in all published articles on this website are solely those of the respective authors, and they do not necessarily reflect the views of the author’s employer or the techbeatly platform. We strive to ensure the accuracy and validity of the content published on our website. However, we cannot guarantee the absolute correctness or completeness of the information provided. It is the responsibility of the readers and users of this website to verify the accuracy and appropriateness of any information or opinions expressed within the articles. If you come across any content that you believe to be incorrect or invalid, please contact us immediately so that we can address the issue promptly.


Avatar photo
Schnauzer Lover | Amazon Web Services | Microsoft Azure | An individual passionate in commercial cloud - design, operations & ever changing automation on infrastructure. Evergreen learning is what i believe , it is a journey not a destination


1 Response

  1. […] on the previous article of setting up Amazon Managed Service for Prometheus (AMP) with open source Grafana , we will now be provisioning Amazon Managed Grafana to integrated with […]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.