DISCLAIMER: The examples in this article come with no support or warranty, explicit or implied. Caveat Emptor!

 

The ServiceNow Kubernetes Informer (KVA) discovers and reports Kubernetes cluster topology to your ServiceNow instance. When running on Amazon Elastic Kubernetes Service (EKS), you can supply the informer's ServiceNow credentials via AWS Secrets Manager rather than storing them as a plain Kubernetes secret. This article walks through the complete setup end-to-end, from creating the EKS cluster through to a running informer pod.

The flow has five stages:

1. Create the EKS cluster
2. Store credentials in AWS Secrets Manager
3. Create the IAM role that lets EKS pods read the secret (IRSA)
4. Install the AWS Secrets and Configuration Provider (ASCP) in the cluster
5. Install the KVA informer via Helm

Versions used in this article: KVA Informer Helm chart 2.7.1, eksctl 0.195, AWS CLI v2. Adjust commands as needed for your environment.

Prerequisites

• AWS CLI v2 installed and configured (aws configure)
• eksctl installed
• Helm 3 installed
• kubectl installed
• An AWS account with permissions to create EKS clusters, IAM roles, and Secrets Manager secrets
• A ServiceNow instance username and password you want the informer to use

Step 1 — Create the EKS Cluster

If you already have an EKS cluster, skip to Step 2. The command below creates a minimal two-node cluster. Adjust the region, node type, count, and other options for your needs.

eksctl create cluster \
  --name my-kva-cluster \
  --region us-east-1 \
  --node-type m5.large \
  --nodes 2 \
  --nodes-min 2 \
  --nodes-max 4 \
  --managed \
--with-oidc

Cluster creation takes 10–15 minutes. When complete, eksctl updates your local kubeconfig automatically. Verify the nodes are ready:

kubectl get nodes

Both nodes should show Ready status before proceeding.

OIDC provider: eksctl creates an OIDC provider for the cluster automatically when you use --managed. If you created your cluster another way, associate one now:

eksctl utils associate-iam-oidc-provider \
  --cluster my-kva-cluster \
  --approve

The OIDC provider URL is required in Step 3.

Step 2 — Store Credentials in AWS Secrets Manager

The informer expects the secret to be a JSON object with exactly two keys: username and password. These are the credentials for your ServiceNow instance.

aws secretsmanager create-secret \
  --name kva/informer-credentials \
  --region us-east-1 \
  --secret-string '{"username":"svc-kva-user","password":"YourPasswordHere"}'

Note the ARN returned in the output — you will need it in Steps 3 and 5. It will look like:

arn:aws:secretsmanager:us-east-1:123456789012:secret:kva/informer-credentials-AbCdEf

Key names matter. The informer Helm chart maps username and password via JMESPath. If the keys in your secret do not match exactly (case-sensitive), the CSI driver will mount the volume but the informer will fail to start with a warning like JMES Path - username ... does not point to a valid object.

Verify the key names without exposing values:

aws secretsmanager get-secret-value \
  --secret-id arn:aws:secretsmanager:us-east-1:123456789012:secret:kva/informer-credentials-AbCdEf \
  --query 'SecretString' \
  | python3 -c "import sys,json; print(list(json.loads(json.load(sys.stdin)).keys()))"

Expected output: ['username', 'password']

Step 3 — Create the IAM Role (IRSA)

IAM Roles for Service Accounts (IRSA) lets a Kubernetes service account assume an IAM role without storing AWS credentials anywhere in the cluster. The CSI driver uses this role to call Secrets Manager on behalf of the informer pod.

3a. Create the IAM policy

Save the following to kva-secret-policy.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:kva/informer-credentials-AbCdEf"
    }
  ]
}

aws iam create-policy \
  --policy-name kva-informer-secret-policy \
  --policy-document file://kva-secret-policy.json

Note the policy ARN returned.

3b. Create the IAM service account

eksctl create iamserviceaccount creates the IAM role, builds the trust policy (scoped to the correct OIDC provider and service account), and creates the Kubernetes service account — all in one step.

Service account name matters. The KVA Helm chart creates a service account named servicenow-{instance.name} in the release namespace. Use that exact name here so the IAM trust policy matches what the pod presents. In the example below, the instance name is myinstance and the namespace is sn-kva.

eksctl create iamserviceaccount \
  --name servicenow-myinstance \
  --namespace sn-kva \
  --cluster my-kva-cluster \
  --attach-policy-arn arn:aws:iam::123456789012:policy/kva-informer-secret-policy \
  --approve \
  --override-existing-serviceaccounts

3d. Get the ARN for the role created by eksctl

aws cloudformation describe-stacks --stack-name <stack name from above> \
  --query "Stacks[0].Outputs[?OutputKey='Role1'].OutputValue" \
  --output text

3e. Verify the trust policy

Confirm the trust policy references the correct service account name and namespace:

aws iam get-role \
  --role-name <role-name-from-above> \
  --query 'Role.AssumeRolePolicyDocument.Statement[0].Condition'

The :sub condition must read:

"system:serviceaccount:sn-kva:servicenow-myinstance"

Common pitfall. If you created the IAM role separately (e.g. through the console or for a different cluster), the trust policy :sub condition may reference a different service account name. This causes a 403 AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity error at mount time. Correct it with:

aws iam update-assume-role-policy \
  --role-name <role-name> \
  --policy-document '{ ... correct trust policy ... }'

Then restart the informer deployment.

3f. Annotate the service account for use by the informer

kubectl annotate sa servicenow-myinstance -n sn-kva \
  meta.helm.sh/release-name=k8s-informer-myinstance \
  meta.helm.sh/release-namespace=sn-kva \
  --overwrite && \
kubectl label sa servicenow-myinstance -n sn-kva \
  app.kubernetes.io/managed-by=Helm \
  --overwrite

Step 4 — Install the AWS Secrets and Configuration Provider (ASCP)

The ASCP consists of two Helm charts: the Secrets Store CSI Driver and the AWS provider. Both must be installed before the informer. Follow the AWS documentation or use the commands below.

# Add repos
helm repo add secrets-store-csi-driver \
  https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
helm repo add aws-secrets-manager \
  https://aws.github.io/secrets-store-csi-driver-provider-aws
helm repo update

# Install CSI driver
helm install secrets-store-csi-driver \
  secrets-store-csi-driver/secrets-store-csi-driver \
  --namespace aws-secrets-manager \
  --create-namespace \
  --set syncSecret.enabled=true
--set tokenRequests[0].audience="sts.amazonaws.com"

# Install AWS provider
helm install secrets-provider-aws \
  aws-secrets-manager/secrets-store-csi-driver-provider-aws \
  --namespace aws-secrets-manager --set secrets-store-csi-driver.install=false

Verify all pods are Running before proceeding:

kubectl get pods -n aws-secrets-manager

You should see secrets-store-csi-driver-* and aws-secrets-store-csi-driver-provider-* pods in Running state.

Step 5 — Install the KVA Informer via Helm

Create the target namespace if it does not already exist:

kubectl create namespace sn-kva

Install the informer, substituting your values for instance name, cluster name, secret ARN, and role ARN:

helm install k8s-informer-myinstance \
  https://install.service-now.com/glide/distribution/builds/package/informer/2.7.1/informer-helm-2.7.1.tgz \
  --namespace sn-kva \
  --set acceptEula=Y \
  --set instance.name=myinstance \
  --set clusterName="my-kva-cluster" \
  --set secretProvider=aws \
  --set awsSecretManagerSecretArn="arn:aws:secretsmanager:us-east-1:123456789012:secret:kva/informer-credentials-AbCdEf" \
  --set awsSecretManagerRoleArn="arn:aws:iam::123456789012:role/eksctl-my-kva-cluster-addon-iamserviceaccount-..."

The Helm chart uses the role ARN to annotate the service account it creates (eks.amazonaws.com/role-arn), and creates a SecretProviderClass that tells the CSI driver which secret to mount and which JSON keys to extract.

Monitor the pod coming up:

kubectl get pods -n sn-kva --watch

The pod first shows Init status while the CSI driver mounts the secret volume, then transitions to Running.

Troubleshooting

Symptom	Likely cause	Resolution
FailedMount ... Failed to fetch secret from all regions	IAM trust policy :sub references wrong service account name or namespace	Check trust policy with aws iam get-role ... --query 'Role.AssumeRolePolicyDocument'. Update :sub to match system:serviceaccount:<namespace>:servicenow-<instance.name>, then restart the deployment.
AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity	Trust policy OIDC provider ID references a different cluster	Verify the OIDC ID in the trust policy matches aws eks describe-cluster --name <cluster> --query 'cluster.identity.oidc.issuer'.
JMES Path - username ... does not point to a valid object	Secret JSON keys do not match username / password	Inspect key names (without exposing values) and update the secret value to use exactly username and password.
Pod stuck in Pending with node(s) had untolerated taint(s)	Nodes not Ready or tainted	Run kubectl get nodes and kubectl describe node <name>. Check CNI pod logs in kube-system if nodes show NotReady.
ASCP pods not found	CSI driver installed in wrong namespace	Run `kubectl get pods -A

Useful diagnostic commands

# Check pod events
kubectl describe pod -n sn-kva -l app=k8s-informer-myinstance

# CSI driver logs (mount-level errors)
kubectl logs -n aws-secrets-manager -l app=secrets-store-csi-driver | grep -i "error\|fail"

# AWS provider logs (IAM / Secrets Manager errors)
kubectl logs -n aws-secrets-manager <aws-secrets-store-csi-driver-provider-pod> | tail -30

# Verify SecretProviderClass rendered correctly
kubectl get secretproviderclass -n sn-kva -o yaml

# Verify service account has correct IAM annotation
kubectl get sa servicenow-myinstance -n sn-kva -o yaml | grep role-arn

# Force pod restart after trust policy fix
kubectl rollout restart deployment/k8s-informer-myinstance -n sn-kva