Jul 6, 2025·15 min read·

Using AWS Q CLI with EKS MCP for Production Troubleshooting

Stop memorizing kubectl commands. Use Amazon Q + EKS MCP to diagnose pod crashes, debug networking, analyze logs, and fix production incidents through natural language.

AWS QEKSMCPKubernetesProduction

Why This Matters

Production Kubernetes troubleshooting is complex — you need to check pods, events, logs, resource limits, network policies, HPA status, and node conditions. With AWS Q CLI connected to an EKS MCP server, you describe the problem in plain English and the AI runs the right kubectl/AWS commands for you.

Without MCP

Manually run 10+ kubectl commands, cross-reference logs, check CloudWatch, inspect service mesh — 30+ minutes per incident.

With MCP

Ask Q: "Why is my payment service crashing?" — it checks pods, logs, events, and gives you the root cause in seconds.

Prerequisites

▹AWS Q Developer CLI installed (q --version)
▹An EKS cluster running in production
▹kubectl configured with cluster access
▹AWS CLI configured with EKS permissions
▹uv/uvx installed (for MCP server)

How It All Connects

AWS Q CLI with EKS MCP Server - Architecture for Production Troubleshooting

Step 1: Install Amazon Q Developer CLI

macOS

# Install via Homebrew

brew install --cask amazon-q

# Authenticate

q login

Ubuntu / Linux (CLI only — no GUI needed)

# Download CLI zip

curl --proto '=https' --tlsv1.2 -sSf https://desktop-release.codewhisperer.us-east-1.amazonaws.com/latest/q-x86_64-linux.zip -o q-cli.zip

# Extract and install

unzip q-cli.zip
sudo mv q /usr/local/bin/q
chmod +x /usr/local/bin/q

# Verify and login

q --version
q login

Auth: Use AWS Builder ID (free) or IAM Identity Center. The free tier includes full CLI + MCP support.

Step 2: Install uv & uvx (MCP Runtime)

MCP servers run via uvx — a Python package runner from the uv toolchain.

# Install uv (includes uvx)

curl -LsSf https://astral.sh/uv/install.sh | sh

# Verify

source $HOME/.local/bin/env
uv --version
uvx --help

Step 3: Configure EKS MCP Server

Add the EKS MCP server to your Amazon Q configuration at ~/.aws/amazonq/mcp.json

{
  "mcpServers": {
    "eks": {
      "command": "uvx",
      "args": [
        "awslabs.aws-eks-mcp-server@latest"
      ],
      "env": {
        "AWS_REGION": "us-east-2",
        "KUBECONFIG": "/home/ubuntu/.kube/config",
        "FASTMCP_LOG_LEVEL": "ERROR"
      }
    }
  }
}

Note: Set AWS_REGION to your EKS cluster region. The server uses your existing kubeconfig and AWS credentials.

Step 4: Verify Connection

# Start Q chat

q chat

# Ask Q to check cluster access

> List all my EKS clusters and their status

Amazon Q response:

Using EKS MCP tool: list_clusters

Cluster: production-app
├── Status: ACTIVE
├── Version: 1.29
├── Region: us-east-2
├── Nodes: 6 (3 on-demand, 3 spot)
└── Endpoint: https://xxxxx.gr7.us-east-2.eks.amazonaws.com

Step 5: Production Troubleshooting Scenarios

Scenario 1: Pod CrashLoopBackOff

You:

"My payment-service pods are in CrashLoopBackOff in the production namespace. Find out why and suggest a fix."

Amazon Q:

Investigating payment-service in namespace: production

1. Pod Status:
   payment-service-7d4f8b-x2k9p  CrashLoopBackOff  (5 restarts)
   payment-service-7d4f8b-m3n1q  CrashLoopBackOff  (5 restarts)

2. Last Container Logs:
   Error: FATAL - Cannot connect to database
   Connection refused: postgres-primary.database.svc:5432
   
3. Events:
   Warning  BackOff  Restarting container (exit code 1)

4. Root Cause:
   The postgres-primary service in the database namespace
   is not running. The payment-service can't start without
   its database connection.

5. Suggested Fix:
   kubectl get pods -n database -l app=postgres-primary
   kubectl describe pod postgres-primary-0 -n database

Scenario 2: High Latency / Slow Responses

You:

"The API gateway is responding slowly. P99 latency jumped from 200ms to 3s. Check what's happening."

Amazon Q:

Investigating high latency in production cluster...

1. HPA Status:
   api-gateway: 4/4 pods (max reached, CPU at 92%)
   
2. Resource Usage:
   Pod api-gateway-5f8c-k2m1: CPU 980m/1000m (98%)
   Pod api-gateway-5f8c-n4p2: CPU 945m/1000m (94%)
   
3. Node Pressure:
   ip-10-0-1-45: MemoryPressure=True (93% used)
   
4. Root Cause:
   HPA has scaled to max replicas but pods are CPU-starved.
   Node memory pressure is causing throttling.
   
5. Recommended Actions:
   a) Increase HPA maxReplicas: 4 → 8
   b) Increase pod CPU limit: 1000m → 2000m
   c) Add a new node group or scale existing:
      aws eks update-nodegroup-config         --cluster-name production-app         --nodegroup-name workers         --scaling-config minSize=3,maxSize=10,desiredSize=6

Scenario 3: Service-to-Service Communication Failure

You:

"The order-service can't reach the inventory-service. Getting connection timeout errors. Both are in the production namespace."

Amazon Q:

Diagnosing connectivity: order-service → inventory-service

1. Service Check:
   inventory-service ClusterIP: 10.100.45.23:8080 ✓ exists
   Endpoints: 0 active endpoints ✗ PROBLEM

2. Pod Status:
   inventory-service pods: 0/3 Ready
   All pods stuck in Pending state
   
3. Events:
   0/6 nodes available: 3 insufficient memory,
   3 had taint node.kubernetes.io/disk-pressure

4. Root Cause:
   inventory-service pods can't be scheduled — nodes are
   under disk and memory pressure. The service has 0
   endpoints, causing connection timeouts from order-service.
   
5. Immediate Fix:
   # Cordon the unhealthy nodes
   kubectl cordon ip-10-0-2-67
   
   # Scale up node group
   aws eks update-nodegroup-config      --cluster-name production-app      --nodegroup-name workers      --scaling-config desiredSize=8
   
   # Once new nodes are ready, pods will schedule

Scenario 4: OOMKilled Pods

You:

"The analytics-worker keeps getting OOMKilled. Show me the memory usage pattern and recommend proper limits."

Amazon Q:

Analyzing analytics-worker memory usage...

1. Current Limits:
   requests: 256Mi | limits: 512Mi
   
2. Actual Usage (last 1h from metrics-server):
   Average: 480Mi
   Peak: 612Mi (exceeds 512Mi limit → OOMKilled)
   
3. Restart History:
   6 OOMKilled events in last 2 hours
   Pattern: spikes during batch processing (every 15m)
   
4. Recommendation:
   requests: 512Mi → guarantees scheduling
   limits: 1Gi → accommodates batch processing peaks

   kubectl patch deployment analytics-worker -n production      --type='json' -p='[
       {"op":"replace","path":"/spec/template/spec/containers/0/resources/requests/memory","value":"512Mi"},
       {"op":"replace","path":"/spec/template/spec/containers/0/resources/limits/memory","value":"1Gi"}
     ]'

Step 6: Extended MCP Config for Full Observability

Combine the EKS MCP server with CloudWatch and AWS docs for complete production troubleshooting:

{
  "mcpServers": {
    "eks": {
      "command": "uvx",
      "args": ["awslabs.aws-eks-mcp-server@latest"],
      "env": {
        "AWS_REGION": "us-east-2",
        "KUBECONFIG": "/home/ubuntu/.kube/config",
        "FASTMCP_LOG_LEVEL": "ERROR"
      }
    },
    "cloudwatch": {
      "command": "uvx",
      "args": ["awslabs.aws-cloudwatch-mcp-server@latest"],
      "env": {
        "AWS_REGION": "us-east-2",
        "FASTMCP_LOG_LEVEL": "ERROR"
      }
    },
    "aws-docs": {
      "command": "uvx",
      "args": ["awslabs.aws-documentation-mcp-server@latest"],
      "env": {
        "FASTMCP_LOG_LEVEL": "ERROR"
      }
    }
  }
}

Useful Troubleshooting Prompts

Pod Issues

"Show me all pods that have restarted more than 3 times in the last hour across all namespaces"

Resource Exhaustion

"Which nodes are running above 80% CPU or memory? Should I scale the node group?"

Deployment Rollback

"The latest deployment of user-service is failing health checks. Roll it back to the previous version and show me what changed."

Network Debugging

"Test connectivity from the frontend pod to the backend service on port 8080. Check if NetworkPolicy is blocking it."

Log Analysis

"Get the last 100 error logs from all pods in the checkout namespace and identify the most common error pattern."

Certificate Expiry

"Check if any TLS secrets in the cluster are expiring within the next 30 days."

Best Practices

Use Read-Only Access First

Configure your kubeconfig with a read-only ClusterRole for the MCP server. Only escalate to write access when you need to apply fixes — and always review before executing.

Be Specific About Namespaces

Always include the namespace in your prompts. "Check pods in production" is better than "check pods" — avoids accidentally looking at staging or dev clusters.

Combine with CloudWatch MCP

For full-stack debugging, add the CloudWatch MCP server. Ask Q to correlate pod metrics with ALB logs and RDS performance insights — all in one conversation.

Create Runbook Prompts

Save commonly used prompts as team runbooks: "Run the standard health check for the payments cluster" — consistency across on-call engineers.

How It Works

┌──────────────┐      MCP       ┌──────────────────┐
│              │◄──────────────►│  EKS MCP Server  │
│  Amazon Q    │                 │  (kubectl + AWS) │
│  Developer   │      MCP       ├──────────────────┤
│  CLI         │◄──────────────►│  CloudWatch MCP  │
│              │                 │  (logs/metrics)  │
└──────┬───────┘                 └────────┬─────────┘
       │                                  │
       │  Natural Language                │  API Calls
       │                                  │
       ▼                                  ▼
┌──────────────┐                 ┌──────────────────┐
│   Engineer   │                 │   EKS Cluster    │
│   (You)      │                 │   (Production)   │
└──────────────┘                 └──────────────────┘

Summary

Install AWS Q Developer CLI and authenticate
Configure EKS MCP server in ~/.aws/amazonq/mcp.json
Verify cluster connectivity via q chat
Describe production issues in natural language
Q runs kubectl/AWS commands and returns root cause + fix
Add CloudWatch MCP for full observability stack
Use read-only access by default, escalate when needed

All Posts Get in Touch