Why Your ECS Service Isn't Routing Traffic: Lessons from a Target Group Registration Issue

by Ilya Verbitskiy

Introduction

Recently, I ran into an interesting issue while deploying a new container to an Amazon ECS service using our CI/CD pipeline. Everything looked great on the surface—the pipeline executed successfully, the ECS service picked up the new task definition, and the container was reported as healthy.

But despite all signs pointing to a successful deployment, something was off: our application wasn't receiving any traffic.

This blog post walks through the symptoms, root cause, and resolution—focusing on the crucial step of registering a target group with an ECS service.

Problem Recap

Here's what happened step-by-step:

  1. Our CI/CD pipeline deployed a new container image to ECS using a Fargate service.
  2. ECS successfully launched a new task using the updated task definition.
  3. The task entered a RUNNING state and passed its health checks.
  4. However, the application behind the ALB (Application Load Balancer) wasn't responding.

There were no errors in the logs and no failed tasks. But when I checked the ALB Target Group, I noticed something critical: the target group health checks were failing.

Detective Work: Identifying the Root Cause

When I realized our ECS deployment wasn't routing traffic, my first thought was to confirm the service and task statuses. Using the AWS CLI, I began a step-by-step investigation.

Step 1: Verify ECS Service Health

I listed all ECS services in the cluster to confirm that my target service, Report-Service, was active:

aws ecs list-services \
  --cluster EcsClusterAAAAAAAA-000000000000

{
  "serviceArns": [
    "arn:aws:ecs:us-east-1:000000000000:service/EcsClusterAAAAAAAA-000000000000/Report-Service"
  ]
}

✅ Result: The service Report-Service was listed, confirming it exists and is managed by the cluster.

Step 2: Check Task Status

Next, I checked if the service had launched any tasks:

aws ecs list-tasks \
  --cluster EcsClusterAAAAAAAA-000000000000 \
  --service-name Report-Service

✅ Result: A task was running as expected:

{
  "taskArns": [
    "arn:aws:ecs:us-east-1:000000000000:task/EcsClusterAAAAAAAA-000000000000/cdb13d4a4fa2439584114ad90d5aab9a"
  ]
}

To be thorough, I described the task to confirm its runtime status:

aws ecs describe-tasks \
  --cluster EcsClusterAAAAAAAA-000000000000 \
  --tasks "arn:aws:ecs:us-east-1:000000000000:task/EcsClusterAAAAAAAA-000000000000/cdb13d4a4fa2439584114ad90d5aab9a" \
  --query "tasks[*].[taskArn, lastStatus]"

✅ Result: Status was RUNNING — so far, so good:

[
  [
    "arn:aws:ecs:us-east-1:000000000000:task/EcsClusterAAAAAAAA-000000000000/cdb13d4a4fa2439584114ad90d5aab9a",
    "RUNNING"
  ]
]

Step 3: Check ALB Target Group Health

Now it was time to look at the target group attached to the Application Load Balancer (ALB). First, I listed all target groups:

aws elbv2 describe-target-groups \
  --query 'TargetGroups[*].[TargetGroupArn, TargetGroupName]'

I located the target group named reports-service, which was expected to be associated with the Report-Service:

[
  [
    "arn:aws:elasticloadbalancing:us-east-1:000000000000:targetgroup/reports-service/e4869714669ce12f",
    "reports-service"
  ]
]

Then, I checked the target health status:

aws elbv2 describe-target-health \
  --target-group-arn "arn:aws:elasticloadbalancing:us-east-1:000000000000:targetgroup/reports-service/e4869714669ce12f"

❌ Result: This indicated that the task was not responding to health checks, which usually means the ALB can't reach the container on the expected port.

{
  "TargetHealthDescriptions": [
    {
      "Target": {
        "Id": "10.1.28.89",
        "Port": 8080,
        "AvailabilityZone": "us-east-1a"
      },
      "HealthCheckPort": "8080",
      "TargetHealth": {
        "State": "unhealthy",
        "Reason": "Target.Timeout",
        "Description": "Request timed out"
      },
      "AdministrativeOverride": {
        "State": "no_override",
        "Reason": "AdministrativeOverride.NoOverride",
        "Description": "No override is currently active on target"
      }
    }
  ]
}

Step 4: Is the Target Group Even Registered with ECS?

At this point, I suspected a misconfiguration—maybe the service wasn't even hooked up to the target group. To confirm, I ran:

aws ecs describe-services \
  --cluster EcsClusterAAAAAAAA-000000000000 \
  --services Report-Service \
  --query "services[0].loadBalancers" \

❌ Result: [] (an empty array)

That was the smoking gun.

Despite having a healthy ECS task and a defined target group, the ECS service wasn't actually registered with the target group at all. As a result, the task wasn't formally added as a target behind the load balancer.

Fixing the Issue: Registering the ECS Service with a Target Group

Once I realized that the ECS service wasn't associated with any load balancer target group, I knew exactly what had to be done: manually register the service with the correct target group. Here's how I did it, step by step.

Step 1: Identify the Container Name

aws ecs describe-tasks \
  --cluster EcsClusterAAAAAAAA-000000000000 \
  --tasks "arn:aws:ecs:us-east-1:000000000000:task/EcsClusterAAAAAAAA-000000000000/cdb13d4a4fa2439584114ad90d5aab9a" \
  --query "tasks[*].containers[*].[name]"

✅ Result:

[[["report-service"]]]

The container name is report-service.

Step 2: Determine the Container Port

Since the ECS service is running on Fargate, the container port (used by the ALB) must be retrieved from the task definition:

First, get the task definition ARN:

aws ecs describe-tasks \
  --cluster EcsClusterAAAAAAAA-000000000000 \
  --tasks "arn:aws:ecs:us-east-1:000000000000:task/EcsClusterAAAAAAAA-000000000000/cdb13d4a4fa2439584114ad90d5aab9a" \
  --query "tasks[*].taskDefinitionArn"

✅ Result:

["arn:aws:ecs:us-east-1:000000000000:task-definition/Report-Service-Task:5"]

Now, describe the task definition:

aws ecs describe-task-definition \
  --task-definition "arn:aws:ecs:us-east-1:000000000000:task-definition/Report-Service-Task:5" \
  --query "taskDefinition.containerDefinitions[*].[name, portMappings[*].containerPort]"

✅ Result:

[["report-service", [8080]]]

So the container listens on port 8080.

Step 3: Register the Target Group with the ECS Service

With the targetGroupArn, containerName, and containerPort in hand, I was ready to associate the target group with the ECS service:

aws ecs update-service \
  --cluster EcsClusterAAAAAAAA-000000000000 \
  --service Report-Service \
  --load-balancers "[
    {\"targetGroupArn\": \"arn:aws:elasticloadbalancing:us-east-1:000000000000:targetgroup/reports-service/e4869714669ce12f\", \"containerName\": \"report-service\", \"containerPort\": 8080}
  ]"

⚠️ Tip: Be very careful with JSON formatting in the --load-balancers argument. Even minor formatting issues—like a stray space before the ARN—can result in cryptic errors such as:

An error occurred (InvalidParameterException) when calling the UpdateService operation:
Unable to assume role and validate the specified targetGroupArn.
Please verify that the ECS service role being passed has the proper permissions.

This error isn't always clear, so make sure:

  • The ARN is exact (no extra spaces).
  • The IAM service role has permissions to register targets with the load balancer.

Step 4: Confirm the Fix

Once the service was updated, I re-checked the target group health:

aws elbv2 describe-target-health \
  --target-group-arn "arn:aws:elasticloadbalancing:us-east-1:000000000000:targetgroup/reports-service/e4869714669ce12f"

This time, the target transitioned from unhealthy to healthy, and traffic started flowing as expected.

This debugging experience was a great reminder that a successful ECS deployment doesn't always mean a functional application. Even when tasks are running and marked healthy, traffic won't flow unless the ECS service is correctly registered with a target group.

The key takeaways are:

  • Always verify that your ECS service includes a loadBalancers configuration when using an ALB.
  • Ensure the correct containerName and containerPort are specified based on the task definition.
  • Check target group health status directly—ECS won't warn you if your service isn't properly attached.
  • Be meticulous with AWS CLI JSON formatting—small errors can trigger misleading messages.

Being thorough with these checks can save hours of head-scratching and get your applications routing traffic as expected.