Kong API Gateway: Best Practices for Upstreams, Targets, and Active/Passive Health Checks
This article explores Kong API Gateway’s support for load balancing, upstreams, and targets, as well as its active and passive health check capabilities. It also delves into the timer issues that can arise with active health checks and provides best practices for configuring health checks in Kong to optimize performance and maintain stability.
Load balancing in Kong
Kong API Gateway provides built-in load balancing functionality that distributes incoming API requests among multiple backend servers. Kong allows for configuring multiple upstreams and targets, enabling horizontal scaling and failover. Additionally, Kong provides health checks and circuit breaker features to ensure service availability and fault tolerance in a distributed system.
Kong supports several load balancing strategies, including
- Consistent-hashing: This load balancing strategy uses a hash function to map requests to a specific backend server. Kong uses the hash_on parameter to determine which part of the request to hash (e.g., a specific header or a custom value), and the hash_fallback parameter to specify a fallback behavior if the hash value doesn’t match any available backend servers.
- Round-robin: This load balancing strategy distributes requests evenly among available backend servers in a circular manner, so each server receives an equal share of requests.
- Least-connections: This load balancing strategy routes requests to the backend server with the fewest active connections. This helps to ensure that the server with the least load receives the next incoming request.
Upstreams and Targets
In Kong API Gateway, upstreams and targets are used together to route incoming API requests to the appropriate backend servers.
An upstream represents a group of backend servers that can handle requests for a specific API or set of APIs. An upstream defines how Kong should load balance requests among its associated targets.
A target represents an individual backend server within an upstream. Kong will use the configured load balancing algorithm to distribute incoming requests across the targets within the upstream.
_format_version: "3.0"
_transform: true
plugins:
- name: rate-limiting
service: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
config:
second: 20
minute: 20
hour: 20
policy: local
services:
- id: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
name: login-service
host: login-service.upstream
path: /
port: 80
protocol: http
routes:
- name: all-paths
id: 4d00558f-9c29-4c00-bc28-eaf8a7b3c7fd
service: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
paths:
- /
protocols:
- http
strip_path: false
upstreams:
- algorithm: round-robin
id: ccaf7511-37f6-458a-a676-cdf2d4f93dee
name: login-service.upstream
targets:
- id: 1cb2af1f-454e-5205-a31e-c9a73ebc3fe2
target: 10.130.12.43:8080
upstream: ccaf7511-37f6-458a-a676-cdf2d4f93dee
weight: 50
- id: 1122a7b2-86bd-4a1b-a712-45a8b0052860
target: 10.130.12.164:8080
upstream: ccaf7511-37f6-458a-a676-cdf2d4f93dee
weight: 50When to use upstreams and targets in Kong:
- Multiple backend servers: If you have multiple backend servers that can handle requests for a particular API or set of APIs, you can define them as targets within an upstream. This allows Kong to distribute the incoming requests across the targets.
- Load balancing: If you want to distribute incoming API requests across multiple backend servers, you can define an upstream with multiple targets. Kong provides several load balancing algorithms, such as round-robin, consistent hashing, and ring-balancer, to distribute the requests among the targets in the upstream.
- High availability and fault tolerance: If you have multiple instances of a backend service for high availability and fault tolerance, you can define them as targets within an upstream. Kong can then automatically reroute requests to other targets if a particular target becomes unavailable.
- Scaling: If you need to scale the backend service horizontally by adding more servers, you can add them as targets within the upstream. Kong can then distribute requests across all targets in the upstream, helping to evenly balance the traffic across all servers.
Upstream health checks
Upstream health checks in Kong periodically check the health of backend servers in a specified upstream group to ensure that only healthy servers receive incoming requests from the API Gateway.
Kong supports active and passive health checks that can be used to monitor the health of backend servers.
Active health checks involve Kong actively sending requests to the backend servers to check their health status. In this type of health check, Kong acts as a client and sends regular requests to the backend servers, expecting a valid response within a specific timeframe. If the response is not received or is invalid, Kong marks the server as unhealthy and stops sending requests to it.
_format_version: "3.0"
_transform: true
plugins:
- name: rate-limiting
service: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
config:
second: 20
minute: 20
hour: 20
policy: local
services:
- id: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
name: login-service
host: login-service.upstream
path: /
port: 80
protocol: http
routes:
- name: go-service-all
id: 4d00558f-9c29-4c00-bc28-eaf8a7b3c7fd
service: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
paths:
- /
protocols:
- http
strip_path: false
upstreams:
- algorithm: round-robin
id: ccaf7511-37f6-458a-a676-cdf2d4f93dee
name: login-service.upstream
healthchecks:
active:
http_path: /health
healthy:
http_statuses:
- 200
successes: 3
interval: 10
unhealthy:
http_failures: 5
tcp_failures: 5
timeouts: 5
http_statuses:
- 429
- 404
- 500
- 501
- 502
- 503
- 504
- 505
interval: 5
targets:
- id: 1cb2af1f-454e-5205-a31e-c9a73ebc3fe2
target: 10.130.12.43:8080
upstream: ccaf7511-37f6-458a-a676-cdf2d4f93dee
weight: 50
- id: 1122a7b2-86bd-4a1b-a712-45a8b0052860
target: 10.130.12.164:8080
upstream: ccaf7511-37f6-458a-a676-cdf2d4f93dee
weight: 50Passive health checks, on the other hand, rely on Kong monitoring the server’s behavior and status through external sources, such as logs or system metrics. Kong continuously monitors the logs and metrics of the backend servers and marks them as unhealthy if they exceed specific thresholds or criteria. Passive health checks do not require Kong to send requests to the servers actively.
_format_version: "3.0"
_transform: true
plugins:
- name: rate-limiting
service: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
config:
second: 20
minute: 20
hour: 20
policy: local
services:
- id: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
name: login-service
host: login-service.upstream
path: /
port: 80
protocol: http
routes:
- name: go-service-all
id: 4d00558f-9c29-4c00-bc28-eaf8a7b3c7fd
service: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
paths:
- /
protocols:
- http
strip_path: false
upstreams:
- algorithm: round-robin
id: ccaf7511-37f6-458a-a676-cdf2d4f93dee
name: login-service.upstream
healthchecks:
passive:
healthy:
http_statuses:
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
successes: 3
type: http
unhealthy:
http_failures: 5
http_statuses:
- 429
- 404
- 500
- 501
- 502
- 503
- 504
- 505
tcp_failures: 5
timeouts: 5
targets:
- id: 1cb2af1f-454e-5205-a31e-c9a73ebc3fe2
target: 10.130.12.43:8080
upstream: ccaf7511-37f6-458a-a676-cdf2d4f93dee
weight: 50
- id: 1122a7b2-86bd-4a1b-a712-45a8b0052860
target: 10.130.12.164:8080
upstream: ccaf7511-37f6-458a-a676-cdf2d4f93dee
weight: 50Problem with active health checks
Kong uses timers to manage health requests and responses, which can create pressure on the timer system when there are too many upstreams to monitor. Each active health check requires a timer to be set, and if there are many timers set, it can affect the overall performance of the system. Additionally, if there are many requests to monitor, it can increase the chances of false positives or false negatives, which can lead to incorrect health status reporting and downstream issues.
Learn more above Kong timer system from my article Tick Tock Woes — Tackling Timer Troubles in Kong Production
Another problem with active health checks in Kong is that they can overwhelm the upstream service with too many health check requests. If you have several Kong nodes running, each node will independently generate its own health check requests, which can quickly add up and saturate the upstream service. This can cause significant performance issues for the upstream service, leading to slower response times or even service downtime.
Hybrid health check
Kong offers a way to combine both active and passive health checks to provide a more efficient and reliable way to monitor the health of backend servers. This approach reduces the reliance on timers and provides a more scalable solution for larger environments. In this hybrid approach, the passive health check or circuit breaker monitors the response codes of requests to a service and determines whether the service is healthy or not. If the actual requests record errors, the target is marked as unhealthy. Once the target is marked as unhealthy, the active.unhealthy checkers begin making API calls to the targets until the service becomes healthy again.
For example, you can configure a circuit breaker to monitor the response codes of requests to a service and mark the service as unhealthy if it upstream returns certain response codes. Once the service is marked as unhealthy, active health checkers starts making API calls to the targets to verify their health status. If the service becomes healthy again, the active health checkers are stopped until the service is marked as unhealthy by the passive health checker.
_format_version: "3.0"
_transform: true
plugins:
- name: rate-limiting
service: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
config:
second: 20
minute: 20
hour: 20
policy: local
services:
- id: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
name: login-service
host: login-service.upstream
path: /
port: 80
protocol: http
routes:
- name: go-service-all
id: 4d00558f-9c29-4c00-bc28-eaf8a7b3c7fd
service: b0fbe2ea-49b9-41a8-98d5-1d770f9a7d54
paths:
- /
protocols:
- http
strip_path: false
upstreams:
- algorithm: round-robin
id: ccaf7511-37f6-458a-a676-cdf2d4f93dee
name: login-service.upstream
healthchecks:
active:
http_path: /health
healthy:
http_statuses:
- 200
successes: 3
interval: 0
unhealthy:
interval: 5
passive:
healthy:
http_statuses:
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
successes: 3
type: http
unhealthy:
http_failures: 5
http_statuses:
- 429
- 404
- 500
- 501
- 502
- 503
- 504
- 505
tcp_failures: 5
timeouts: 5
targets:
- id: 1cb2af1f-454e-5205-a31e-c9a73ebc3fe2
target: 10.130.12.43:8080
upstream: ccaf7511-37f6-458a-a676-cdf2d4f93dee
weight: 50
- id: 1122a7b2-86bd-4a1b-a712-45a8b0052860
target: 10.130.12.164:8080
upstream: ccaf7511-37f6-458a-a676-cdf2d4f93dee
weight: 50The hybrid approach of combining active and passive health checks in Kong API Gateway provides several benefits:
- More efficient and reliable monitoring: Passive health checks are continuously monitoring the response codes of requests to a service, which reduces the reliance on timers and provides more accurate information about the health of the backend servers. This allows for a faster response time and reduces the risk of false positives or false negatives.
- Scalability: The hybrid approach is more scalable and can handle larger environments with a large number of backend servers. This is because passive health checks do not require additional API calls or resources, which can put a strain on the system.
- Faster recovery time: Once a backend server is marked as unhealthy by the passive health checker, the active health checkers start making API calls to the targets until the service becomes healthy again. This means that the recovery time is faster as the active health checkers are already monitoring the backend servers.
Overall, the hybrid approach of combining active and passive health checks in Kong API Gateway provides a more efficient, reliable, and scalable way to monitor the health of backend servers, reducing the risk of downtime and improving the overall performance of the system.
My Other Kong Blogs
Tick Tock Woes — Tackling Timer Troubles in Kong Production
KongAPI Gateway Behind the Scenes: Overcoming Reliability Challenges
GitOps Approch to Configuration Management In Kong DBless Mode
