Why Cilium Replaces kube-proxy
Our checkout frontend needs to ask the payments service whether order 42 has been paid. The code makes an ordinary HTTP call. Underneath, Kubernetes must find a payments Pod, get the request there, and return the response without making the frontend track changing Pod addresses.
That works with kube-proxy. So why do teams replace it with Cilium?
This Cilium vs kube-proxy walkthrough follows one connection, then asks what changes when the same cluster runs thousands of Services while Pods keep starting, stopping, and moving. We will compare kube-proxy's iptables mode with Cilium's eBPF Service handling, including where each fits and what to measure before switching.
How a Kubernetes Service request reaches a Pod
The frontend runs on Node 1. The selected payments Pod runs on Node 2. Both belong to the shop namespace.
A Pod runs an instance of our application. A node is the machine hosting Pods. A Service gives a group of Pods a stable name and virtual IP. The backend is simply the Pod chosen to handle this connection.
| Component | Example address | Why we need it |
|---|---|---|
| Frontend Pod, Node 1 | 10.244.1.7 | Runs the client making the request |
payments-svc, a ClusterIP Service | 10.96.0.10:80 | Gives clients a stable address as backend Pods change |
| Payments Pod, Node 2 | 10.244.3.4:8080 | Runs the payments application |
curl http://payments-svc:80/payments/42
Start with kube-proxy in iptables mode. iptables is a tool for configuring packet-handling rules in Linux. Assume network policy allows the traffic and the frontend's source IP can stay unchanged, so no source NAT is needed.
- The frontend calls
payments-svc:80. It knows the Service name, not the payments Pod's location. - DNS resolves the name. On a cache miss, the resolver asks CoreDNS and gets
10.96.0.10. CoreDNS provides discovery; it does not forward the HTTP request. - The frontend opens a TCP connection to
10.96.0.10:80. This is a virtual Service address, not another Pod or a separate proxy server. - Node 1's kernel matches the Service rules. The kernel is the operating system code that handles networking. kube-proxy previously installed these iptables rules by watching Services and EndpointSlices. An EndpointSlice records a Service's backend addresses. kube-proxy itself does not relay this packet.
- The rules select
10.244.3.4:8080. Destination NAT, or DNAT, changes the destination IP and port from10.96.0.10:80to10.244.3.4:8080. - The Pod network carries it to Node 2. A Container Network Interface (CNI) implementation, such as Calico or Cilium, sets up Pod networking. Kernel routes carry the packet across nodes. Some clusters wrap it in an extra network packet, called an overlay tunnel, to cross the underlying network.
- Node 2 delivers it to the payments Pod. The application is listening on port
8080. - The application handles
/payments/42. It returns{"status":"paid"}. Neither DNS nor kube-proxy executes this endpoint. - The reply reaches the frontend. Connection tracking, usually called conntrack, remembers which connection used which address translation. On Node 1, that state lets the kernel reverse the translation so the reply appears to come from
10.96.0.10:80. The physical route need not retrace every network link.
If the frontend uses the temporary source port 53000, the packet addresses look like this. Outer tunnel headers, if any, are omitted.
| Observation point | Source | Destination |
|---|---|---|
| Frontend sends | 10.244.1.7:53000 | 10.96.0.10:80 |
| After Node 1 applies DNAT | 10.244.1.7:53000 | 10.244.3.4:8080 |
| Payments Pod replies | 10.244.3.4:8080 | 10.244.1.7:53000 |
| After Node 1 applies reverse NAT | 10.96.0.10:80 | 10.244.1.7:53000 |
The Service networking documentation explains this kernel forwarding model. Backend selection normally belongs to the TCP connection. Several HTTP requests over one persistent connection can therefore reach the same Pod.
How kube-proxy iptables scales with 1,000 microservices
Our checkout system starts with frontend, payments, and a few other applications. Later we add inventory, shipping, search, recommendations, and services owned by other teams. Each needs a stable destination, and most run more than one Pod.
Assume each microservice has one ClusterIP Service, one TCP port, and three Pod replicas. A ready Pod IP and port is a backend endpoint.
| Microservices | Service destinations | Backend endpoints to track |
|---|---|---|
| 10 | 10 | 30 |
| 100 | 100 | 300 |
| 1,000 | 1,000 | 3,000 |
N | N | 3 x N |
These are counts for our example, not benchmarks or exact iptables rule counts. A Service and an endpoint can each require several rules. Multiple Service ports add more state. Each node running kube-proxy normally needs the relevant Service information, including backends on other nodes.
A Service lookup is work the kernel has to do
Think of the iptables Service chain as a series of tests. For a new connection, the kernel checks rules until it finds the matching Service, then follows that Service's backend-selection rules. A simplified view might look like this:
| Rule in the example chain | Match this destination | If it matches |
|---|---|---|
| Authentication Service | 10.96.0.2:80 | Follow the authentication backend rules |
| Inventory Service | 10.96.0.3:80 | Follow the inventory backend rules |
| Payments Service | 10.96.0.10:80 | Follow the payments backend rules |
| Other Services | Other Service IPs and ports | Follow their backend rules |
This is a teaching view, not a literal iptables-save dump. Services do not have to appear in this order. The point is that adding Services and replicas adds rule state, and traversing chains can cost more as they grow. It also becomes harder for an engineer to inspect thousands of generated rules while debugging one failed payments request.
Existing connections normally reuse conntrack state, so every packet does not repeat the full selection. More microservices also do not automatically mean more latency for every request. The pressure depends on the rule layout, connection rate, and workload.
Deployments keep changing that state
Now payments deploys a new version while inventory scales up. Pods get new IPs, become ready, or stop accepting new work. EndpointSlices change, and each node's kube-proxy must update its rules. With many teams deploying at once, keeping forwarding state current becomes a second source of work.
I would look for slower new connections, high node CPU during rollouts, and delays between an endpoint change and nodes using it. Counting microservices alone does not tell us whether the cluster has a networking bottleneck.
kube-proxy is distributed across nodes, so this is not one central proxy becoming overloaded. Modern iptables mode also supports incremental updates. Claims that every Pod change always rewrites the entire rule set describe older behavior. The Kubernetes performance notes explain these improvements.
How Cilium eBPF replaces kube-proxy Service handling
eBPF lets Linux run small programs at specific points in the kernel. A verifier checks these programs against safety rules before they can run. A hook is an attachment point, such as an application opening a connection. Cilium is the networking system that uses these programs. Installing Cilium for Pod networking and enabling its kube-proxy replacement are separate configuration choices.
With replacement enabled, the Cilium agent watches Kubernetes state and maintains Service and backend information in eBPF maps. A map is a data structure that a program can look up using a key. Here, the key can identify a Service by its IP, port, and protocol, such as 10.96.0.10, 80, and TCP.
The program looks up that Service and finds its backend information instead of walking an iptables Service chain. With 1,000 Services, Cilium still needs those Services and their endpoints in memory, and still needs to update them when Pods change. It changes how the information is stored and searched; it does not make the state disappear.
For our frontend's TCP connection, Cilium's socket load balancer can act when the application calls connect(), the operating system call that opens the connection. The application requests 10.96.0.10:80; the socket is connected to the selected 10.244.3.4:8080 backend. Packets can then leave toward the Pod directly, without an additional Service DNAT operation in the lower packet path.
That means steps 4 and 5 happen earlier than in the iptables example. Cilium can also handle Services in the packet path when socket load balancing is bypassed or does not apply. Its socket load balancer documentation describes this distinction.
CoreDNS, the Service, EndpointSlices, and cross-node Pod networking still have jobs. Cilium can provide both Pod networking and Service handling. It also supports network policies that control which applications may communicate. Its optional Hubble component helps us see who called whom and which traffic was dropped. Those are additional reasons a platform team may choose it, beyond replacing Service lookup. See the Cilium overview and Hubble documentation.
When to choose Cilium over kube-proxy
For the checkout platform, I would evaluate Cilium when deployments create measurable Service update lag, new connections consume significant node CPU, or we also need its policy and flow visibility. I would test the actual workload before expecting a latency improvement.
| Situation | Architecture decision |
|---|---|
| Small, stable cluster meeting its latency targets | Keeping kube-proxy may be the simpler choice |
| Large Service set with frequent endpoint changes | Compare Cilium replacement against the current datapath under rollout load |
| iptables is the main limitation | Also evaluate kube-proxy's nftables mode on a supported platform |
| Need integrated Pod networking, policy, and flow visibility | Cilium can address several platform requirements together |
The nftables backend, which uses a newer Linux packet-rule system, improves kube-proxy's scalability too. An iptables comparison alone does not prove that replacing kube-proxy is necessary.
In a staging test, I would measure new-connection p99 latency (the time within which 99% of new connections finish establishing), node networking CPU, failed connections, and how quickly nodes pick up endpoint changes during a rollout. I would also check kernel support, map capacity, policy behavior, and recovery after a node restart. Changing Service implementations can disrupt existing connections, so migration needs its own rollout plan.
The frontend should still call payments-svc:80 after the change. The architectural benefit is a different implementation underneath that stable contract: eBPF-based Service selection, combined with the networking and visibility features the team needs.
Related reading
- Kubernetes networking notes collect the request flows and networking architecture decisions in this series.
- LLM serving on Kubernetes follows an inference request through gateways, replica selection, and GPU model servers. Those application-level routing decisions sit above the Pod network described here.
References
About the author
Ritesh Yadav works as an AI/ML Engineer. He writes independent research notes on ML performance, infrastructure, and systems, covering CUDA, low-latency inference, generative AI, distributed training, Kubernetes, and LLMOps.