Services and EndpointSlices

Key points:

Default to ClusterIP. Expose to the outside through a Gateway or Ingress, never per-app NodePorts.
A Service is a selector. EndpointSlices hold the actual pod addresses - debug a broken Service there first.
Use headless Services only when clients need individual pods: StatefulSets, peer discovery, client-side balancing.
A Service is not a security boundary. Pods can reach any pod IP directly; only NetworkPolicy restricts traffic.
Block NodePort, ExternalName, externalIPs and public load balancers with admission policy; allow exceptions explicitly.

The types

Each type builds on the one before it: a LoadBalancer Service is also a NodePort and a ClusterIP Service.

Type	What you get	Use for	Avoid when
ClusterIP	A virtual IP and DNS name inside the cluster	Almost everything - the default	Never; it’s the base
NodePort	ClusterIP plus a port (30000-32767) open on every node	Bare metal without a load balancer controller; debugging	Anything regulated or internet-facing
LoadBalancer	NodePort plus a cloud load balancer	The cluster’s entry points (Gateway, ingress), non-HTTP services needing their own IP	One per app - put apps behind a shared Gateway
Headless (`clusterIP: None`)	No virtual IP; DNS returns each ready pod’s IP	StatefulSets, databases, peer discovery, gRPC client-side balancing	Ordinary stateless apps
ExternalName	A DNS CNAME to an outside hostname	Rarely - aliasing an external dependency	HTTPS and regulated environments (see below)

How Services and EndpointSlices fit together

The Service only declares which pods (a label selector) and which ports. The EndpointSlice controller watches pods and writes their addresses into EndpointSlices. Everything that routes traffic reads the slices, not the Service.

flowchart LR
    SVC["Service<br/>selector: app=payments"] --> CTRL[EndpointSlice controller]
    PODS["Pods with app=payments<br/>and their readiness"] --> CTRL
    CTRL --> ES["EndpointSlices<br/>up to 100 endpoints each"]
    ES --> KP["kube-proxy / Cilium<br/>routes the ClusterIP"]
    ES --> DNS["CoreDNS<br/>answers headless lookups"]
    ES --> LB["Load balancer controllers<br/>in IP target mode"]

Large Services span several slices. Each holds up to 100 endpoints by default, so one pod change rewrites one small slice instead of a giant object.
The old Endpoints API is deprecated (Kubernetes 1.33) and truncates at 1,000 addresses. Read EndpointSlices in scripts and tools.
Every endpoint carries conditions:

Condition	Means
`serving`	The pod passes readiness and can take traffic
`terminating`	The pod is shutting down
`ready`	`serving` and not `terminating` - what proxies normally route to

During a rollout, if every endpoint is terminating, proxies fall back to ones that are both serving and terminating, so traffic isn’t dropped.

Debug any Service in this order:

kubectl get endpointslices -l kubernetes.io/service-name=payments -o wide   # any addresses at all?
kubectl get pods -l app=payments --show-labels                               # does the selector match?
kubectl describe pod payments-xyz | grep -A3 Readiness                       # failing readiness empties the slice

Headless Services

DNS returns one A record per ready pod, and the client picks. There’s no virtual IP and no proxying.

StatefulSets get stable per-pod names: postgres-0.postgres.db.svc.cluster.local.
publishNotReadyAddresses: true publishes pods before they’re ready - needed when cluster members must find each other to become ready (quorum systems).
Clients own the balancing. gRPC with client-side balancing can spread load across pods where a ClusterIP would pin each connection to one.

Pointing a Service at something outside the cluster

Use a selectorless Service with a manual EndpointSlice instead of ExternalName. Clients keep a normal in-cluster name, and the target is an IP you control and can put in NetworkPolicy.

apiVersion: v1
kind: Service
metadata:
  name: legacy-db
  namespace: payments
spec:
  ports:
    - name: postgres
      port: 5432
---
apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
  name: legacy-db-1
  namespace: payments
  labels:
    kubernetes.io/service-name: legacy-db
    endpointslice.kubernetes.io/managed-by: platform-team
addressType: IPv4
ports:
  - name: postgres
    port: 5432
endpoints:
  - addresses: ["10.20.30.40"]

Traffic policies

Field	Values	Effect
`externalTrafficPolicy`	`Cluster` (default) / `Local`	`Local` keeps the client’s source IP and skips the extra node hop, but only nodes running a pod receive traffic
`internalTrafficPolicy`	`Cluster` (default) / `Local`	`Local` sends in-cluster traffic only to pods on the same node - for node agents
`trafficDistribution`	`PreferSameZone`, `PreferSameNode` (GA in 1.35)	A preference, not a guarantee - keeps traffic in-zone to cut latency and cross-zone cost
`sessionAffinity`	`None` / `ClientIP`	Pins a client IP to one pod

Compliance headaches

These are the findings auditors and pen testers raise. Enforce the controls with admission policy, not documentation.

Risk	Why it matters	Control
NodePort	Opens a port on every interface of every node, widening the network you must segment and justify	Block it; expose through a Gateway. Restrict node security groups
Public LoadBalancer	One `type: LoadBalancer` can put a service on the internet. Defaults vary: in-tree controllers, GKE and AKS default to public; the AWS Load Balancer Controller defaults to internal	Require the internal annotation; set `loadBalancerSourceRanges` on approved public ones
`externalIPs`	Anyone who can create a Service can claim any IP and intercept traffic to it (CVE-2020-8554). Deprecated in 1.36	Block it, or enable the `DenyServiceExternalIPs` admission plugin
ExternalName	A CNAME to any host: bypasses egress review and can make an in-cluster name point outside	Block it; use a selectorless Service with a reviewed EndpointSlice
Write access to EndpointSlices	Lets someone point a Service at any IP, including pods in other namespaces (CVE-2021-25740)	Never grant app teams `create`/`update` on EndpointSlices or Endpoints
Lost client IPs	With `externalTrafficPolicy: Cluster`, backends and their audit logs see node IPs, not users	`externalTrafficPolicy: Local` or PROXY protocol at the edge
Plaintext in-cluster traffic	ClusterIP traffic is unencrypted by default	mTLS through a service mesh, or node-to-node encryption
Services aren’t access control	Any pod can connect to any pod IP directly, bypassing the Service	Default-deny NetworkPolicy per namespace

Internal load balancer annotations per cloud:

Cloud	Annotation
AWS (Load Balancer Controller)	`service.beta.kubernetes.io/aws-load-balancer-scheme: internal`
GKE	`networking.gke.io/load-balancer-type: "Internal"`
AKS	`service.beta.kubernetes.io/azure-load-balancer-internal: "true"`

The policy - tested with the Kyverno CLI against ClusterIP, NodePort, ExternalName, externalIPs, public and internal load balancers. Swap the annotation for your cloud.

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: restrict-service-exposure
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE", "UPDATE"]
        resources: ["services"]
  variables:
    - name: type
      expression: "object.spec.?type.orValue('ClusterIP')"
  validations:
    - expression: "variables.type != 'NodePort'"
      message: "NodePort Services are not allowed. Expose through the Gateway."
    - expression: "variables.type != 'ExternalName'"
      message: "ExternalName Services are not allowed."
    - expression: "object.spec.?externalIPs.orValue([]).size() == 0"
      message: "spec.externalIPs is not allowed."
    - expression: >-
        variables.type != 'LoadBalancer' ||
        object.metadata.?annotations[?'service.beta.kubernetes.io/aws-load-balancer-scheme'].orValue('') == 'internal'
      message: "LoadBalancer Services must be internal."
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: restrict-service-exposure
spec:
  policyName: restrict-service-exposure
  validationActions: ["Deny"]
  matchResources:
    namespaceSelector:
      matchExpressions:
        - key: kubernetes.io/metadata.name
          operator: NotIn
          values: ["envoy-gateway-system"]   # the approved public entry point

Gotchas

A selector typo fails silently. The Service is created with zero endpoints and no error. Check kubectl get endpointslices before anything else.
Readiness empties the Service. If every pod fails readiness at once, the Service has no endpoints and every request fails. Keep readiness about the pod itself, never its dependencies.
port, targetPort and nodePort are three different things. port is what clients call, targetPort is the container’s port, nodePort is the port on the node. Use a named targetPort so container port changes don’t break the Service.
Headless clients cache stale pod IPs. Runtimes and libraries that cache DNS keep connecting to pods that are gone. Lower client DNS caching, and reconnect on failure.
Long-lived connections ignore endpoint changes. A connection opened before a scale-up stays on its old pod. Add L7 balancing or recycle connections periodically.
sessionAffinity: ClientIP collapses onto one pod behind externalTrafficPolicy: Cluster, because every client appears as a node IP. Use Local, or affinity at the load balancer.
internalTrafficPolicy: Local drops traffic on nodes with no local pod. Run the backend as a DaemonSet when you use it.
PreferSameZone can overload a small zone. If one zone has fewer pods but just as many clients, those pods take more than their share. Spread pods evenly across zones.
LoadBalancer Services allocate NodePorts too. When the load balancer targets pods directly (IP mode), set allocateLoadBalancerNodePorts: false to stop opening unused node ports.
ExternalName breaks HTTPS. Clients send the in-cluster name as the TLS SNI and HTTP Host, which the external server doesn’t recognize. Call the real hostname instead.
Fixed NodePorts collide. Hand-picked nodePort values clash across teams. Let Kubernetes allocate them, or reserve a documented range.