Author: Jacob Mammoliti


Blog on Authentication Policy and auto mTLS in Istio 1.5

There have been some significant changes in the Istio 1.5 release, the biggest one being the consolidation of Citadel, Galley, Pilot, and the sidecar injector all into one workload called istiod, which now runs as a single pod.

The other change I’ve seen is how we enforce security and mTLS into the mesh. Prior to 1.5, we used Istio’s meshpolicy CRD to enable mTLS globally in the cluster. We then used the policy CRD to further customize our mesh, for example, allow both plaintext and mTLS traffic to all services in a specific namespace.

Istio 1.5 brings the concept of PeerAuthentication, which is a CRD that allows us to enable and configure mTLS at both the cluster level and namespace level. Anything we want to apply globally on the mesh, must live in the same namespace Istio lives in (in most cases that’s in istio-system). From reading the docs, this CRD was to move the focus of forcing policy at the service level and move the focus to the workload level.

I am using the auto mTLS feature in the following cluster, which allows me to not have to define any destination rules.

Enabling the Mesh Wide Policy

Diving right into things, if we want to enable mTLS globally across the cluster and enforce all services to send encrypted traffic, we can define the following policy:

$ kubectl apply -f - <<EOF
apiVersion: "security.istio.io/v1beta1"
kind: "PeerAuthentication"
metadata:
  name: "default"
  namespace: "istio-system"
spec:
  mtls:
    mode: STRICT
EOF
peerauthentication.security.istio.io/default created

Let’s verify that this is policy is working. I will do that by creating a sample application in a namespace that is Istio enabled, meaning any pod I deploy in that namespace will get an envoy sidecar injected into it to intercept all traffic intended for that pod. The envoy sidecar is what will verify the traffic is receiving is encrypted. I will then be using a debug pod to curl against my sample app to verify Istio is enforcing mTLS traffic.

$ kubectl get pods,svc -n beer
NAME                   READY   STATUS    RESTARTS   AGE
beer-796b7fbcd-swcjx   2/2     Running   0          46m

NAME           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/beer   ClusterIP   10.128.232.152   <none>        8080/TCP   48m

# Create a pod that is not in the Istio mesh
$ kubectl run -i --tty --rm debug \
  --image=curlimages/curl:7.68.0 --restart=Never -- sh
If you don't see a command prompt, try pressing enter.

# Try to cURL the mesh service
/ $ curl -I http://beer.beer.svc.cluster.local:8080
curl: (56) Recv failure: Connection reset by peer

When we try to cURL the service, we get a Connection reset by peer which is what we expected. Envoy is blocking that traffic because we are sending unencrypted traffic to it, which means the policy is working. Now how do we get a successful connection? In order to have applications communicate over mutual TLS, they need to be onboarded onto the mesh, meaning the namespace they live in must be enabled with Istio injection, so let’s do that now.

$ kubectl create ns sandbox 
namespace/sandbox created

$ kubectl label namespace sandbox istio-injection=enabled
namespace/sandbox labeled

$ kubectl -n sandbox run -i --tty --rm debug \
  --image=curlimages/curl:7.68.0 --restart=Never -- sh
If you dont see a command prompt, try pressing enter.

/ $ curl -I http://beer.beer.svc.cluster.local:8080
HTTP/1.1 200 OK
server: envoy
...

We’re now getting a 200 HTTP response back, which means we’re able to successfully connect to the service and also means that traffic between the two services is fully encrypted. Now, let’s assume we are onboarding an application and to start, we want our application to accept both plaintext and TLS traffic. The PeerAuthentication CRD gives us the ability to do this and to scope it specifically to either a namespace, a service or even a specific port. We’ll go through those three examples right now.

Scoping to a Namespace

Right now, we are enforcing mTLS across the mesh. Still, in the event we want to also allow plaintext traffic to a specific namespace we can configure that by creating an additional policy as shown below in the application’s namespace. We then run another debug pod that is not in the mesh to verify we can send unencrypted traffic to that service.

# Allow both mTLS and plain text traffic into a specific namespace
$ kubectl apply -f - <<EOF
apiVersion: "security.istio.io/v1beta1"
kind: "PeerAuthentication"
metadata:
  name: "beer"
  namespace: "beer"
spec:
  mtls:
    mode: DISABLE
EOF
peerauthentication.security.istio.io/default created

$ kubectl run -i --tty --rm debug \
  --image=curlimages/curl:7.68.0 --restart=Never -- sh 
If you dont see a command prompt, try pressing enter.

/ $ curl -I http://beer.beer.svc.cluster.local:8080
HTTP/1.1 200 OK
Server: nginx/1.15.7
...

We are now seeing another 200 response, as expected.

Scoping to a Workload

What if we want to tighten up this allowance of unencrypted traffic to our applications? Instead of targeting all workloads in a specific namespace, we can target specific workloads to allow unencrypted traffic and enforce mTLS on all the others by matching the label of our application in the policy.

# Delete the previous policy defintion
$ kubectl delete peerauthentication beer -n beer
peerauthentication.security.istio.io "beer" deleted

# Only accept mTLS traffic on beer workload
$ kubectl apply -f - <<EOF
apiVersion: "security.istio.io/v1beta1"
kind: "PeerAuthentication"
metadata:
  name: "beer"
  namespace: "beer"
spec:
  selector:
    matchLabels:
      app: beer
  mtls:
    mode: DISABLE
EOF
peerauthentication.security.istio.io/beer created

$ kubectl run -i --tty --rm debug \
  --image=curlimages/curl:7.68.0 --restart=Never -- sh
If you dont see a command prompt, try pressing enter.

/ $ curl -I http://beer.beer.svc.cluster.local:8080
HTTP/1.1 200 OK
Server: nginx/1.15.7
...

We are now seeing another 200 response, as expected and if we were to try and hit another service from that debug pod, we would get a Connection reset by peer error that we saw earlier.

Scoping to a Port

Finally, we can take our scoping one step further by only allowing plaintext traffic on a specific port of a workload. For example, if the service for my application is exposing 2 ports and I don’t want to allow unencrypted traffic on both of them, I can create a policy that allows unencrypted traffic on that one port. Since we have mTLS enforced cluster-wide, mTLS will still be enforced on that other port.

# Delete the previous policy defintion
$ kubectl delete peerauthentication beer -n beer
peerauthentication.security.istio.io "beer" deleted

# Allow both plaintext traffic and mTLS on port 8080 only
$ kubectl apply -f - <<EOF
apiVersion: "security.istio.io/v1beta1"
kind: "PeerAuthentication"
metadata:
  name: "beer"
  namespace: "beer"
spec:
  selector:
    matchLabels:
      app: beer
  mtls:
    mode: STRICT
  portLevelMtls:
    8080:
      mode: DISABLE
EOF

$ kubectl run -i --tty --rm debug \
  --image=curlimages/curl:7.68.0 --restart=Never -- sh
If you dont see a command prompt, try pressing enter.

/ $ curl -I http://beer.beer.svc.cluster.local:8080
HTTP/1.1 200 OK
Server: nginx/1.15.7
...

We see another 200 HTTP response as expected, which is awesome because we have now narrowed our scope to a specific port number while still enforcing mTLS traffic everywhere else in the mesh.

Wrapping Up

Enabling mutual TLS globally in the mesh is a great way to add additional security in the cluster, and since PeerAuthencation allows us to scope so finely to either a namespace, workload, or port, it allows application on-boarding to be a smoother process because you don’t have to enforce encryption right off the bat. You can start off by disabling encryption just for that application, and then slowly enable encryption once the application is successfully on-boarded.

Have additional questions about Istio or mTLS? Reach out in the comments below, or through the chat and I’d love to have a conversation about it!

Tagged:



//comments


//blog search


//other topics