Author: Jacob Mammoliti


Running Consul’s Ingress Gateway in Kubernetes

Introduction

With the release of Consul 1.8, we were introduced to new features that help users connect services inside and outside of the service mesh. The two features that stood out to me in this release were the addition of the Ingress Gateway and the Terminating Gateway. In this blog, I’m going to focus on the Ingress Gateway and demonstrate how to configure it in a Kubernetes cluster.

Requirements

To follow this blog, you will need:

  • A Kubernetes cluster (I’m using Google Kubernetes Engine)
  • Consul 1.8.0+ binary installed locally

Deploying Consul on Kubernetes

I have a Terraform module that handles the deployment of a GKE cluster and the Consul cluster for me, but to level the playing field for the readers that are not using GKE, I’ll show the steps on how to use the Helm chart outside of Terraform, to deploy Consul.

First let’s pull down the Helm Chart from HashiCorp’s GitHub and create a config file called values.yaml. In this config, we enable the UI by creating a Kubernetes LoadBalancer type service and also enable Connect injection so that our services can be injected with a sidecar proxy so that it is able to communicate with other services over Consul service mesh. The final parameter is defining the Ingress Gateway, where we enable it and define it as a LoadBalancer type service again so that it gets an external IP address and we can hit it from outside of the cluster.

# clone Helm Chart
➜  ~ git clone [email protected]:hashicorp/consul-helm.git

# generate the Helm values config file
➜  ~ cat <<EOF > values.yaml
global:
    image: consul:1.8.0
server:
    replicas: 3
    bootstrapExpect: 3
ui:
    service:
        type: LoadBalancer
    enabled: true
connectInject:
    enabled: true
    default: false
ingressGateways:
    enabled: true
    defaults:
        service:
            type: LoadBalancer
    gateways:
        - name: ingress-service
EOF

Once we have the Helm chart and configuration file, the last preparation needed is to create the Consul namespace and then deploy Consul through Helm (you can see why I have this all done through Terraform).

# create a consul namespace
➜  ~ kubectl create ns consul
namespace/consul created

# use helm to template out the kubernetes manifests and apply it to the cluster
➜  ~ helm template consul consul-helm -f values.yaml -n consul | kubectl apply -f -
...
pod/consul-consul-test created

We can verify that in fact our Kubernetes pods and services have come up successfully.

# verify the consul pods are all running
➜  ~ kubectl get pods -n consul
NAME                                         READY   STATUS      RESTARTS   AGE
consul-consul-c8sdb                          1/1     Running     0          94s
consul-consul-connect-injector-...           1/1     Running     0          94s
consul-consul-ingress-service-...            1/2     Running     0          94s
consul-consul-ingress-service-...            2/2     Running     0          94s
consul-consul-mhsxv                          1/1     Running     0          94s
consul-consul-nxpk7                          1/1     Running     0          94s
consul-consul-server-0                       1/1     Running     0          94s
consul-consul-server-1                       1/1     Running     0          94s
consul-consul-server-2                       1/1     Running     0          93s
consul-consul-test                           0/1     Completed   0          93s

# verify the consul services are up and we have two exernal IP addresses
# (I've redacted some data for readability)
➜  ~ kubectl get svc -n consul
NAME                                TYPE          EXTERNAL-IP     PORT(S)
consul-consul-connect-injector-svc  ClusterIP     <none>          443/TCP
consul-consul-dns                   ClusterIP     <none>          53/TCP,53/UDP
consul-consul-ingress-service       LoadBalancer  35.225.120.235  8080:32652/TCP,...
consul-consul-server                ClusterIP     <none>          8500/TCP,...
consul-consul-ui                    LoadBalancer  35.192.169.5    80:30600/TCP

Deploying our Application

Now that Consul is up and running, let’s deploy a sample application so that our Ingress Gateway can direct traffic to something. I’ll use my Pokedex application in this demo which can be found here. To deploy the application we apply the Kubernetes deployment config against our cluster. If you look at the deployment YAML, you will notice the Consul specific annotation which will tell Consul to register to app to it’s service registry and inject the application pod with a sidecar container so that it can be on-boarded to the Consul service mesh. I won’t dive much deeper into service mesh in this blog, but if you are interested to learn more, reach out!

# deploy the pokedex application
➜  ~ kubectl apply -f https://raw.githubusercontent.com/arctiqjacob/consul-kubernetes-demo/master/kubernetes/pokedex_simple.yaml
namespace/pokedex created
deployment.apps/pokedex created

# ensure the pokedex application is successfully deployed
➜  ~ kubectl get pods -n pokedex
NAME                       READY   STATUS    RESTARTS   AGE
pokedex-6b9654f48c-5z86t   3/3     Running   0          32s

Configure our Consul Ingress Gateway

Now that our application is up and running, let’s configure it as an upstream to our Ingress Gateway so we can allow traffic from outside of the cluster.

We will be configuring the Ingress Gateway by accessing Consul directly. At this time, there are not any Kubernetes native ways of creating this through a CRD (Custom Resource Definition) for example.

# export an environment variable with the Consul external IP
➜  ~ export CONSUL_HTTP_ADDR=http://146.148.54.11

# configure the default protocol for all sidecar proxies to be http
➜  ~ consul config write - <<EOF
Kind      = "proxy-defaults"
Name      = "global"
Config {
  protocol = "http"
}
EOF

# configure the ingress gateway to point to our pokedex service on port 8080
➜  ~ consul config write - <<EOF
Kind = "ingress-gateway"
Name = "ingress-service"

TLS {
  enabled = true
}

Listeners = [
 {
   Port = 8080
   Protocol = "http"
   Services = [
     {
       Name  = "pokedex"
       Hosts = ["pokedex.ingress.dc1.consul"]
     }
   ]
 }
]
EOF

Now that we have configured our Ingress Gateway, let’s review what we just configured. We enabled our sidecar proxies default protocol to HTTP and then configured our gateway to enable TLS, listen on port 8080, and direct traffic to the pokedex service when we hit our gateway through pokedex.ingress.dc1.consul. To learn more about the ingress-gateway config kind, you can find the documentation here.

Verifying our Ingress Gateway

Let’s have a look at our Ingress Gateway in the UI to ensure everything is set up correctly. When we hit Consul’s main page, we will see the consul service itself as well as our ingress-service representing our Ingress Gateway and then the application.

services

If we click on the ingress-gateway, we will see our pokedex service under the Upstreams tab. Looking good so far. The final step is to cURL against our gateway which can be seen below.

upstream

Note: I added a record for pokedex.ingress.dc1.consul in my /etc/hosts so that my computer can resolve that domain to my Ingress Gateway External IP. Remember, it’s necessary to hit the gateway through a defined Host in order to be directed to the correct service. In a more real life situation, you would create a DNS entry here and enter that as your Host.

# grab the root CA from Consul that signed the ingress gateway certificate 
➜  ~ curl 35.192.169.5/v1/connect/ca/roots | jq -r '.Roots[0].RootCert' > root.cert

# cURL against the Ingress Gateway to reach the pokedex service externally from the cluster
➜  ~ curl --cacert root.cert \
     https://pokedex.ingress.dc1.consul:8080/v1/api/pokemon/charmander | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   164  100   164    0     0    395      0 --:--:-- --:--:-- --:--:--   396
{
  "name": "charmander",
  "weight": 85,
  "height": 6,
  "order": 5,
  "sprites": {
    "front_default": "https://raw.githubusercontent.com/PokeAPI/sprites/master/sprites/pokemon/4.png"
  }
}

We can successfully cURL against our gateway, and have successfully exposed our service to the outside world through the Consul Ingress Gateway!

Expanding on this Exercise

This blog has been an introduction on how to use the Consul Ingress Gateway with Kubernetes. There are many additional features that can be added to this to get it in a more “production ready” state, but I wanted to try and keep it focused on the Ingress Gateway for this exercise. Some areas of these areas of expansion include:

  • Tightening up service mesh communication with Intentions
  • Deploy in an environment with Consul ACLs enabled
  • Use a custom domain for the Ingress Gateways Host entry

If you are interested in learning more about Consul on Kubernetes or Consul in general, let me know in the comments below and I’d love to have a conversation about it!

Tagged:



//comments


//blog search


//other topics