After the initial excitement of successfully creating your first Anthos cluster and experimenting with all of the powerful tools that come with Anthos, you might ask: how can I productionize this shiny platform and let other teams collaborate on it?
The first thing you’ll need to work on will be letting others access the Anthos platform. You might be accessing your Anthos cluster using the almighty kubeconfig, but when promoting to production, you would like your team members to put their real names on the sign-in sheet and so that they can only do what they are allowed to. Thus here comes the two big As: Authentication and Authorization. In this blog, we will cover how to leverage existing OIDC provider to handle Anthos authentication and use RBAC for authorization.
OpenID Connect (OIDC)
OpenID Connect is a simple identity layer on top of the OAuth 2.0 protocol. If a specific scope
openid is used when initiating the OAuth flow, an ID Token is also returned from the OAuth provider. The ID Token is a JSON Web Token (JWT). The data inside the ID Token are called claims. Claims can include information like expiration date, user ID, username, email, etc.
Most major OAuth providers (Google, Microsoft, Okta, Apple, etc.) support OIDC. Since all providers follow the standard from OpenID Connect Federation, it is easy to integrate OIDC with your application, in this case Anthos.
Configuring OAuth Provider
In your OAuth provider, register a client application (Google) / App registration (Microsoft Azure) (for other providers it might be named differently). From here you can get your OAuth provider’s information to configure your Anthos cluster. Below is an example of Microsoft Azure App Registration.
In your OAuth provider, you can also configure what information should be included in your ID Token.
Configuring Anthos Clusters with OIDC
Currently Anthos GKE on-prem supports OIDC for authenticating against user clusters. The integration has to be configured and deployed during the creation of user clusters.
OIDC configuration is included in the user cluster configuration file.
authentication: oidc: issuerURL: kubectlRedirectURL: clientID: clientSecret: username: usernamePrefix: group: groupPrefix: scopes: extraParams: deployCloudConsoleProxy: caPath:
clientSecretcan be found from your OAuth provider configuration.
groupare where you define how Anthos cluster processes the ID Token. For example, if you would like to use users’ emails as the username for Anthos cluster, configure
usernameas email, or the claim representing users’ emails.
groupPrefixcan be configured to customize usernames and group names in Anthos cluster. For example, if you are planning to use not only OIDC for authentication, you can add
oidc-as prefix to differentiate OIDC users from other users.
caPathare used to further customize the integration.
oidc section populated, the Anthos user cluster’s kube-apiserver will be created with the oidc configurations.
Anthos is also equipped with a few CLI tools to ease the OIDC login process:
- After user cluster is created, a login config file can be created with information of OIDC provider:
gkectl create-login-config --kubeconfig [USER_CLUSTER_KUBECONFIG]
- End users can use the login config file to automate the full authentication flow:
gcloud anthos auth login --cluster [CLUSTER_NAME] --login-config [LOGIN_CONFIG_FILE]
Users can also login to the Anthos GKE on-prem cluster from Google Cloud Console using OIDC. Note that sufficient permissions are required to properly login to the cluster in Google Cloud Console.
Now user and group information is available through OIDC. Before you can actually use those users or groups in OIDC, you need to authorize them first. All API calls to Anthos apiserver are evaluated against the cluster RBAC policies. With no policy defined, OIDC users are unable to perform any actions.
At this point, your organization’s security team should be involved in defining roles and permissions for different teams/individuals. For example, the SRE team might be able to read all resources in Anthos cluster, whereas the dev team might be only allowed to list the deployments of their own namespaces.
Below is an example of dev team RBAC definition:
# dev-role.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: <dev team namespace> name: dev-role rules: - apiGroups: [""] resources: ["deployment", "pods"] verbs: ["get", "watch", "list"] - apiGroups: [""] resources: ["secrets"] verbs: ["list"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: dev-role-binding namespace: <dev team namespace> subjects: - kind: Group name: <OIDC groupPrefix><OIDC dev group name> # "name" is case sensitive apiGroup: rbac.authorization.k8s.io roleRef: kind: Role #this must be Role or ClusterRole name: dev-role # this must match the name of the Role or ClusterRole you wish to bind to apiGroup: rbac.authorization.k8s.io
There are a few debugging tools that can help with the Anthos OIDC integration:
- OIDC debugger: can mimic OIDC requests to OAuth provider.
- JWT debugger: can decode the ID Token returned from OAuth provider.
- kube-apiserver logs: can log errors or warning for OIDC requests.
Compared to other means to access Anthos clusters, authenticating using OIDC has 3 major benefits in my opinion.
Authentication Security Compliance
With OIDC integration, Anthos clusters are piggybacking all authentication tasks onto your existing OAuth provider. As a result, all your security tools like multi-factor authentication and password management are still applicable to Anthos clusters. No extra tasks are required to meet the authentication compliance for Anthos platform.
Managing access to Anthos cluster can be a full-time job. Luckily, with OIDC integration, Anthos access can be assigned and managed on group level and configurations can be part of Anthos Config Management. This means all permissions and policies can be reused and auto-applied to newly deployed Anthos clusters as well. Single user onboarding and offboarding are all handled by the OAuth provider. Thus, as long as the groups have no changes, there should be no need to reconfigure any RBAC policies.
OAuth providers usually provide detailed logs on the login activities of single users. With OIDC integration, audit logs are immediately available for Anthos.
To conclude, authentication and authorization are two key components you should add when bringing your Anthos platforms to production; and integration with OIDC is proven to be an efficient and reliable approach. If you are interested to learn more about Anthos Day 2 operations or OIDC integration with Kubernetes, please leave a comment or check out our Anthos Blueprint Solutions for more details.