vCluster with OIDC

This post covers the use of vCluster  with the addition of OIDC. The outcome will be a namespaced control plane (vCluster), preconfigured for OIDC with a specified group added to the cluster-admin role. This is a stepping stone to fully automated vCluster deployment. With this setup, we can use email, or Slack, or whatever we’d like to distribute the kubeconfig files.

There are additional automation requirements remaining before we get to a completely turnkey control plane provisioning level. This is just the first step in that direction.

I’m using the default vCluster Helm chart with some added values. The vCluster Helm templates and value files would need additional work for complete automation. This is primarily due to the way vCluster constructs the kubeconfig files. Beyond the Chart, vCluster uses some ‘helper’ code from a pod to construct it. That is likely another area we’d have to dive into. Short of that, we can modify the kubeconfig files manually (which I’m doing in this post).

One additional part of the automation we’d want to expand on is the inclusion of an Ingress definition for each cluster (would also need to wrap back into the kubeconfig file).  I’m shortcutting and using LB service.

As always, I’ll transition to a GitHub repo for the rest of this…

OIDC – DevOps and SRE Level – Part 3

Putting Our OIDC Knowledge to Use

In this final OIDC post, I’ll cover setting up an Identity Provider/IdP with  OAuth + OIDC + Directory Service, configuring an OAuth/OIDC application definition, configuring kube-apiserver to trust the OIauth/OIDC service, and configuring  kubectl as an OAuth Client. We’ll rely a lot on the understandings from the first two posts. So if you haven’t already, go over the first and second post before diving into this one. Continue reading

OIDC – DevOps and SRE Level – Part 2

OIDC and the Circle (or Triangle) of Trust

In the first post, I spent time describing what a JWT is and how it is signed with an x509 certificate. That will be useful in understanding this second piece. In this second part of OIDC for SRE and DevOps level, I’ll cover the primary entities involved and their relationships for identity authentication. As I mentioned in the first post, OIDC is an extension of OAuth. Without OAuth, there is no OIDC. OIDC is actually a very small addition to the OAuth specification. So a lot of the explanation here requires explaining OAuth. I will get to OIDC by the end. Continue reading

OIDC – DevOps and SRE Level -Part 1

I know what I’d be thinking if I were you, “what, another OIDC explanation, why?”. There is so much written on OIDC, much of which makes it seem way more complicated to me than it is. So I’m writing a series that explains it in a way I think will make more sense to Ops/SREs. In my previous post, I started out planning to layout an entire recipe for setting up an automated Crossplane cluster tenant provisioning process. Including automated OIDC configuration so that only the proper identities could access them Then I pumped the brakes. Continue reading

K8s – Stacked etcd to External – Zero Downtime

Because sometimes you start off with stacked etcd nodes, and then decide you really wanted external.

First and foremost, my blog disclaimer is in full effect, especially on this one. Use this info at your own risk! In this post, I’ll cover the steps to convert a Kubeadm deployed stacked etcd cluster into one consuming external etcd nodes, with no downtime.

While this guide is based on Kubeadm clusters, the process can be applied to any cluster if you have access to the etcd config and certs/keys.

Note: This will split the etcd service away from the Kubeadm upgrade process. You will need to ensure etcd upgrades and version compatibility manually. You will also need to update your kubeadm-config ConfigMap to tell Kubeadm that the etcd is external. Continue reading

Install etcd Cluster with TLS

This topic has been written up multiple times, so not exactly cutting edge info here. But I’ve found so many of the tutorials to be dated, and/or lacking in specific detail, and/or using tools I don’t have interest in (e.g. cfssl). So, I decided to post this no-nonsense, just works guide. This post will cover installing an etcd cluster, secured with TLS. In my previous post, I covered some basics on creating self-signed certs with openssl. The one additional openssl detail in this post will be the openssl config file to configure generated CSRs. Continue reading

Openssl Self-Signed Certs – 2023

I set out this morning to write a post on configuring external ETCD for Kubernetes, with openssl self-signed certs (you know, the kind you use for your home lab). I got sidetracked on openssl and all of its ever-changing/deprecated options. So, this will be a preamble to that original intent.

Other than key sizes and algorithms, only a few command options have changed in openssl v3. I won’t dive into key size and algorithms (too much to cover there). In this post, I will cover the why and how of creating self-signed certificates with openssl, along with up to date commands for v3.  I think the first time I grappled with understanding SSL (now TLS) was 1997. Although SSL has now become TLS, not a lot has changed with the underlying basic dynamics of public key encryption. Continue reading

Kubeadm Upgrade

A few posts back, I revisited setting up a Kubernetes cluster from scratch with Kubeadm. In this post, I’m going to cover upgrading a Kubeadm deployed cluster.

The Kubernetes version I deployed previously is v1.26.1 and now 1.26.2 is available. So lets go through the process of upgrading to 1.26.2. The first thing you should do (not covered here) is to backup your Kubernetes cluster. In the event you encounter an issue that Kubeadm cannot recover from, the backup is your next recourse. At a minimum, make a backup of the etcd database.

To find the latest release of K8s, we can look at https://github.com/kubernetes/kubernetes/releases. Here we see a 1.27.0 alpha pre-release is posted, and the latest is 1.26.2. (Steps also available in this repo) Continue reading