TL;DR: Steps to install CRI-O and Kubeadm provisioned cluster on Ubuntu 22.04.

Hard for me to believe, it has nearly  been five years since my first (and only) post on Kubeadm. That was circa Kubernetes version 1.09, and I was sort of posting what I had learned about Kubeadm, as I went through it.

I recently came up on my latest expiring home lab vSphere license. This annual event always causes me to reconsider how I use my server and whether handing out more cash is the best option. This year, I’ve decided to let go of the automation niceness, and see how I do without. You may have read some of my past posts where I tried Proxmox/KVM for a bit. I never really got the network and storage performing well enough, but may give it another try at some point. Continue reading →

This post is a published draft. I will be adding to it as I can. If you have items you think should be added, I’ve enabled the comments section for it. Happy to add your troubleshooting scenarios.

This post will cover basic troubleshooting of Crossplane. I have other posts that describe the components of Crossplane, so I won’t rehash all of that here. This is certainly not all encompassing. It is simply a list of common issues and how to resolve them.

Additional tips can be found here: link to Crossplane.io troubleshooting tips.

Everything that happens before the Managed Resource is created is controlled by the Crossplane core. If you are experiencing issues with Claims, XRs, or Compositions being created, then troubleshoot at the Crossplane core. Continue reading →

From two recent slack threads, I was reminded I’ve set some things between Argo CD and Crossplane, that I haven’t posted here.

This setting is required whether you use Argo CD or not. But it came up on a question about why the Argo CD custom health check wasn’t working. When creating a ProviderConfig for provider-kubernetes or provider-helm in a Composition, you must add base.readinessChecks[0].type: None (And because wordpress destroys all of my yaml formatting, see example of it here. at lines 291-292):

The other recent issue that came up was how Argo CD complains that a Crossplane ProviderConfig CRD doesn’t exist before the Provider is fully initialized. The way around that is with a simple annotation.

For that annotation, see line #7 of this ProviderConfig:

Hopefully this single post’s title shows up in searches for both issues.

TL;DR: Repo = https://github.com/n8sOrganization/cp-aws-irsa

The previous three posts were overviews of the six types of patches available to Crossplane Compositions. In this post, I’ll walk through a configuration that uses all of them. I’ll also touch on transforms when they come up. While the configuration is for EKS IRSA, explaining IRSA is not the intent of this post. In a nutshell, IRSA enables a K8s Role to attain the privileges of an AWS Role. In the case of this config, we are using IRSA so that the EBS CSI containers have appropriate privileges to provision EBS volumes.

The config consists of four XRs: XCluster, XK8s, XNetwork, and XChart. XCluster exposes a claim that accepts basic input of spec.id, spec.cloud, spec.parameters.nodes.size, and spec.parameters.nodes.count. XCluster selects a Composition that has XK8s and XNetwork nested. XNetwork provisions the VPC and all of the networking required for our EKS cluster. XK8s provisions the EKS cluster, IAM Roles, creates a ProviderConfig for provider-helm, and creates an instance of XChart to deploy the AWS EBS CSI helm chart. The XChart also configures IAM resources for IRSA. Continue reading →

In the previous two posts, I covered FromCompositeFieldPath and ToCompositeFieldPath. In this post, I’ll cover CombineFromComposite and CombineToComposite.

The Combine patches enable us to include multiple string values into a single patch. For this example, I’ll focus on CombineFromComposite, you can infer the CombineToComposite  patch from the previous ToCompositeFieldPath post. Continue reading →

By the end of this post, combined with my previous Crossplane intro posts, you should be capable of implementing a fairly robust Crossplane configuration. There is a lot more to be familiar with, but you can get a long way just knowing what I’ve covered so far.

Continuing with the series on Crossplane Composition patches, Im covering ToCompositeFieldPath here. As the name implies, this is the inverse of FromCompositeFieldPath. It’s used when we need a value created in one resource template (Managed Resource) supplied to another resource template or nested XR. We can even use combinations of ToCompositeFieldPath and FromCompositeFieldPath to pass values from one nested XR to another,

Continue reading →

I’ve covered Composite Resources (XRs) and Compositions in previous posts. I’ll provide a brief summary here, but you can review them in more detail if needed.. In the next series of posts, I’ll cover how we use Composition patches to pass values between/to related resources (all patch methods are documented here).

It is important to keep the concepts of Composite Resource and Composition clearly separated in your understanding. A Composite Resource (XR) defines a custom API, exposed via kube-apiserver. It selects a Composition that composes it into a collection of Managed Resources . The Composition is simply a ‘recipe’ to compose the Composite Resource(s). We patch to, from, and within Composite Resources via a Composition. We do not patch Compositions.

Continue reading →

As an update to my previous config of Crossplane with Argo CD, we can now configure Argo CD to use Annotations for sync tracking. This means we no longer have to configure resource ignore/deny rules per Composite Resource as we did with Label based tracking.

That said, I’m still undecided on XRC or XR as starting point for Crossplane in a GitOps pattern. Remove namespace boundary (that is a daydream at best for K8s multi-tenancy) by using vcluster, and you don’t need XRC. I’d personally throw XRC out the window and just use XR with vcluster. Makes the rest of this sort of irrelevant.

To configure Argo CD for Annotation resource tracking, edit the argocd-cm ConfigMap in the argocd Namespace (Argo CD version 2.4.8 or greater is recommended). Add application.resourceTrackingMethod: annotation to the data section as below: Continue reading →

External Secret Store is currently a beta feature of Crossplane. This feature enables us to publish connection details (Crossplane currently outputs these to a Kubernetes Secret in the Crossplane cluster) to an external vault, This then enables us to reference those secrets/connection details from a context outside of the Crossplane cluster. See this post for more on External Secret Store. But there is another interesting option when we simply need secrets from the Crossplane cluster to be available to Pods in a remote cluster. Continue reading →

Update: Upbound Cloud has been retired. So the basic premise of this post is no longer relevant. I believe it still has merit for similar use case, so I’ll leave it  active.

Disclaimer: Commercial product post. Ok, disclaimer out of the way, this topic comes up a lot in my day-to-day at work and I figured it would be helpful to provide some detail here.

Upbound Cloud is Upbound’s commercial SaaS offering backed by Crossplane. Argo CD is a popular GitOps platform that benefits greatly from  Crossplane. So why don’t they work nicely together ‘out of the box’? I have a series of previous posts that cover Crossplane and Argo CD in general. In this post, I’ll cover Upbound Cloud with Argo CD.  I’ll try to keep it short and to the point.

To follow along with this post, you’ll need an Upbound Cloud control plane and an instance of Argo CD running in a separate cluster.  You can follow the directions here for an Upbound Cloud trial. In that process, you’ll find directions to download a kubeconfig for your UBC instance. This will be used in proceeding steps. To install Argo CD, refer to Argo CD project page here. Continue reading →