Crossplane Packages – Self-signed Registry Cert

~ NateReid

Crossplane uses Composite Resource Definitions, Compositions, Providers, and Resource Manifests to elevate a Kubernetes cluster to a Universal Control Plane (Plenty on that in my previous posts). We can package all of these things into Crossplane Packages, serialized as OCI images, and then store and install them to/from any OCI image registry.

In this post, I’ll cover something you might run into if you’re testing Packages with a private registry that is configured with a self-signed certificate. It is relatively simple to implement a private registry running in Kubernetes with a free ACME cert configured via cert-manager. This avoids needing to configure things like Docker, containerd, and Crossplane to trust the signer of the cert. But it requires publicly  resolvable DNS records, and some other details that make it more of a burden when all we want to do is basic testing.

In most cases, we can configure things to trust a self-signed cert. It’s a bit different between each, but the end result is the same. We provide the thing with a copy of the CA (and any Intermediary) cert(s) and say ‘if you see a cert signed by this, it’s trustworthy’. For Crossplane, we can provide these details via Helm parameters. If you are not installing Crossplane or Upbound UXP with Helm, I strongly encourage it. It allows us to use Helm upgrade to non-destructively reconfigure Crossplane.

In this example, I’ll cover the steps for configuring with a Harbor registry running in a Kubernetes cluster. There are a lot of less-than-useful Harbor how-to resources on the web, so I’ll provide a step-by-step to install and test as I did. While there are some folks that seem to have gotten Harbor functioning on a KinD cluster, I’ve not had any success. So these steps will require that you have a ‘real’ cluster. Essentially, a cluster with a functioning K8s LoadBalancer and Ingress controller.

One important point to understand is that there are two registry images involved with hosting your own provider images. The first is what we’re making here, a provider package. Within the crossplane.yaml file used for the provider package, we specify the location to pull the actual provider image from. So you need to pull the image from its source (docker hub in most cases), and also push it to your registry. Then provide that path in the crossplane.yaml file used while building the provider package image.

To make life a little easier (hopefully), I’ve created a Git repo for all the config files you’ll need to follow along. Use:

git clone https://github.com/natereid72/cp-pkg-cert-post.git

Install Kubernetes and Harbor

Harbor is a CNCF container registry. We will use it to store and retrieve Crossplane Packages.

  1. Create a K8s cluster (=> v1.20) with a LoadBalancer and install an ingress controller (I used Contour).
  2. You need Helm installed on your client.


helm repo add harbor https://helm.goharbor.io
helm repo update
helm install -n harbor harbor/harbor
This will install Harbor and expose it via an ingress rule at core.harbor.domain. Add this with your ingress controller LB IP address to your local hosts file. You’ll also need to update with the same on the kubelet/worker nodes of the cluster your Crossplane will be installed in. If you install Crossplane in a separate KinD cluster, you can docker exec into the KinD node containers to achieve this. If you install in the same cluster as Harbor, you’ll likely need to access them via ssh.

Once you’ve done that, confirm that you can access the portal from your browser at https://core.harbor.domain. The default username/password is admin/Harbor12345. Create a new Project in Harbor called ‘crossplane’.

Configure Docker for Self-signed Cert

Before we can push our Crossplane Packages to the Harbor registry, we need to authenticate to it first. We will do this via Docker login, which requires Docker to trust the CA signed cert.

This will vary based on your OS. I’ve provided links to the Docker docs at the bottom of this post, I’ll provide steps for MacOS here. Because we are using an ingress controller, that is the CA cert we need, not the Harbor registry cert.

From the repo /package directory:

kubectl get secrets -n harbor harbor-ingress -o "jsonpath={.data.ca\.crt}" | base64 -d ca.crt

From Finder, locate the ca.crt file you should now have in your directory and double click on it.

    1. Choose System, click Ok, then Always Trust
    2. You will need to restart Docker Desktop for this to take effect
docker login core.harbor.domain (admin/Harbor12345)

Build and Push a Crossplane Provider Package

At this point, we are authenticated to the registry and Crossplane can push the image. Before Crossplane will trust the registry’s CA signed cert to pull and install the image, we will need to configure Crossplane to trust it (coming up).

  1. Install the kubectl crossplane plugin: curl -sL https://cli.upbound.io | sh
  2. From the repo /package directory:
    kubectl crossplane build provider
    kubectl crossplane push provider core.harbor.domain/crossplane/provider-gcp:v0.14.0

Install Crossplane

  1. kubectl create ns upbound-system
  2. helm repo add upbound-stable https://charts.upbound.io/stable
  3. helm repo update
  4. helm install uxp –namespace upbound-system upbound-stable/universal-crossplane –devel

Configure Crossplane for Registry Self-signed Cert

Here is where we configure Crossplane to trust the CA singed cert so that we can install packages from the registry.

  1. From the repo /package directory:
    1. kubectl -n upbound-system create cm ca-bundle-config –from-file=ca-bundle=./ca.crt
    2. helm upgrade uxp –namespace upbound-system upbound-stable/universal-crossplane –devel –set registryCaBundleConfig.name=ca-bundle-config,registryCaBundleConfig.key=ca-bundle 

Install the Provider Package

kubectl crossplane install provider:
core.harbor.domain/crossplane/provider-gcp:v0.14.0
kubectl get providers

Summary

This walked you through setting up Crossplane to trust registries with self-signed certs. This is primarily a setup encountered during testing. From here, checkout the docs on Crossplane Packages and see what else we can do. Packages are a great way to build out a self-service market place for your devops users.

Useful Links

Configure Docker client to trust self-singed CA certs:
MacOS
https://docs.docker.com/desktop/mac/#add-tls-certificates
All other
https://docs.docker.com/registry/insecure/#use-self-signed-certificates