Now that we’ve covered the basics, let’s go a bit deeper (If you missed my previous 100 level series, start here). In this post I’ll move into some 200 level concepts, beginning with nested Composite Resources and Composition Selectors.

A nested XR is one created via a higher-order XR/Composition pair. So far, we’ve looked at a virtual network created by directly instantiating an XR/Composition. While network infrastructure is required in nearly all cases, it isn’t necessarily something we want the creator of ‘things’ to be engaged with.

They often aren’t familiar with best practices, variances by cloud, and/or organizational/regulatory compliance requirements. More often than not, they simply want to create something that results in an addressable application/service  (e.g. database, K8s cluster, app server, etc.), and have no interest in  how ‘the sausage is made’. In the DevOps practice, this is a natural demarcation point between Application Developers and the Platform Team. Let’s see how platform teams can provide application developers with a flexible interface to create a K8s cluster, while maintaining control of the underlying infrastructure details.

We’ll use the previous network Composition example in the form of a reusable component. We create an XRD that allows an application developer to define only three parameters: identifier, node count and node size. These three parameters will feed into a Composite Resource (I’ll refer to this higher-order XR as ‘entry XR’) that calls on a Composition to create a K8s cluster.

Rather than expose required attributes for the network, we’ll have the higher-order Composition pass values to a network XR, which in-turn selects another Composition to create the network. With this pattern, we completely abstract network creation and maintain governance.

We’ve seen how the basic XR -> Composition -> MR -> Provider -> Cloud API pattern works. Now, we’re adding a diverged pattern with XR -> Composition -> XR -> Composition -> MR -> Provider -> Cloud API.

As you can see from the diagram, the Entry XR takes three values (ID, Node Count, and Node Size) from a person or system. With those three values, the initial Composition performs a string transform on ID to create a cluster with the name ‘[ID]-cluster’, and then passes the ID value to a Network XR.

The Network XR passes the ID into its selected Composition where another string transform is performed to name and create a Resource Group, Virtual Network, and Subnet. The resulting MRs are processed by the Provider, resulting in objects in the target Cloud. The network Composition will also utilize the ID value passed from cluster Composition to apply a K8s label to the subnet MR so the cluster MR can correlate the relationship between both. The end result is a virtual network with subnet that the K8s cluster is placed on.

If we wanted to offer the option of creating a cluster in more than one cloud, we could add an additional field to the entry XR that represents desired Cloud and a Composition Selector. The entry XR will select which Composition to use based on the value in the Cloud field. This enables us to use a single XR with multiple Compositions. We’d have Entry XR -> to either Cloud 1 Composition or Cloud 2 Composition: In both cases, we’re enabling someone to create a K8s cluster, in some cloud provider., without requiring them to understand, or even allowing them to interact with, the provisioning of underlying/supporting cloud infrastructure.

If you follow me on LinkedIn, you may have seen a recent update regarding the completion my first project here at Upbound. That project entailed extending our Kubernetes Cluster + Data Services platform reference architecture guides to include Azure (Hence all the Azure examples in previous posts). Upbound announcement here.

Rather than repeat all of that YAML in this post, I’ll point you to the repo for examples of nested XRs. There are similar reference architectures for AWS and GCP if those better suit your needs. To see Composition Selector examples, check out the Multi-cloud Kubernetes Reference Platform here.

The Platform Reference Architectures include an exercise to walk you through creating a Crossplane, connecting it with a Cloud Provider, and provisioning all of the concepts discussed so far.

Invest time in reviewing the Composite Resource Definitions and Compositions to see how the K8s cluster is provisioned with minimal parameters exposed to a consumer, how the network is provisioned without input from the consumer, and how values are passed from cluster Composition to network Composition.

Look for how the cluster Composite Resource is exposed via a Composite Resource Claim rather than a cluster scoped Composite Resource. And how the network XR is not exposed with an XRC. This goes to the ‘least privilege’ security model previously discussed.

Note: You may have noticed the diagrammed examples above use an  XR, while the examples in the platform-ref-azure repo use an XRC to begin the process. Either will work, but in a multi-tenant cluster it is better to expose a namespace scoped XRC and employ K8s RBAC to restrict access to cluster scoped XRs.