TL;DR: Repo = https://github.com/n8sOrganization/cp-aws-irsa
The previous three posts were overviews of the six types of patches available to Crossplane Compositions. In this post, I’ll walk through a configuration that uses all of them. I’ll also touch on transforms when they come up. While the configuration is for EKS IRSA, explaining IRSA is not the intent of this post. In a nutshell, IRSA enables a K8s Role to attain the privileges of an AWS Role. In the case of this config, we are using IRSA so that the EBS CSI containers have appropriate privileges to provision EBS volumes.
The config consists of four XRs: XCluster, XK8s, XNetwork, and XChart. XCluster exposes a claim that accepts basic input of spec.id, spec.cloud, spec.parameters.nodes.size, and spec.parameters.nodes.count. XCluster selects a Composition that has XK8s and XNetwork nested. XNetwork provisions the VPC and all of the networking required for our EKS cluster. XK8s provisions the EKS cluster, IAM Roles, creates a ProviderConfig for provider-helm, and creates an instance of XChart to deploy the AWS EBS CSI helm chart. The XChart also configures IAM resources for IRSA.
In all of that flow, we are passing just a few values in via our Claim, then generating a lot of AWS resources with values. We pass subnet IDs and Security Group IDs from XNetwork to XK8s (This is the earlier example of patching values from one nested XR to another). We pass OIDC Provider and AWS Account ID via PatchTo XK8s and then PatchFrom to XChart where we use a CombineFrom patch to configure IRSA IAM.
Beginning with the claim, we see the limited info we need to supply in order to kick everything off.
The claim is converted to an XR, which selects the xclusters.platformref.crossplane.io Composition. The first resource template creates the XNetwork XR and patches the required fields from the XCluster XR to it. notice we are patching from XCluster spec.cloud to XNetwork compositionSelector.matchLabels.cloud. We passed the value ‘aws’ from the claim, the XNetwork XR will select an appropriate Composition with the label cloud: aws. This is a good example of a fromFieldPath/toFieldPath patching to architect fro multi-cloud Compositions..
Look at the patches at lines 22 and 27. As mentioned in the previous post, this is patching from the nested XNetwork status.subnetIds and status.securityGroupIds to the XCluster status.securityGroupIds and status.securityGroupIds. This requires the XNetwork XR to perform a ToCompositeFieldPath patch when subnetIds and securityGroupIds are created, That in turn is picked up by this patch.
The second resource template in XCluster creates the XK8s XR. On line 46, you see a toFieldPath patch carries the namespace of the claim forward. This is to supply the XK8s Composition with the namespace connection secrets should be written to. On lines 54 and 57, you see the patches from XNetwork to XCluster being patched to the XK8s XR.
On line 40, you see a field explicitly defined for spec.providerConfig.aws. This is then patched at line 52. We’ll see how this is used for a patchSet in the XK8s Composition.
Near the beginning of the XK8s Composition, we define a patchSet to patch the spec.providerConfigRef.name value into all of the AWS resource templates. This is useful when you have a large Composition and are patching from and to the same field often.
At line 43 of the XK8s Composition, we are patching toComposite the OIDC issuer URL that was created by the EKS cluster. This is required later by the OIDCProvider resource template.
At line 48, we patch from the same field, but use a Regexp (regex) transform to strip the https:// from the beginning. This is the format required later by the AWS IAM Role we configure for IRSA. Another Regexp transform is used at line 60 to derive the bare AWS account ID (also required for the IRSA config)..
At line 270 of the XK8s Composition we have a resource template that patches the connection secret created by the EKS cluster resource template to create a providerConfig for provider-helm.
The last resource template in the XK8s Composition create an XChart XR for the EBS CSI driver helm chart. You can see the OIDC AWS account id discussed above being patched to it.
Finally, in the XChart Composition, you can see the multiple values required to define the AWS Role for IRSA.
That’s every patch in the context of a complete configuration. I’ve left the repo that contains all of this config public. Feel free to clone it and give it a spin for yourself. This config will provision VPC, network components for EKS, EKS with EBS CSI, and all required IAM resources configured. You should be able to create a PVC and successfully bind a volume. You can easily modify this config to work with any helm chart that provides k8s service account annotation. Crossplane isn’t difficult once you understand the basics.