Best Practices for Cluster Sharing

Cluster Sharing Best Practices with the Rafay Terraform Provider

This document outlines recommended practices for configuring and managing cluster sharing across projects using the Rafay Terraform Provider. It also highlights important enforcement changes introduced in provider version v1.1.47, which aim to eliminate misconfigurations and ensure consistency.

For more details, visit the official Terraform Registry page for the Rafay Provider:
Rafay Terraform Provider Documentation

---

Overview

In Rafay, clusters can be shared across projects to enable broader access. Terraform offers multiple ways to define this sharing, but only one method should be used at a time to avoid configuration drift or errors.

---

Recommended Sharing Methods

To ensure long-term maintainability and safe Day-2 operations (e.g., adding or removing projects from shared access), Rafay recommends using external sharing resources instead of embedding sharing within the cluster resource.

Use the following resources:

• rafay_cluster_sharing – When sharing a cluster with multiple projects.
• rafay_cluster_sharing_single – When sharing a cluster with exactly one project.

Why?
Keeping sharing logic separate from the cluster spec helps decouple cluster lifecycle operations from access management and reduces the risk of accidental cluster modifications.

---

Enforcement in Provider v1.1.47

Starting with provider version v1.1.47, you can no longer use multiple sharing mechanisms for the same cluster in a single configuration.
If you mix embedded sharing and external sharing resources, Terraform plan/apply will fail with an error.

---

Common Error Scenarios

1. Mixing Embedded and External Sharing

If you define both:

resource "rafay_eks_cluster" "example" {
  name = "my-cluster"

  sharing {
    projects = ["project-a", "project-b"]
  }
}

resource "rafay_cluster_sharing" "share" {
  cluster = rafay_eks_cluster.example.id
  projects = ["project-a", "project-b"]
}

Error:

Detected conflicting cluster sharing configurations in both 'rafay_*_cluster' and 'rafay_cluster_sharing' resources.
Please consolidate the sharing settings into a single resource.

2. Adding Embedded Sharing After External Sharing Exists

If the cluster is already shared using rafay_cluster_sharing, adding an embedded sharing block will raise a similar error.

---

How to Remove Existing Sharing

• Embedded sharing block
  → Delete the sharing block from the rafay_<cluster_type>_cluster resource
  → Run terraform apply

• External sharing resource
  → Run terraform destroy on the corresponding rafay_cluster_sharing or rafay_cluster_sharing_single resource

---

Migrating Mixed Configurations

If your current config uses both embedded and external sharing, follow these steps:

1. Remove the embedded sharing block from the cluster resource
2. Run terraform apply
3. Ensure sharing is defined in either rafay_cluster_sharing or rafay_cluster_sharing_single
4. Run terraform apply again to enforce the updated config

Note: During un-share/re-share operations, clusters may be temporarily inaccessible to affected projects for up to few seconds.

---

Sharing Update Timing

• Changes to project sharing such as unsharing a project from the cluster spec typically reflect in the platform within a short period.
• Expect a brief delay between applying changes in Terraform and seeing the updated state in the Rafay console.

---

Handling UI/API Drift

If someone manually updates cluster sharing using the UI or API, Terraform will detect a drift on the next plan:

• Example:
  Cluster is shared with project-a and project-b via embedded block,
  A third project project-c is added via the UI.
  → terraform plan will show that project-c is not part of the config and should be removed.

• Another case:
  One of the shared projects is removed via the UI.
  → terraform plan will detect the drift and show it needs to be re-added to match Terraform state.

To avoid such conflicts, always manage cluster sharing exclusively through Terraform once it's being used.

---