-
Notifications
You must be signed in to change notification settings - Fork 654
[RFC-0010] Multi-Tenant Workload Identity #5209
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
b795adf
to
3034741
Compare
5a5f4ac
to
614bbc8
Compare
f505036
to
3e9b36a
Compare
f3dc52d
to
2f0b445
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@matheuscscp good job putting this together. From a high-level it is a great feature and would further bolster Flux's least privileged model.
I added a few nits, but overall multi-tenancy is a charged term, so it is quite important to define what precisely we mean by that and what guarantees we are providing as part of this RFC.
My take is that this supports out-of-the-box the multiple teams model. The multiple customers model is achievable, but requires stronger security boundaries at cluster topology level - for coping with more hostile environment.
It would be nice to explicitly call out in the RFC:
- The cache keys format, so that it is crystal clear that it will be resistant to cross-tenant takeovers. As you already pointed out, when sharing the same cache storage, the cache key will require some level of tenant or object level information to provide isolation. What in the format picked will result in a malicious tenant not being able to forge it (e.g. the fully qualified service account name plus x, y and z)?
- What security controls are in place (or can be implemented by the user) to ensure that even if
tenant A
knows the Cloud Provider annotations fortenant B
, it won't be able to impersonate them.
- This can be as simple as they enforcing rules via Admission Controllers (e.g. Kubewarden), as per some examples of Flux multi-tenancy. Or something more sophisticated as part of the new feature.
- The limitations of this feature for multi-tenancy in hostile environments where the tenants are not trustworthy. Suggestions on how to overcome them is optional, as that can be a larger topic - e.g. stronger isolation so that tenants can't impersonate each other's identities even if they know them and can bypass controls that may be operating at a degraded state (i.e. admission controllers).
@pjbgf Thank you very much for the review!
Agreed! Added a section to clarify this on the beginning 👍
Format added 👍 And specifically about:
In the Cache Key section, the five paragraphs following the list of components are there to justify the presence of each component, be it for preventing malicious tenants from forging/stealing tokens from other tenants, or for making sure that served tokens are according to the request specifications and hence will be valid for the use case in question. I added a
Great point, there's a paragraph in the Technical Background section pointing out how this works. This is inherently built into the workload identity features of each cloud provider, you must create a strong link between the Kubernetes
Here's the updated paragraph with a bit more details at the end:
So essentially the identities are not secret, knowing what cloud identity a tenant uses gives no advantages to a malicious neighbor tenant whatsoever.
As discussed right above, no admission controllers are required, the impersonation permission is implemented and enforced by the cloud provider. You grant impersonation permission for Workload identity is pretty solid :) |
New SOPS version 3.10.0 released with the GCP KMS |
af55097
to
ac8a8de
Compare
I'm now addressing the offline comments I got during KubeCon EU 2025. From @stealthybox:
From @hiddeco:
From @stefanprodan:
|
@stealthybox @hiddeco @stefanprodan Comments from KubeCon addressed, please feel free to do another pass/approve. |
6de8de0
to
c76ea14
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
64e29f8
to
0f4cf9f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀
0f4cf9f
to
d0a69fe
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Thanks @matheuscscp 🏅
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
0ca7ef6
to
a7e41df
Compare
In this RFC we aim to add support for multi-tenant workload identity in Flux, i.e. the ability to specify at the object-level which set of cloud provider permissions must be used for interacting with the respective cloud provider on behalf of the reconciliation of the object. In this process, credentials must be obtained automatically, i.e. this feature must not involve the use of secrets. This would be useful in a number of Flux APIs that need to interact with cloud providers, including all controllers except helm-controller.
Link: https://github.com/fluxcd/flux2/blob/main/rfcs/0010-multi-tenant-workload-identity/README.md
Umbrella issue for implementation: #5022