Hello all

I'd like a sanity check please on a repo-structure that had been working okay in labs but recently seems to not.

I'm using a mono-repo approach and installing Flux using the bootstrap command with --path=clusters/<environment>. Elsewhere in the repo I have an apps directory with various Kustomize configurations to apply to a cluster.

I would then add a kustomisation.yaml file in clusters/<env> and use this as the root for things to be applied to the cluster, giving me a single location to add/remove things which felt quite tidy.

Example:

This seemed to be working fine as I played around with things a while back, but typically as I'm investigating this for 'proper' use I'm having a big problem: as soon as the kustomisation.yaml is present in clusters/<env> Flux promptly uninstalls most of its controllers.

If I run kubectl describe kustomization flux-system -n flux-system in the cluster prior to adding the file I can see all of Flux's own components listed. Once I add the kustomization and re-run the command it appears to consider these no longer required and starts to remove them. Of course because it's using it's own controllers to remove itself it only gets part way before erroring. It's like Flux considers it the new root config and promptly uninstalls most of its controllers because i've not referenced them.

What is strange is that one of my lab clusters is working fine and when I run the describe command on it, Flux's components are not present, yet still seem to function.

Currently, the fix seems to be to add the flux-system directory to the kustomization to ensure that Flux understands to keep this. What I find odd is that It feels like this would cause a self-referencing loop?

This is what I thought Flux was doing to apply to the cluster:

I may well be missing something obvious here about how Flux works through these files but happy to be corrected :)

thank you