Should Kubernetes controllers+CRDs managing resources in the external world (non-k8s APIs) try to fix drifts (such as out of band modifications made in the external API) periodically –or assume they're the only actor, and wait for the next modification to reconfigure? This makes scaling harder.
Comments
Not every system needs to fit the k8s paradigm.
There's no point of writing KRM paradigm off that quickly. We achieve marvelous things with it. Any alternative would look like writing hefty temporal workflows that are not event-driven. I like this better.
There are some systems (like Prodspec/Annealing) that learn from the current state and bring it back to the configuration SoT, but they're few and rare.
IMO not being able to manipulate them is a feature. :P when we need to manipulate we set a field called `spec.paused:true`.
cc/ @a-hilaly.bsky.social
But in the case where watching is not possible, polling may be acceptable.
For scaling, consider adding a component that centralizes the polling and transforms it into watches.
I feel if there's an assumption like CRD will always overwrite the external API, not sure if there's value in reconciling sooner (vs eventually when the resource.generation++).
In ConfigSync we have multiple layers and user can decide which they want:
1. Periodic Remediation
2. (Reactive) Drift Remediation
3. Drift Prevention
Most users prefer reactive remediation, with the option to pause remediation to deliberately apply drift in case of emergency.
I'm an optimist. :)