Add: Location metadata for clusters

[marians]

Towards

• Spec location metadata for clusters roadmap#3859

[piontec]

I like the idea, but using the standard k8s label, not our own

[marians]

Thanks @piontec for the review! FYI: I made some substantial changes after your review, mainly regarding the default location ID system for provide for on-prem.

[vxav]

That makes sense to me. Although would we use generic characters if we want to specify the city but not the subdivision? Otherwise the split(-)[2] won't always refer to the same category.

[pipo02mix]

LGTM

[marians]

That makes sense to me. Although would we use generic characters if we want to specify the city but not the subdivision? Otherwise the split(-)[2] won't always refer to the same category.

It's not really intended to be able to resolve the code again, semantically. So if there happens to be a code ab-cd-ef, we don't know whether ef is a city or a subdivision, and to me that's fine.

The reason I'd like to keep this open is to avoid overcomplication, as some countries might now have meaningful subdivisions, and in many cases there might just not be the need for the entire possible depth (especially if a customer has only one location in the entire country).

[stone-z]

This looks fine to me. I agree with Xav that there could be ambiguity about the meaning of the optional parts, but if we need to rely on that for some reason, that use case probably warrants its own proper support.

Requiring the country to be indicated for all clusters will help with compliance stories. Borders might change, but the label can follow.

I'd offer a preemptive suggestion that there should probably be a required, incrementing number for each location path. The first time a customer has two locations in a given city, we'll have ab-cd-ef-gh (which is the first location) and ab-cd-ef-gh-1 which is, counterintuitively, the second location. It' easy now to start with -1 and increment.

[stone-z]

Bumping the numbering suggestion here -- does anyone have thoughts for/against?

[marians]

The only downside I see is that the number isn't self-explanatory. But that's what labels are for.

When writing the proposal, I had in mind that the system would have to grow with the customer's needs. To use your example scenario, given eu-fr-cityname, they would have to change that to eu-fr-cityname-foo and add eu-fr-cityname-bar. But of course it would be less complicated if changing wasn't necessary.

[vxav]

It makes sense to me to have a trailing numeric ID, it certainly won't hurt at least.
In that case do we need the subdivision code? It's starting to be quite a long string.

[stone-z]

Something like what 3 words could also work and maybe be human-friendlier.

However, the exact locations of some sites are secret, so we would definitely have some opt-outs from this approach.

[anvddriesch]

I think this makes a lot of sense since I've seen the same pattern in terms of how customers think about their clusters and it will only grow once we have more smart factory installations.
Most importantly imo

• the goal of the default system is for as many customers as possible to use it so it might make sense to get some direct feedback there as well
• automation, monitoring etc on our side should be possible to parse the default system
• customers have to be able to choose any other system they want to use

[stone-z]

customers have to be able to choose any other system they want to use

I agree that it would be nice to allow customers to assign "friendly names" regardless of the provider, but I feel like that should be a separate field from "where is this infrastructure physically located?" I wonder if by allowing total flexibility for friendly naming, we limit the usefulness of the field for our own purposes

[marians]

customers have to be able to choose any other system they want to use

I agree that it would be nice to allow customers to assign "friendly names" regardless of the provider, but I feel like that should be a separate field from "where is this infrastructure physically located?" I wonder if by allowing total flexibility for friendly naming, we limit the usefulness of the field for our own purposes

To me, friendly naming should happen on the UI level, if the customer is interested. Like this:

If they use our default system, where e. g. eu-de-he-frankfurt would be available as a code, they could also make use of our default translation config on the UI level, which translates this to EU/Germany/Hessia/Franfurt (for example). If they want to maintain custom codes, they also would have to add custom UI config to get them translated. Otherwise they would be displayed in raw form.

That said, if a customer insisted in setting topology.kubernetes.io/region with strings like "Lille, France", they should be allowed to do so.

[anvddriesch]

To me, friendly naming should happen on the UI level, if the customer is interested. Like this:

If they use our default system, where e. g. eu-de-he-frankfurt would be available as a code, they could also make use of our default translation config on the UI level, which translates this to EU/Germany/Hessia/Franfurt (for example). If they want to maintain custom codes, they also would have to add custom UI config to get them translated. Otherwise they would be displayed in raw form.

That said, if a customer insisted in setting topology.kubernetes.io/region with strings like "Lille, France", they should be allowed to do so.

Those two solutions sound good to me. :)

[marians]

With the recent commit a9d86ca I added content related to populating the region label in metrics.

I would welcome feedback on this especially from @giantswarm/team-atlas .

[QuentinBisson]

Did we not just talk about scale to the cloud in the engineering workshop @JosephSalisbury ?

[JosephSalisbury]

yeah, i'm not 100% sure this is going to be true forever

however, even if we do start with some sort of "scale-to-the-cloud" solution in future, i think it's still reasonable to consider the cluster as being housed in one place (i.e: the control plane would still be colocated somewhere)

[QuentinBisson]

That's true. This just made me think that maybe we should show the node labels but that would bé for later I guess

[QuentinBisson]

Was this not supposed to be a node label used for scheduling? I am not sure if the topology label makes sense here

[marians]

The linked docs page states that it's used on Node, PersistentVolume. So far it seems it's not used on Cluster.

If we are uncertain about this, we can decide for a custom (giantswarm.io) label instead.

[QuentinBisson]

I would rather we use well known labels tbh, i'm wondering if there are not some in the CAPI or kcp work that would fit better but it's still okay as is

[JosephSalisbury]

overall, like the direction, some questions, no blocking

[JosephSalisbury]

yeah, i'm not 100% sure this is going to be true forever

however, even if we do start with some sort of "scale-to-the-cloud" solution in future, i think it's still reasonable to consider the cluster as being housed in one place (i.e: the control plane would still be colocated somewhere)

[marians]

Update:

• I changed from the suggestion to re-use the topology.kubernetes.io/region label to introducing our own one, so it's clear that this is our system in our jurisdiction, and no future change in the Kubernetes project can interfere.
• Added the suggestion of a number suffix to custom location codes (optional)

[stone-z]

Thanks for incorporating the number suffix 👍 I'll mull over the missing TOPIC.

Also, maybe we should include consideration of clusters that aren't on earth 😛
... mostly joking