Update Autoscaler when IPRange Configuration is possible

[tloesch]

Hello,

I am currently working on an implementation that offers the option of specifying an IPRange for the entire autoscaler and/or for specific pools.
This solves a problem within this project where the autoscaled nodes appear to be placed randomly within the private network subnets.

I am also currently working on an implementation for this project.
My plans are:

• Set the autoscaler subnet as an IPRange globally within the autoscaler's ClusterConfig.
• Allow optional setting of IPRange per pool.
• Update the readme file.
• Update the autoscaler Helm chart or image tag.

I have a few questions:
Does anything need to be changed to update the autoscaler?
How can the autoscaler be updated if no new Helm chart version is released? Is it okay to change the version tag of the autoscaler image?

PR Autoscaler: kubernetes/autoscaler#8570

[M4t7e]

Hey @tloesch 🙂

That’s awesome! Thank you so much for working on this. I was really hoping this issue would be addressed someday, and it’s great to see you making it happen 🚀

Allow optional setting of IPRange per pool.

Can you share a bit more on why you want to implement this feature? Will Hcloud subnets be created automatically based on it, or do you expect them to already exist and simply reference them? Would you also allow using smaller subnets within it? Btw, we should also make sure not to go outside the network_node_ipv4_cidr range.

Does anything need to be changed to update the autoscaler?
How can the autoscaler be updated if no new Helm chart version is released?

The Helm charts are rendered but not applied directly. Talos applies them using inline manifests, so no Helm chart version change is needed for an update. A simple change in the rendered manifests is sufficient.

The ClusterConfig is mounted into the pod as a file and will be automatically updated. However, I’m not certain whether a file event listener is in place in Hetzner's CA implementation to reload the config when it changes. If not, a pod restart might be necessary, either triggered by a config change that modifies the output of helm_template.cluster_autoscaler so that a restart is needed or done manually:

kubectl -n kube-system rollout restart deployment cluster-autoscaler-hetzner-cluster-autoscaler

Is it okay to change the version tag of the autoscaler image?

In theory yes, if necessary. I assume you want to add this before the release in the CA Helm chart? The main point to consider is compatibility with the Kubernetes version. If your CA change is released in any 1.33.* version, we can reference it directly as a tag. If it is released as 1.34.*, I would prefer to wait until we are also running Kubernetes 1.34.

[tloesch]

Hey @M4t7e 😃

Can you share a bit more on why you want to implement this feature? Will Hcloud subnets be created automatically based on it, or do you expect them to already exist and simply reference them? Would you also allow using smaller subnets within it? Btw, we should also make sure not to go outside the network_node_ipv4_cidr range.

The reason for implementation within this project is a problem we encountered.
We expanded the network and subnets created by this project with an additional subnet in which we place gateway servers, for example.

Since the default setting for Hetzner Server network attachment is dynamic, it can happen (and did happen in our case) that the automatically scaled nodes are placed outside the network_node_ipv4_cidr, rendering these nodes unusable. In our case, the nodes are placed in the before mentioned (gateway) subnet.

The solution to this is to tell the autoscaler in which subnet/IP range the automatically scaled nodes should be placed.
In the case of this project, this would be hcloud_network_subnet.autoscaler.ip_range. This also ensures that it remains within network_node_ipv4_cidr.

With the new autoscaler feature, it is also possible to give each pool an assignment specifying the subnet in which its nodes should be placed. This is not necessary to fix the problem mentioned above, but could be a useful feature if needed. Consider it optional.

I have created a PR draft that could help to understand what changes might be necessary for this:
https://github.com/tloesch/terraform-hcloud-kubernetes/pull/1/files

The ClusterConfig is mounted into the pod as a file and will be automatically updated. However, I’m not certain whether a file event listener is in place in Hetzner's CA implementation to reload the config when it changes. If not, a pod restart might be necessary, either triggered by a config change that modifies the output of helm_template.cluster_autoscaler so that a restart is needed or done manually:

It has been applied in my tests so far. But I will take another look to make sure that it is actually being applied. Thank you!

In theory yes, if necessary. I assume you want to add this before the release in the CA Helm chart? The main point to consider is compatibility with the Kubernetes version. If your CA change is released in any 1.33.* version, we can reference it directly as a tag. If it is released as 1.34.*, I would prefer to wait until we are also running Kubernetes 1.34.

I wasn't aware of that. Then I would also want to make it dependent on compatibility. Thank you!