Migrate away from ingress-nginx

[maxju]

ingress-nginx will no longer receive updates after March '26.

https://kubernetes.io/blog/2025/11/11/ingress-nginx-retirement/

https://gateway-api.sigs.k8s.io/guides/getting-started/migrating-from-ingress-nginx/

List of Gateway API implementations:
https://gateway-api.sigs.k8s.io/implementations/

List of Ingress Controller implementations:
https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/

[maxju]

Starting with MicroK8s 1.35, the ingress addon uses Traefik as the default ingress controller. Prior versions (< 1.35) used NGINX Ingress Controller.

https://canonical.com/microk8s/docs/addon-ingress

Minikube uses ingress-nginx by default, but provides a Kong Ingress Controller add-on
https://minikube.sigs.k8s.io/docs/handbook/addons/kong-ingress/

FLAME node already adds Kong as a dependency. Is this meant as a nginx replacement?

[tada5hi]

Starting with MicroK8s 1.35, the ingress addon uses Traefik as the default ingress controller. Prior versions (< 1.35) used NGINX Ingress Controller.

https://canonical.com/microk8s/docs/addon-ingress

Minikube uses ingress-nginx by default, but provides a Kong Ingress Controller add-on https://minikube.sigs.k8s.io/docs/handbook/addons/kong-ingress/

FLAME node already adds Kong as a dependency. Is this meant as a nginx replacement?

no it is used as proxy between an analysis and different services (e.g. storage, message-broker, fhir etc.)

[tada5hi]

There is also a gateway api implementation based on nginx. I would go for that: https://gateway-api.sigs.k8s.io/implementations/#nginx-gateway-fabric

[maxju]

There is also the official F5 nginx ingress controller which I am testing in minikube right now. But unlike the ingress-nginx add-on I can not get it to bind to ports 80 or 443.

However:
minikube service <ingress controller svc name> --url prints the ip and port of the ingress controller which can then be used to access the workloads. This might also help others with troubles reaching the cluster.

[maxju]

Temporary Conclusion:

• Microk8s already uses Traefik for the ingress addon in its latest version (which is compatible with nginx annotations)
• Binding minikube ingress to ports 443 or 80 only works with the minikube ingress addon (ingress-nginx)
• Only reproducable way to use a different ingress controller in minikube to use high port numbers
• If we use Microk8s for production and keep the limitations of minikube in mind, we are okay. I will test the latest Microk8s once the VMs are stable again.

Ingress Controller independent chart (Hub)

• The user should be able to decide what ingress controller to use. No ingress-nginx specific annotations should be hardcoded in the chart
• The FLAME Node already implements annotations via helm values
• ** feat(hub): Ingress controller independent ingress resources & merge root and hub sub path ingress #105 changes ingress annotations in the FLAME hub chart to be configurable via helm values.**
  This effectively enables the user to choose an ingress controller freely. This can only be merged once it has been tried in Microk8s, however.

[maxju]

Traefik implements some legacy ingress-nginx annotations, but not "rewrite-target", which is a problem for the Hub.
https://doc.traefik.io/traefik-hub/api-gateway/reference/routing/kubernetes/ref-ingress-nginx

[brucetony]

Minikube is used primarily for local development and debugging, I do not think we should support it in the future as a k8s distro for production use especially with our platform, let's focus on microk8s and maybe k3s for now.

To ensure long-term capability, we will need to support the new Gateway API as well as ingress. These periods of transition between standards are annoying, especially since locations that already use k8s will be slow to adopt the Gateway API while new installs will likely want to use the Gateway API immediately to avoid issues later on. For the node side, I will be creating Gateway template files for our components and making it so that users can choose which API to use during deployment.

[maxju]

Which gateway API implementation have you been using @brucetony ?

[brucetony]

For simplicity I have been testing everything using NGINX Gateway Fabric