Nginx-Ingress controller doesn't working after Kubernetes 1.24+ upgrade in Azure AKS

My customer require that we have supported versions of software installed, and I'm forced to upgrade Kubernetes as well according to the support policy of Kubernetes releases in Azure.

Last Kubernetes 1.24+ upgrade was not very clean - Nginx-ingress controller failed, and all our applications stopped responding.

So, I'm getting timeouts while accessing applications, and seeing these messages in logs of controller pods:

Service 'XXX' does not have any active Endpoint.

Solution

The problem was, that Azure AKS somewhat incorrectly updating its load balancer (Kubernetes 1.24 started to use HTTP/S health-checks instead of TCP). It was required to re-deploy ingress controller with additional annotation:

service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path: /healthz

Seems that re-deploy of updated configuration is forcing AKS to re-check load balancer settings and fix them according to the current behavior. Here's what I had prior to update:

And here's what after (notice the Path column):