Query pods logs of Azure Kubernetes from Log Analytics

Query pods logs of Azure Kubernetes from Log Analytics

When you create Azure Kubernetes Service cluster (AKS), you can specify Log Analytics resource for logging and monitoring of Kubernetes resources: pods, services, configmaps, etc. This thing is actually installing Log Analytics agent inside your cluster (there will be several pods and related resources like configmaps), which then will send stdout and stderr outputs from pods to Log Analytics.

Unfortunately, the schema of these logs inside Log Analytics is quite complex, mostly because that the log record is coming from container, but not the pod, so they have multiple tables, which we must connect to get human-readable logging data.

You can check, to which Log Analytics resource AKS is sending logs here:

Query pods logs of Azure Kubernetes from Log Analytics

After long searches through the Internet, I've finally managed to find and customize kusto-query, that you can use to flexibly get and filter pod logs:

// Get logs from all containers in pod. Filter them by with 'has' operator. Then get latest 

ContainerLog
  | join kind = inner (KubePodInventory
    | project
        ContainerID,
        PodName=Name,
        ControllerKind,
        ControllerName,
        Namespace,
        ContainerName
    | distinct *
    )
    on ContainerID
  | where PodName startswith "${pod_name}"
  | where Namespace == "${namespace}"
  | project
    TimeGenerated,
    Namespace,
    PodName,
    ContainerName,
    LogEntry,
    ControllerKind,
    ControllerName,
    LogEntrySource
  | sort by TimeGenerated desc
  | where LogEntry has "${search_string}"
  | where TimeGenerated > ago(${time_span})

Note placeholders in the query ${some-text}, which you must replace to something that you need. Or remove if you wish to.

azure (en), aks (en)

  • Hits: 4740
Add comment

Comments  
Thanks!
I was looking for this! Thank you a lot!
SysAdmin
Thank you for this query, this is a great workaround for our inability to add loki to a cluster with windows nodes.

Related Articles