Useful Labels for Running Kubectl Commands Against Pods in the Compute Namespace

Depending on your needs there are useful labels applied to each of the pods Domino creates on the k8s cluster that can help you quickly isolate your target, or get a complete view of all runs of a certain type. Following below are examples of the labels and associated kubectl commands using the -l option to filter for pods of a certain label. Values enclosed in angle brackets are variable and may differ in your cluster.

To find all run pods associated with a project owned by a particular user:

label:

dominodatalab.com/project-owner-username=integration-test

example:

kubectl get pods -n <domino-compute> -l dominodatalab.com/project-owner-username=<domino-username>

To find all run pods started by a particular user:

label:

dominodatalab.com/starting-user-username=integration-test

example:

kubectl get pods -n <domino-compute> -l  dominodatalab.com/starting-user-username=<domino-username>

To find all run pods associated with a project:

label:

dominodatalab.com/project-name=<project-name>

example:

 kubectl get pods -n <domino-compute> -l dominodatalab.com/project-name=<project-name>

To find all Workspaces:

label:

dominodatalab.com/workload-type=Workspace

example:

kubectl get pods -n <domino-compute> -l dominodatalab.com/workload-type=Workspace

To find all Apps:

label:

dominodatalab.com/workload-type=App

example:

kubectl get pods -n <domino-compute> -l dominodatalab.com/workload-type=App

To find all Jobs:

label:

dominodatalab.com/workload-type=Batch

example:

kubectl get pods -n <domino-compute> -l dominodatalab.com/workload-type=Batch

To find all Scheduled Jobs:

label:

dominodatalab.com/workload-type=Scheduled

example:

kubectl get pods -n <domino-compute> -l dominodatalab.com/workload-type=Scheduled

Model APIs use slightly different labels, so will not be retrieved through the examples above. Examples of model API labels follow below:

To find all model pods its best to get all pods in the compute namespace and pipe to grep:

kubectl get pods -n <domino-compute> | grep model

To find all pods associated with a particular model:

labels:

modelName=<model-name>

examples:

kubectl get pods -n <domino-compute> -l modelName=<model-name>

If you've found a pod you're interested in, you can then use the kubectl describe command to perform further inspection:

kubectl describe pod -n <domino-compute> <pod-name>

You'll find additional labels that may be of use, and useful troubleshooting information in the events section of the describe.

If you've found an issue occurring in your cluster that requires further assistance, please open a ticket with us and share screenshots of the problematic pods, and the output from the kubectl describe command on affected pods.