How to use GPUs

If you need a GPU, you can request one by adding a resource limit for a number of nvidia.com/gpu, for example 1 (this is constrained by the number of GPUs available, and your quotas).

apiVersion: v1
kind: Pod
metadata:
  name: gpu-test
spec:
  containers:
    - name: cuda-container
      image: nvcr.io/nvidia/cuda:9.0-devel
      resources:
        limits:
          nvidia.com/gpu: 1 # Request 1 NVIDIA GPU
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - weight: 1
          preference:
            matchExpressions:
              - key: nvidia.com/gpu.product
                operator: In
                values:
                  # Use one of those GPU models
                  - "Tesla-P100-PCIE-12GB"
                  - "Tesla-v100-SXM2-16GB"

If you are using a Job, put the resource limit in the pod template:

apiVersion: batch/v1
kind: Job
metadata:
  name: gpu-test
spec:
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: cuda-container
          image: nvcr.io/nvidia/cuda:9.0-devel
          resources:
            limits:
              nvidia.com/gpu: 1 # Request 1 NVIDIA GPU
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              preference:
                matchExpressions:
                  - key: nvidia.com/gpu.product
                    operator: In
                    values:
                      # Use one of those GPU models
                      - "Tesla-P100-PCIE-12GB"
                      - "Tesla-v100-SXM2-16GB"

List of available GPUs

Number	GPU model
39	Tesla-P100-PCIE-12GB
16	Tesla-P40
4	Tesla-V100-SXM2-16GB
4	Quadro-RTX-8000
4	NVIDIA-A100-PCIE-40GB
2	NVIDIA-A100-80GB-PCIe