when serviceAcountName is gdr add the IPC_LOCK capability (#100)

This is a kludge to sidestep file system permission problems in
shared PVs for namespaces where multi-pod jobs run using
roce_gdr (thus as root). In these namespaces, using the gdr
serviceAccount will now always imply running as root.

Author: dgrove-oss

tools/pytorchjob-generator/chart/templates/_helpers.tpl

@@ -277,7 +277,7 @@ imagePullSecrets: []
 
 
 {{- define "mlbatch.securityContext" }}
-{{- if gt ( int .Values.numRoceGdr ) 0 }}
+{{- if or (gt ( int .Values.numRoceGdr ) 0) (eq .Values.serviceAccountName "gdr") }}
 securityContext:
   capabilities:
     add: