If you're seeing Dataset Details Status stuck in Pending or Failed status with Calculation errors (like the below) when attempting to register a Model Monito that uses a file from datasource in Domino releases of 5.0.0, 5.0.1, 5.0.2 or 5.1.0...
DMM compute node it logs,
No AWS Credentials provided by InstanceProfileCredentialsProvider : com.amazonaws.SDKClientException: Unable to load credentials from service endpoint
And Plier log has
ERROR: root: Model variable not found for model id:
...Please contact your local administrator to check and add the label:
aws-metadata-client: "true"
to spark3-worker nodes as a workaround (this issue is fixed in Domino 5.2.0).
[ec2-user@ip-10-0-0-9 ~]$ kubectl -n domino-compute get sts spark3-worker -o yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
annotations:
meta.helm.sh/release-name: spark3
meta.helm.sh/release-namespace: domino-platform
creationTimestamp: "2022-03-28T17:35:31Z"
generation: 1
labels:
app.kubernetes.io/component: worker
app.kubernetes.io/instance: spark3
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: spark
helm.sh/chart: spark3-3.0.2-0.5.3
name: spark3-worker
namespace: domino-compute
resourceVersion: "34752"
uid: 5ec369e3-6340-4d5a-9865-62f3895918f4
spec:
podManagementPolicy: OrderedReady
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: worker
app.kubernetes.io/instance: spark3
app.kubernetes.io/name: spark
serviceName: spark3-worker-headless
template:
metadata:
annotations:
prometheus.io/path: /metrics/
prometheus.io/port: "8081"
prometheus.io/scrape: "true"
creationTimestamp: null
labels:
app.kubernetes.io/component: worker
app.kubernetes.io/instance: spark3
app.kubernetes.io/name: spark
aws-metadata-client: "true"
compute-client: "true"
zookeeper-client: "true"
Comments
0 comments
Please sign in to leave a comment.