After upgrade to Domino 5.1/5.2 my Environment build or Model API publishing is slow

Issue

• After the upgrade, there is a different build process - that is slow or not as reactive
• I've upgraded and have REPOCLONER errors in my build log

Applies to

• Upgrades to Domino 5.1/5.2
• Potential remediation for Build pods prior to the change
• Feature Flags are set or unset (ImageBuilderV2, RepoclonerImageBuilds)

Solution

Revert the Build process to 'forge' or 'V1', Domino usually has two different methods available to manage the  build process and you may have a problem and this can get you quickly back to working state.

You should still contact Domino, and we can work with you to resolve any underlying problems!

# Enable Forge or V1 if builds are not working properly.

# Downgrade to V1/V2 builder : Go to Admin -->Advanced--> FeatureFlags
# Search for ShortLived.ImageBuilderV2 and set to false for V1 
# The ImagebuilderV2 tag needs to continue to be true to rollback to forge.
# Search for ShortLived.RepoclonerImageBuilds and set to false

kubectl delete -n domino-compute containerimagebuild --all
kubectl scale -n domino-platform deploy/hephaestus-manager --replicas=0
kubectl scale -n domino-compute deploy/forge --replicas=1
kubectl scale -n domino-compute deploy/replicator --replicas=1

For V1, remember to label a build node:

kubectl label no --overwrite <node> domino/build-node=true

and to use the V3 builder, hephaestus (as of 5.2)

# Enable hephaestus manager and turn off forge (for the new builder)

# Enable V2 builder : Go to Admin -->Advanced--> FeatureFlags
# Search for ShortLived.ImageBuilderV2 and set to true [ Default ]
# Search for ShortLived.RepoclonerImageBuilds and set to true [ Default ]

kubectl scale -n domino-compute deploy/forge --replicas=0
kubectl scale -n domino-compute deploy/replicator --replicas=0
kubectl scale -n domino-platform deploy/hephaestus-manager --replicas=1
kubectl delete -n domino-compute containerimagebuild --all 

References

https://docs.dominodatalab.com/release_notes/5-2-0/#_imagebuilder_v3

Alternative if the extract of images appears slow is here.