Problem:
A customer was building a Compute Environment after modifying dockerFile instructions and the revision build failed with "killed", example line from build log for the revision:
Jan 28 2022 11:50:22 -0700 Solving environment: ...working... ip-10-62-114-166.ec2.internal domino-build-61f43a600ff91805e7f45c7a domino-build-61f43a600ff91805e7f45c7a
Jan 28 2022 11:51:52 -0700 [91mKilled
Isolating snippets or individual instructions/commands from the dockerFile helped focus on one specific line:
RUN conda create -c conda-forge -n cross-sell-env python=3.7.11 hvac msal \
oauth2client google-cloud-bigquery google-api-python-client memory_profiler \
boto3=1.18.21 lightgbm=3.3.2 matplotlib=3.5.0 numpy=1.20.3 pandas=1.3.5 \
pyxlsb=1.0.9 scikit-learn=1.0.2 scipy=1.7.3 shap=0.39.0 ipykernel line_profiler \
cairocffi=1.2.0 pandas-gbq=0.17.0 shapely
Testing the same command within a workspace terminal also resulted in the "Killed":
root@run-61f8407b7ba961687265c27c-shlsm:~# /usr/local/anaconda/bin/conda create -c conda-forge -n cross-sell-env python=3.7.11 hvac msal oauth2client google-cloud-bigquery google-api-python-client memory_profiler boto3=1.18.21 lightgbm=3.3.2 matplotlib=3.5.0 numpy=1.20.3 pandas=1.3.5 pyxlsb=1.0.9 scikit-learn=1.0.2 scipy=1.7.3 shap=0.39.0 ipykernel line_profiler cairocffi=1.2.0 pandas-gbq=0.17.0 geopandas shapely build
Collecting package metadata (repodata.json): done
Solving environment: | Killed
Resolution:
The fix was simply to break-up the lengthy command into several individual commands since it seems a timeout or other constraint was occurring during the process of fetching/compiling the various libraries and dependencies.
There is a limit (127) to the number of layers (aka instructions or commands) a docker image can have, and underlying base images may have already used many layers. So limiting the number of commands is wise, but in this case it was necessary to break-up one lengthy command into 3 different commands. https://docs.dominodatalab.com/en/5.1/reference/environments/Environment_management.html?highlight=127#dockerfile-best-practices
Comments
0 comments
Please sign in to leave a comment.