Articles in this section

See more

How do I ensure a Python package from a private repo is usable in both runs and models?

Robert Wineriter
  • Updated
Follow

When developing code for a model you may pull in modules from private Git repos using the Git Repositories setting in the project files

image.png

and adding the line

-e /repos/reponame

to requirements.txt to ensure the module is installed when the run starts. This works when running the code in a Workspace or normal Run but when you try to publish it as a model you'll find that you'll get a ModuleNotFoundError for the module as models do not read or process requirements.txt.

A solution to ensure the code works in both normal runs and a model is to load the module with the following code:

try:
  import packagename
except ImportError:
  import subprocess,sys,site as site
  subprocess.call([sys.executable, "-m", "pip", "install", "-e", "/repos/reponame"])
  site.main()
finally:
  import packagename

Using this, if the requirements.txt is not processed your code will use start a shell to run pip and install the package from the mounted repository location ensuring your code works in both situations.

Related articles

Comments

1 comment

Date Votes
  • Bjorn Robertsson

    This can be extended to pull in a separate requirements file based on OS with:

    import platform
    ...
    subprocess.call([sys.executable, "-m", "pip", "install", "-e", "/mnt/requirements.txt-"+platform.system()])

    This would install files listed in requirements.txt-Windows, requirements.txt-Linux or requirements.txt-Darwin. File location may vary and on Windows you may need to rely on current directory!

    0
    Comment actions Permalink

Please sign in to leave a comment.