Datasets and app for storage: Can I store application persistent data into datastets?

Follow

Comments

3 comments

  • Jaclyn Patterson

    Hi Aly,

    There are a limited number of ways you can write to a Dataset: UI upload, the CLI, by running a script as a Domino Job, or using a Domino workspace (https://docs.dominodatalab.com/en/4.4/reference/data/datasets/Datasets_overview.html#writing-to-a-local-dataset). The only one that seems to fit your use case of writing to a Dataset from within an app is using the Domino CLI, specifically the upload-dataset command.

    For example, if the files you want to write to the Dataset are in /Users/myUser/data, you would run the following command:

    domino upload-dataset <username>/<project-name>/<dataset-name> /Users/myUser/data
    

    Instructions for installing the CLI can be found here: https://docs.dominodatalab.com/en/4.4/reference/cli/Installing_the_Domino_Command_Line_(CLI).html.

    If you have any questions let me know, or feel free to put in a Domino Support ticket at support@dominodatalab.com.

    Best,

    Jaclyn

    Submitted by: jaclyn.patterson

    0
    Comment actions Permalink
  • Jaclyn Patterson

    Hello,

    Thanks for the answer. The solution you suggest would mean we need to install CLI inside the environnement and set a kind of service account to run the command from the app container is that correct?

    CLI requires Domino login for me so it would mean we need to store the secret inside or with environment variables right? Or does it exist and embedded CLI version in the Domino env?

    It would be a good datasets improvement to better interact in read/write with app or with Model API I think.

    Antoine

    Submitted by: aly

    0
    Comment actions Permalink
  • Jaclyn Patterson

    Hi Aly,

    You're correct about needing the CLI inside the environment. An alternative would be to use the Domino API to save the files to your project files and then convert them to a Dataset Snapshot from there via something like a scheduled job for example. Environment variables would be your best bet for storing those credentials, you're correct.

    As for your comment on improvements, I completely agree. There are plans to drastically improve Datasets in the near future, so I'll make sure to add this suggestion as well.

    Best,

    Jaclyn

    Submitted by: jaclyn.patterson

    0
    Comment actions Permalink

Please sign in to leave a comment.