Workspace "errorCode="NoSuchKey" errorMessage="The specified key does not exist." after CLI upload

A defect presents itself when starting a new workspace in early versions of Domino after the project has been updated via the domino upload CLI command.   This is resolved in 4.5.0.  It can occur when com.cerebro.domino.frontend.clientBlobModeOverride is set to API and there are many files being uploaded.

The scenario involves running CLI's domino upload then starting a workspace for the project.  The symptom is a workspace error visible in the User Log like:

Critical error in run 6093f82fc2d98a62b504efdf: java.lang.IllegalStateException: Blob transfer (download) failed because 10 operations failed: Download blob 7bb0ffb6151a86cfc24e39fbccea8fb27a56f452 -> domino.server.sync.S3BlobAccessException: S3ServiceException: endpoint="s3-us-west-2.amazonaws.com" errorCode="NoSuchKey" errorMessage="The specified key does not exist."

RESOLUTION

This error occurs because of a corruption in the upload of project files due to defective handling in dropped requests/files.   Workarounds include

- Upgrade to 4.5.0

- Use Central Config com.cerebro.domino.frontend.clientBlobModeOverride set to S3

- Upload just a single file rather than many.   You can archive your directory via tar,  then do domino upload, then within your workspace use tar -xzvf.   

If the problem occurs frequently the latter option can be scripted with something like:

#!/usr/bin/env bash

# Set a shell flag to have the entire script fail if one command fails
set -e
...
upload_project "quick-start"

function upload_project() (
    # Open the project directory and remove the existing .domino directory
    cd "${1}"
    rm -rf .domino

    # Tar the project, excluding .domino and CLI log
    tar --exclude='./domino.log' -cvzf "../${1}.tar.gz" .

    # Delete everything and move the tar ball back to this directory
    find . -not -name "${1}.tar.gz" -delete
    mv "../${1}.tar.gz" .

    # Initialize the project and pipe in an enter to use the default name
    printf '\n' | domino init

    # Kick off run to untar file, which wil force an upload
    domino run --direct "tar -xzvf ${1}.tar.gz && rm -rf ${1}.tar.gz"
)