How can I make a package persist for my Shiny app so I do not need to (re)install it every time I start my workspace?
You should view the Environment Management article to learn how to build your own Environment which can include OS-related packages, Python and R.
Example: I want to use Markdown and flexdashboard in my app?
In your Dockerfile, add these commands:
RUN R --no-save -e "install.packages(c('flexdashboard', 'rmarkdown'))"