Seems there's an issue with the dependencies currently and all versions of installing tortoise-tts from the existing instructions is doomed to failure without manual intervention.
As mikejgrecojr commented in the linked github issue, the fix for running via docker is:
Try updating your Dockerfile by adding in scipy to the conda install and specifying version 1.13.1 on line 31 like below. That worked for me:
Ruh roh, I don’t know the relevant science to understand the meme :( can anyone grant me an eli5 or tldr?