Remove dependency on prompt_toolkit

prompt_toolkit is nice but we only use a tiny amount from it and it is a large dependency that drags in a lot of other stuff.