It took me way too long to realise that Arc<Mutex<T>> is basically a way to create multiple &mut T references (with runtime constraints).
This means that you can use plain &T and &mut T in the vast majority of your code. Most code doesn't need to care there's a mutex.