I'm fascinated to learn that people are discovering weaknesses in state-of-the-art bots for playing Go, such that a novice player can reliably win: https://goattack.far.ai/human-evaluation
This suggests that self-play doesn't always generalise: it's not sufficient to beat earlier versions.