Great introduction to omniscient debugging, including comments on keeping performance acceptable: https://www.drdobbs.com/tools/omniscient-debugging/184406101