Sweep is a really impressive demo of LLMs with coding: given your source code and an issue, it will generate a PR! Example: https://github.com/edreisMD/plugnplai/issues/65
I'd want to have a really thorough test suite if I was accepting LLM PRs though.