Yup, I couldn't see my own comments or posts on subs that were private. When I tried to delete them via API/script it got me an error too.

However, there's an exception. If you are a mod or approved user for a sub, then you can see and edit/delete as normal. I have never tried this scenario but maybe in this case when it go public again, any deletions are undone (because of the caching issue).