Yes, you definitely do, and there's a lot of them too (e.g.). Some of it is stuff you could conceivably know, even if it's because it's (often outdated) general knowledge, but plenty of it is very cryptic-specific. It does end up being like in-jokes and, I suppose, part of the pleasure.

The Guardian do a fairly accessible crossword for beginners: Guardian Quiptic, as well as the series Cryptic crosswords for beginners.

You can find the answers and explanations on fifteensquared.net which you will need at first! When I started I would do as much as I could then try to read as little of the answers as possible to get myself unstuck. Sometimes the answer would elicit a "well how the f was I supposed to get that?" but less so nowadays.