It's called Theatre of the Mind. I've definitely done it, and it has it's advantages (cheap, lower prep time) but I don't favor it nowadays. Especially in my last campaign, a swashbuckling pirate adventure, I tried to always have at least some kind of visual aid, because it's critical to that swashbuckling feel - the players can't swing from the chandelier if they don't know there's a chandelier.