It depends on how much you trust the drive. If you don't trust it at all, just don't use it. If you trust it completely, use it as-is. In the middle is stuff like sandboxing.

Passing untrusted USB devices to a VM is tricky, though. There are VM guest escape vulns. If you're passing the USB device itself through, you'll probably have it connected to the hypervisor for a short time before enabling passthrough. Is that safe? And if you're passing a discrete PCIe USB controller or something, you have to trust that the hypervisor has implemented that securely.

If you find a USB device on the street, throw it out.