The FAA rules on drones are somewhat excessive, you're not even supposed to fly FPV without a spotter standing next to you keeping line of sight and there are tons of places you just can't legally fly at all, especially around cities.
There is a fairly robust community of home built FPV quadcopter enthusiasts who focus on speed and aerobatics. Ardupilot is a popular Free Software firmware for all kinds of drones.