Yes this is a useful Wikipedia article. Especially the point just a little past the first line which specifically rebuts the point you are making:

"The territorial sea is sovereign territory, although foreign ships (military and civilian) are allowed innocent passage through it, or transit passage for straits"

And just to jump ahead to preempt your next argument, innocent passage is defined in the UN convention on the law of the sea here:

https://www.un.org/depts/los/convention_agreements/texts/unclos/part2.htm

Furthermore the Houthis are not making claims that the vessels attacked are violating innocent passage. Only you are for some reason.