The "speculative investors removing housing stock to drive up costs" folks tend to be corporately owned and industry coordinated properties that deliberately keep units open above the clearing rate, in hopes of driving up the prevailing cost of new housing.

This is dependant on the market (the post didn't say where they are), but I understand is true in the US.

In Australia, the speculation is driven by individuals who get incredible tax incentives if their income is above a certain level. Because of this, the housing market is distorted to the point where housing values are detached from rent potential, with all the value being driven by capital gains and tax offsets. This further leads to a situation where it's often more economically viable to leave a house empty (and therefore not have to maintain the property or deal with tenants) while the value grows and the tax is written down.