It is the most efficient but I guess since demand is so high landlords can increase rents a lot, even if space is used super neat. At least in my country there is still a huge drain into the cities and renting/buying in the countryside is way more affordable.
My second guess is that it's not that easy to build new housing, even if more efficient, in already dense cities. Nobody wants to tear older houses down (expensive) if it still works and you can make a good living on renting them. Maybe some incentive on building newer housing could be a start.