Going out on a limb here but I think it's because of the portrayal of the house elves. A subjugated people who everyone said was better off in slavery, and the notion of emancipating them is ridiculed. Some people write it off as a coincidence, but there sure is a lot of these coincidences, like how the entire wizarding world's economy is run by hook-nosed evil creatures, that even conspired with the dark lord for their own gains.