Yeah that's not correct, downloading pirated content is still illegal, but it makes no sense for copyright holders to go after 1 person for a movie that costs $20.
You go after the uploader strictly because you can sue for damages for each person/ip they uploaded to. It's just strictly a monetary and legal thing as to why only the uploaders are taken to court.
Pretty sure even if you block uploads with your torrent client you will still get a DMCA for downloading from a public tracker.