I ran a classifieds website for over 14 years and battled all kinds of duplicate content on it. First, the homepage itself had a number of versions. I found that many of the URLs that duplicated the homepage had question marks (?) in them, so I blocked them in robots.txt. Then, I found a distinct rewritten URL that was a copy of the homepage, so I blocked that too. As for the categories, I had primary URLs that were rewritten like, /CategoryName/Number/PageX.html, so they were fine. They included a bunch of sort options though and all of those sort URLs had parameters in them with more question marks, so I blocked all of the sort option pages. As for the ad pages, each one led to another page that someone could click on to see just the ad images. Then there was the printer only page. And then there were five pages with URLs that included more parameters. They went to the contact seller, send to a friend, vote on ad, view votes, and a few more. I think there were five in all. So for every ad page, there were seven extra system generated pages. Those were all thin. The printer friendly and the image pages were duplicate content. The sort pages for the categories were duplicates. It was a mess. I blocked all of these extra pages. The only ones I allowed the search engines to crawl were the one homepage, category pages and their associated pagination, and the single ad pages with no extra pages. That seemed to work well. I didn't use 301 redirects or canonical. I tried and never had any luck with those options.
The thing about blocking pages in the robots.txt file is that if you're blocking pages that link directly from another that you're keeping crawled, you shouldn't notice any temporary ranking drop. If you're blocking a directory that's got only one link going to it, but there are tons of pages contained inside, like an entire section of the website, you'll notice a temporary drop in rankings. The reason for this is that the shallow, one hop, pages still have link flow (juice) going directly to them. The section that gets blocked that only has one entrance path becomes problematic because the link flow gets cut off to all the interior pages completely. So if you have 1,000 pages in a section and you block the door way into that section, all of the URLs contained inside have got to be drained of their pagerank completely before they get dropped from Google's index. That can take months or years sometimes, depending on the website. But, it needs to be done, so just do it.
Basically, my point is, block all pages that don't need to be crawled. The ones that you wouldn't expect to show up in the Google results.
