it just punishes anyone who diverges from the herd
So you're saying people get less karma for posts that diverge from the herd. That can only mean those posts get fewer upvotes, which means they rank lower
In other words, the sort-by-votes system makes uncontroversial posts more likely to be seen, while the karma system makes those same posts more likely to exist. My point, therefore, is that both systems have a similar overall effect on the website, and that the result of their conjunction is only that the effect is amplified
This