Arguing about what a good man should be more often than not develops along the lines of good – or more likely, bad – examples. We look at other people’s behavior and classify it. If it stopped there, all would be fine. Why not learn from the successes and mistakes of others?
But that is not what Marcus Aurelius is addressing here. The target of his pithy statement is gossip which has a significant social function, namely to unite those who gossip among themselves against the one who is the subject of the gossip. Try opting out of a round of gossip, and your fellow gossipers will treat you like a traitor. And indeed, a potential traitor you have become because by refusing to participate, you make a judgment about their propensity for gossiping and become a danger: Since you are not guilty of the same “crime”, you might go and tell the subject of the gossip what the others have been saying!
Scripture calls these people “scoffers”, “mockers”, “the scornful”, those who sit “in the chair of pestilence”.
Blessed is the man who hath not walked in the counsel of the ungodly, nor stood in the way of sinners, nor sat in the chair of pestilence.
(Ps 1:1, Douay-Reims Bible)
Incidentally, this harkens back to the Stoic dichotomy of control: Since you do not have complete control over other people’s behavior, stop talking about them and concentrate on what you have control over, i.e. yourself and what you do, think and say.