You did the right thing. Never lower yourself to the loser's level. He has problems and you don't, and when you fight someone like that you accept his problems upon yourself.
Just walk away knowing that he is always going to live with himself being a piece of shit that fails at everything.