He then tells me because “no one tried to stop him” it was ok??

This sounds like a really effective argument as it’s so stupid that it takes all the wind from the sails of the person arguing against you.