I was thinking for about 7 seconds and an idea grew; You know how when you get banned, your username get's marked with a line? Well I thought it'd be very bombing that when you get kicked, a small boot appears next to your name, and a timer.
Example: Iceeman234 [Boot] 0:30
The timer would run the amount of time you're kicked so you can't just come back and raise hell once again. You'll have to wait 30 seconds to come back, and if you still screw up, more time will be added.