The colors bind us together. They proclaim our allegiance. Just as warriors wore them to identify themselves to each other through the ages, they bind people in recognition and support of a common cause. Walking through downtown if I wear a University of Washington shirt, people will inevitably say "Go Purple" or "Go Huskies." If they are from Washington State, they will make rude gestures or worse.
The colors may be silly, I mean who would pick orange and blue for Virginia (it used to be grey and crimson to celebrate the blood shed by the grey confederacy, what can I say, it's Virginia!) No self respecting daughter would let their dad leave the house clothed in maize and blue or purple and gold or green and gold like the Ducks of Oregon. But we do and our fashion conscious daughters and sons join us.
The colors connect us to our own memories of college or youth. They often may be one of the few things students at mega-universities have in common. They let us identify as tribe and nations. They help us participate in the rituals of being connected to others, connecting to a team or a history. And they permit us to do silly things, it's easier to yell and shout when everyone else around you is dressed in the same silly colors and doing the same. This is not always a good thing since it can encourage irresponsible behavior. Have you ever sat next to a group "Red Sox nation" people at a baseball game?
The colors even have their own histories and are very persnickety. Florida State is garnet. Alabama is crimson. Mississippi State is maroon. Don't confuse them. Princeton's orange comes from the William of Orange of the House of Nassau after which Princeton's first building is name. Michigan's notoriously ugly maize and blue comes from the graduation seal on 1859 diploma. Despite college marketing departments best efforts to tone colors down or make them more palatable, they remain gloriously resistant to fashionistas. You can compare old college and pro baseball and football colors, to the more market driven color coordinated marketing schemes of recent teams.
The start of college football is a time to celebrate bad taste in colors, ritual, symbols and the chance to announce who we are by the colors we wear.