Broussard was born in Baton Rouge, Louisiana and is an African-American of Louisiana Creole descent. Due to his father's job as a personnel manager for Traveler's Insurance Co., Broussard and his family moved often during his childhood. He lived in Cincinnati, Ohio, Indianapolis, Syracuse, New York, Des Moines, Iowa, and Cleveland, Ohio, before finishing high school. In 1986, he graduated from Holy Name High School in Cleveland. A standout football and basketball player at Holy Name, Broussard was inducted into the school's Hall of Fame in 2016. He matriculated to Oberlin College, from which he graduated in May 1990 with a bachelor's degree in English. He played point guard on the Oberlin basketball team.