After heart surgery in 2006, he performed at the New Orleans Jazz and Heritage Festival and helped reopen the Bay St. Louis casino. It has since been renamed the Hollywood Casino. He performed his last show at Hollywood Casino on December 8, 2010, before returning to help reopen the resort in 2014, by which point he was mostly retired.
Fountain was a founder and the most prominent member of the Half-Fast Walking Club, one of the best known freelance marching units that parade in New Orleans on Mardi Gras Day. The original name was "The Half-Assed Walking Club," and it was an excuse to take a "lubricated" musical stroll down the parade route. Pete changed the name under pressure exerted by the parade organizers. On Mardi Gras Day 2007, Pete again joined his Half-Fast Walking Club, having missed the event in 2006 due to illness.