In 1939, Drew married Minnie Lenore Robbins, a professor of home economics at Spelman College in Atlanta, Georgia, whom he had met earlier during that year. They had three daughters and a son. His daughter Charlene Drew Jarvis served on Council of the District of Columbia from 1979 to 2000, was the president of Southeastern University from 1996 until 2009, and was a president of the District of Columbia Chamber of Commerce.
Beginning in 1939, Drew traveled to Tuskegee, Alabama to attend the annual free clinic at the John A. Andrew Memorial Hospital. For the 1950 Tuskegee clinic, Drew drove along with three other black physicians. Drew was driving around 8 a.m. on April 1. Still fatigued from spending the night before in the operating theater, he lost control of the vehicle. After careening into a field, the car somersaulted three times. The three other physicians suffered minor injuries. Drew was trapped with serious wounds; his foot had become wedged beneath the brake pedal. When reached by emergency technicians, he was in shock and barely alive due to severe leg injuries.