Fogel was born in New York City, the son of Ukrainian Jewish immigrants from Odessa (1922). His brother, six years his senior, was his main intellectual influence in his youth as he listened to him and his college friends intensely discuss social and economic issues of the Great Depression. He graduated from the Stuyvesant High School in 1944. Upon his graduation he found himself with a love for literature and history and aspired for a career in science, but due to an extreme pessimism about the economy in the second half of the '40s, he shifted his interest towards economics. He was educated at Cornell University, where he majored in history with an economics minor, and became president of the campus branch of American Youth for Democracy, a communist organization. After graduation in 1948, he became a professional organizer for the Communist Party. After working eight years as a professional organizer, he rejected communism as unscientific and attended Columbia University, where he studied under George Stigler and obtained an MA in economics in 1960. He received a PhD from Johns Hopkins University in 1963.