When Yolen was barely one year old, the family moved to California to accommodate Will's new job working for Hollywood film studios, doing publicity on films such as American Tragedy and Knut Rockne. The family moved back to New York City prior to the birth of Yolen's brother, Steve. When Will joined the Army as a Second Lieutenant to fight in England during World War II, Yolen, her mother and brother lived with her grandparents, Danny and Dan, in Newport News, Virginia. After the war, the family moved back to Manhattan, living on Central Park West and 97th Street until Yolen turned 13. She attended PS 93, where she enjoyed writing and singing, and became friends with future radio presenter Susan Stamberg. She also engaged writing by creating a newspaper for her apartment with her brother that she sold for five cents a copy. She was accepted to Music and Art High School. During the summer prior to that semester, she attended a Vermont summer camp, which was her first involvement with the Society of Friends (Quakers). Her family also moved to a ranch house in Westport, Connecticut, where she attended Bedford Junior high for ninth grade, and then Staples High School. She received a BA from Smith College in 1960 and a master's degree in Education from the University of Massachusetts in 1978. After graduating she moved back to New York City.