Shahidullah was born in Peyara village, 24 Pargana, West Bengal (now in India) to a devoutly religious family. His father, Mafizuddin Ahmed, was a warden of a shrine. Shahidullah passed his school final Entrance exam in 1904 from Howrah Zilla School. In 1906 he passed the FA exam from Presidency College in Kolkata. He received the Bachelor of Arts degree with Honors in Sanskrit in 1910 from City College, Kolkata, and Master of Arts in 1912 in comparative philology from the University of Calcutta. He earned his PhD degree from Sorbonne University in 1928 for his research on the dialects of the Charyapada. He was the first Indian Muslim to receive this doctorate degree.