The son of Philo Bradley and Mercy Gardner Bradley, Bradley was born to humble beginnings in Berne, New York. He was the oldest of 12 children. He attended local schools and began teaching at the age of 16. In 1833, the Dutch Reformed Church of Berne advanced Joseph Bradley $250 to study for the ministry at Rutgers University. He graduated in 1836. After graduation, he was made Principal of the Millstone Academy, and decided to study law.