Upon parting ways with Travers in 1981, Aldridge moved to London, England and began working with Gary Moore, recording the Dirty Fingers album. American guitarist Randy Rhoads, who had recently arrived in England to record with Ozzy Osbourne, was a big fan of Moore's guitar playing, and one day he and Bob Daisley arrived to watch Moore's band rehearse. Aldridge and Rhoads became friends immediately. Aldridge had known Osbourne for several years, as Black Oak Arkansas had toured with Osbourne's previous band Black Sabbath extensively in the 1970s, and Aldridge's new friendship with Rhoads would lead him to join Osbourne's band a few months later. Although Aldridge is credited in the liner notes and pictured on the inner sleeve of Osbourne's 1981 album Diary of a Madman, Lee Kerslake actually performed all the drum parts on the original release. Aldridge has stated that working with Rhoads in Osbourne's band was one of the "musical highlights" of his life. Rhoads stated of Aldridge in late 1981 that "he was always my favorite drummer", stating that he was "knocked out" by the drummer after seeing him perform with Black Oak Arkansas on television in the 1970s. Aldridge appeared on Osbourne's Bark at the Moon album in 1983 as well as a pair of live albums (Speak of the Devil and Tribute) before leaving the band in 1984.