Elements of Gertrude's early life hinted at lesbian or perhaps what would now be termed non-binary or trans identity. She spoke of how she longed to be a boy, that her female body was a burden she was given to bear. Gertrude had a dear friend named Esther in her youth with whom a number of love letters were uncovered which made explicit the desires both had for a physical relationship that surpassed female friendship. Esther was the daughter of Richard Morris Hunt, the architect who had built Gertrude's family home in New York City and summer home—The Breakers—in Newport, Rhode Island, as well as many of the other Vanderbilts' mansions. Gertrude considered it one of the "thrills of my life, when Esther kissed me," and her mother, Alice, was so concerned about the friendship that she forbade Gertrude to see Esther. The separation seemed to have worked; for while Esther continued to write heartbroken letters of longing, Gertrude went on to have a bevy of male beaux. At age 21, on August 25, 1896, she married the extremely wealthy sportsman Harry Payne Whitney (1872–1930). A banker and investor, Whitney was the son of politician, William Collins Whitney, and Flora Payne, the daughter of former U.S. Senator from Ohio, Henry B. Payne, as well as sister to a Standard Oil Company magnate. Harry Whitney inherited a fortune in oil and tobacco as well as interests in banking. In New York, the couple lived in town houses originally belonging to William Whitney, first at 2 East 57th St., across the street from Gertrude's parents, and after William Whitney's death, at 871 Fifth Avenue. They also had a country estate in Old Westbury, Long Island. Gertrude and Harry Whitney had three children: