Gordon's sound was commonly characterized as being "large" and spacious and he had a tendency to play behind the beat. He was known for inserting musical quotes into his solos, with sources as diverse as "Happy Birthday" and well known melodies from the operas of Wagner. This is not unusual in jazz improvisation, but Gordon did it frequently enough to make it a hallmark of his style. One of his major influences was Lester Young. Gordon, in turn, was an early influence on John Coltrane and Sonny Rollins. Rollins and Coltrane then influenced Gordon's playing as he explored hard bop and modal playing during the 1960s.

Gordon was known for his genial and humorous stage presence. He was an advocate of playing to communicate with the audience, which was his musical approach as well. His improvisation was remarkably engaging and intelligent, but never gratuitously complex or unusual. It was always a conversation simultaneously delightful and intellectual. One of his idiosyncratic rituals was to recite lyrics from each ballad before playing it.

A photograph by Herman Leonard of Gordon taking a smoke break at the Royal Roost in 1948 is one of the iconic images in jazz photography. Cigarettes were a recurring theme on covers of Gordon's albums.
Gordon was nominated for an Academy Award for Best Actor in a Leading Role for his performance in the Bertrand Tavernier film Round Midnight (Warner Bros, 1986), and he won a Grammy for Best Jazz Instrumental Performance, Soloist, for the soundtrack album The Other Side of Round Midnight (Blue Note Records, 1986). He also had a cameo role in the 1990 film Awakenings. In 2018, Gordon's album Go (Blue Note, 1962) was selected by the Library of Congress for preservation in the National Recording Registry for being "culturally, historically, or aesthetically significant".