«Yellow» was written in Rockfield’s Quadrangle studio near Monmouth in South Wales, where Coldplay began working on their debut album, Parachutes. One night after finishing recording «Shiver», the band took a break and went out of the studio. Outside, there were few lights on and the stars in the sky were visible and «just amazing», according to the song’s co-producer, Ken Nelson. He told the band to look at the stars, which they did. Lead singer Chris Martin was inspired by the sight and the song’s main melody, consisting of a chord pattern, popped into his head. At first, Martin did not take it seriously and sang the tune to the rest of the band in «his worst Neil Young impersonation voice». Martin has said, «The song had the word ‘stars’ and that seemed like a word you should sing in a Neil Young voice.» The melody «started off a lot slower», according to drummer Will Champion, and it sounded like a Neil Young song. Not long after, despite not taking the song seriously, Martin’s idea worked out when he had developed the tempo of the verse. When guitarist Jonny Buckland started playing it and supplemented it with his ideas, they had created the riff, «and it sort of got a bit heavier».

While composing the song’s lyrics, Martin could not find the right words. He was thinking of a specific word, which he deemed a missing keyword in the lyrics, to fit the song’s concept. He looked around the studio and saw the Yellow Pages. The lyrics progressed from there, with the band collaborating. Bassist Guy Berryman came up with the opening line «Look at the stars». That night, having quickly composed the song, the band recorded it.