Clifton All Whites was founded in 1963, originally under the name of Thistledown Rovers. The club's founder members included Dick Lambert and Bill Norwell. During the 1960s and 1970s Clifton All Whites dominated youth football in Nottinghamshire by attracting the best players from all over the county, now most players are local. In 1963 there were only four teams in the Young Elizabethan League, and as the club developed, more teams and clubs were founded, many with advice and help from Clifton All Whites. By 1966 plans were underway for Clifton All Whites to test their skills abroad in a tournament in the Netherlands, at a time when overseas tours were rare. In the early days when football clubs only usually supplied shirts for their players a new standard was set as Clifton All Whites supplied a full strip. Another first was when the under 12's played a top Scottish team before a crowd of over 20,000 before a First Division match between Nottingham Forest and Newcastle United.
Professional clubs attempted to persuade Clifton All Whites to become a nursery club, but still the club policy is not to become tied to any one club. David Staniforth was the first player to make a career as a professional footballer, with Sheffield United.
Clifton All Whites performed well in FA Youth Cup games and eventually gained their own ground. In 1973 the club ventured out of youth football to senior football and joined the Midland League. Mick Walker was the first club manager and coach. He later served as manager of Notts County. The club resigned from the Midland League partway the 1977–78 season. During the 1990s the club played in the Nottinghamshire Alliance and later joined the Nottinghamshire Senior League. In 2010 Clifton joined the Central Midlands League South Division and continues to play in that division. In 2013–14 Clifton won the South Division championship The first team are currently members of the East Midlands Counties League.