The school was founded as the East Anglian School for Boys, incorporating an institution founded in 1873 by Congregationalist minister, Dr John H. L. Christien. It was one of a group of Methodist schools established in response to the growth of the middle class, the launching of the Woodard Schools and the 1867 Taunton Commission, which fuelled an expansion of secondary education in general and of non-conformist boarding schools in particular. The original school was in Northgate Street in Bury St Edmunds, but in 1886 it moved to Thingoe Hill in the town (a site later occupied by the East Anglian School for Girls).

In 1935 the school moved to Culford Park, former home of the 7th Earl Cadogan, and thereafter became known as Culford School. It is at the centre of East Anglia, c.90 minutes from London, 60 from Norwich, 40 from Ipswich, and c.30 minutes from Cambridge.

The school sits in 480 acres (1.9 km²) of Repton parkland with grazing, formal gardens, lake, and the 16th-18th Century Culford Hall. Originally the Hall became dormitories and classrooms; the laundry the sanatorium; the forge the art and woodwork studios (now the Pringle Centre for Design Technology); and the stables the Junior Department (now the Preparatory School).

The first new building to be added was Cadogan House, for junior boys, in 1938. The Leigh Memorial Swimming Pool was built in the same year. The Skinner and Hastings buildings were added in the 1960s, followed during the 1970s-1990s by an auditorium, pre-prep school, medical centre and biology laboratories. Purpose-built boarding houses and the Ashby Dining Hall (named after the then Chairman of the Governors) were constructed in 1972, which was also the year in which Culford amalgamated with its sister school, the East Anglian School for Girls (EASG), becoming one of the first fully co-educational HMC schools