The 15th century wealth of South Suffolk almost completely rebuilt many of its churches, and in the triangle of Clare, Lavenham and Stoke by Nayland are some of the best examples of Perpendicular architecture in England. How strange then to find, right in the middle of this area, a maverick, a church quite unlike any other in the county. A beautiful, ancient place that speaks of more distant days than parvenu Perpendicular can possibly do. Is this the most beautiful setting of any medieval church in East Anglia? Perhaps only Dalham is more sensational. That St Mary is unusual is immediately apparent, for this is the only medieval stone spire in all Suffolk. Medieval spires of any kind are unusual in the county; the only full-sized one is at neighbouring Hadleigh, and there are reconstructed medieval ones at Bramford and Rattlesden. The stone spires of Woolpit and St Mary le Tower, Ipswich, may be grand; but they are, of course, modern inventions.

Polstead churchyard is also clearly ancient, set high and back from the village pond and street. In the churchyard are the remains of the so-called Gospel Oak, attributed with an age of 1300 years, and believed to have been planted by St Cedd, or at least his missionaries. There's no evidence for this, of course, or even for it being that age; although oaks have to come from somewhere, and there may have been an earlier one on the spot. The most beautiful view is to the east, where surprisingly large hills climb to Nayland. Here stands Polstead's war memorial, one of the largest church war memorials in England, remembering the Polstead men who did not come back.

This is an interesting village. As well as the pond, there is an ancient pub, and most famously a murder, for here it was that William Corder slew Maria Marten, and buried her in the red barn, before pretending to elope with her. It makes Polstead the only parish in East Anglia which has had a Tom Waits song written about it. More than 10,000 people witnessed Corder's execution on the market hill in Bury; the account of his trial, bound in his skin, can be seen at the Moyses Hall museum there.

St Mary has an unusual nave roof. Back in the 1980s, essentail repairs had to be carried out economically. Aluminium was chosen, and is a striking sight from a distance on a sunny day. Beneath it, you can see that the Perpendicular age did not completely avoid St Mary, but most details are Decorated, a sign of an early 14th century rebuilding, when faith was still shot through with mystery, and the cold theological rationalism of the 15th century had not yet built tall clerestories to let in the light. You go round to the south side to get in, and because of this many people miss the roundels and fragments of glass set in the windows of the north porch: continental pieces depict Judas betraying Christ with a kiss, the three Magi adoring the infant Jesus, and a pretty St Dorothy, surely part of the same set as the roundels at South Elmham All Saints in the north of the county. There is more old glass inside, but those fragments, in the south side of the chancel, are plainly local, of the Norwich School of the late 15th Century. One depicts a bishop who may be St Leger, and another is a sheep, a fine companion to those in the field to the east of the church.

You step inside to a beautiful space, touched with a patina of age. The most striking aspect of the interior is the colour, the combination between white walls and the red brick of the arcade arches. These bricks bear close examination. They date from the original construction of the arcades, about the year 1200, and yet they are clearly not reused Roman bricks. So, we have here what may be the oldest surviving English bricks still in use for their original purpose - bricks of a similar age can be found at the Hall at nearby Little Wenham, and across the county border at Coggleshall Abbey in Essex.

The arcades predate the aisles, as we have seen, and the little clerestory is hidden by the aisle roofs. This is strange, and also strange is that one of the arcades has been replaced, that to the west end of the south aisle, as though work began, but was not completed. It doesn't take much imagination to suggest that the Black Death of 1348-50 may have finished off (quite literally) the work here. The Norman arcades interlock and shift as we move around the church, opening up new vistas and elevations. There is much to see; an extraordinary brick octagonal font, for instance, which might be any age, but is set on a 13th century base. Sam Mortlock was uncharacteristically uncharitable about the modern fibreglass cover, but I rather like it. On the walls, two consecration crosses sit surprisingly close together, and another wall painting shows a figure, perhaps a bishop. A brass in the north wall of the chancel shows a priest dressed for the Mass - another rare survival. Also in the chancel, a good set of Laudian communion rails.

At the west end of the nave there is a large opening above the tower arch. This might be dismissed as a sanctus bell window, but I think it might be an entrance to the tower itself, that a ladder could be drawn up, as at Thorington. Step through the arch, and turn east, and you see that this, like Westhall, was originally a Norman church with a west entrance; the archway is clearly an exterior doorway, with three bands of heavy chevrons. Above the chancel arch there is a triple window that would have provided a backlight to the rood. Just to the south of the chancel arch, set at an angle, is the memorial to Jacob and Benjamin Brand. The Brands lived at Polstead Hall which you can still see to the west of the church. It is said that the little boy Benjamin was killed in a fall from one of the upper storey windows.

Before leaving Polstead, don't miss the heartbreaking modern memorial just to the east of the churchyard gate. In Memoriam, it is headed, and beneath it the inscription is to Alexander James Sowman aged 32, and his wife Jane aged 29, of this village, both of whom died in 1907 leaving their children Ellen 10 years, Annie 9 years, Alexander 6 years, Ivy 4 years, Rosa 4 months. At the bottom, the inscription concludes Erected by Anthea, the only Grand-daughter. This is an outstandingly lovely church, full of interest, and not simply because of anything in particular that it contains, but because of itself as a whole piece. Just the way it should be.