This fifteenth century Calvary group in the east wall of the the Lady Chapel was dug up from beneath the pavement of the chapel. It was the habit of some reformers to relegate altar tops and Cavalries to the floor, so that they would be trodden upon and desecrated. There was a second stone Calvary on the external wall of the tower. It has not been mutilated to the same extent as this one . The second Calvary was moved into the Church in 1989 and is now found on the north wall opposite the main entrance set into a walled up Saxon doorway that at one time provided an entrance to the Manor house grounds.

Saxon work in the north wall of the nave and chancel makes it certain that there was a Saxon church on this site. The recorded history of the church begins in the 12th century when was in the hands of the Lord of the Manor, Richard de Redvers Earl of Devon. Having an interest in the Abbey of St Mary de Montebourg in Normandy, he presented the manor of Loders to the Abbey. Possession of the manor was taken by monks from Montebourg who established Loders Priory. Depending on the political climate of the time, the manor then passed variously to the Carthusian monetary of St Anne at Coventry in 1399, back the jurisdiction of Montebourg under Henry IV and in 1414 under Henry V to the nunnery of Syon at Isleworth. After the dissolution of the monasteries in 1539 under Henry VIII it is presumed Loders Priory was either given or sold to supporters of the King.

In the 16th century the church suffered from the iconoclastic zeal of the reformers who smashed all the medieval glass except the few panels that remain high in the south window of the Lady Chapel. When restoration began in 1899 the Bishop of the time described the church as having "the most numerous relics of bygone ages discovered in any church in this diocese".