The castle was built by Lancelot 1st du Lac, whose second spouse was Louise de Coligny, the sister of field-marshal de Chatillon, the father of Admiral Gaspard II de Coligny. The latter was close to King Louis XII and governor of Orléans. His grandson Lancelot du Lac joined the Protestant forces in the early days of the wars of religion, in 1562. He was commander of the field-marshal's personal troops. The Chapel in Chamerolles was used as a place of worship.

The castle, surrounded by a moat and Renaissance gardens, was recently purchased by the Loiret local authorities.

The Castle in Chamerolles is an important landmark in the history of Protestantism north of the Loire River, as is Châtillon-Coligny.

In 1989, during renovation work in the chapel, behind a painting featuring the Assumption of the Virgin, was discovered a text of the Tables of the Law in gold letters on a blue background, along with traces of the Lord's Prayer, and a Huguenot version of the Apostles' Creed (" the holy universal church ").

The chapel was the place of worship for the reformed of Chillieurs-aux-Bois.