The James family inherited Penralley House in 1774 from James Jones, who’s family possibly owned the property since 1656. The James family remained at Penralley and developed the estate from wealth acquired by their employment in the Royal Navy. The present house believed to have been built in 1776.

In 1862 Maria James, the youngest daughter of Commander Horatio James RN married Stephen William Williams, the county surveyor for Radnorshire and later an architect.

In 1876 Stephen William Williams extended Penralley House by removing the entire rear section of the house and replacing it with the Victorian yellow brick we see today. This was an enormous engineering project and would be a feat of engineering even with present construction equipment and knowledge.

He also designed a new house and offices in 1893 on the site of the old Penralley cottages, next to Penralley House, which has his monogram above the door, known as Bryniago House.

During the construction of the Elan Valley dams many buildings were submerged including Nantgwilt church. Stephen produced drawings in 1897 from the old church, which were used to construct the present church now next to Garreg Ddu dam. A sculpture of his face can be seen in the eaves of the church with other well-known men of Radnorshire.

Stephen William Williams was involved with the surveying and construction of the Elan Valley dams, in the designs for many churches and large houses in the area. He became the Sheriff of Radnorshire and was admitted as a fellow to the Royal Institute of British Architects in 1886.

Stephen and Maria did not have children so the property was inherited in about 1914 by Brigadier General Horatio James Evans, the eldest son of Maria’s sister.

The graves of many of the families who lived in Penralley House can be found in St Clements church in Rhayader.