It was created a Burgh of Barony in 1541 and retained that status until 1617. The area is dominated by the ruins of medieval Ardstinchar Castle, once home to the Kennedys of Bargany, sitting at the mouth of River Stinchar. A notable member of the clan who resided there was Hugh Kennedy of Ardstinchar, a captain in the Scots army supporting Joan of Arc. He returned to Scotland with many gifts from the grateful French and was granted permission to incorporate the Fleur-de-Lys in his coat of arms.
The original church of Ballantrae stood on the lands of Kirkholm, at the junction of the Tig with the Stinchar. The church was dedicated to St. Cuthbert, and gave the parish its former name of Kirkcudbright Invertig. In 1604, the laird of Bargany erected a new church in the middle of the present cemetery, and when that church became unfit as a place of public worship, the present parish church was built on a different site.
The current Parish Church was built in 1819 in the Gothic style. In the kirkyard lies the Kennedy Aisle, a remnant of the earlier church which contains the particularly fine Kennedy Monument dating from 1601.
The inhabitants were chiefly fishermen, and in early times the smuggling of tea, tobacco, and brandy formed one of the staple industries of the place.