Fife combined moral laxity and financial extravagance with seriousness of mind and concern for the welfare of the tenants and labourers on his extensive estates in north-east Scotland. His pecuniary difficulties were worsened by the extraordinary will of his uncle James Duff, 2nd Earl Fife (1729-1809), who left all his disposable property to his bastard son Sir James Duff of Kinstair (1755-1839). Fife, who succeeded his father, the 3rd earl, to the entailed estates in 1811, went to law in 1816 in a bid to recover his uncle’s property, which was reckoned to be worth over £20,000 a year, with timber valued at £100,000. The costly case dragged on in the Scottish courts and the Lords until 1826, when Fife was finally successful.
Although Fife, an old personal friend and roistering companion of George IV, held a place in his household from January 1819, he was a law unto himself in electoral matters. At the general election of 1820, when he was again returned unopposed for Banffshire, he created great difficulties for the Liverpool ministry and their leading local supporters in Elgin Burghs, where he was seeking to re-establish his family’s interest: his younger brother Alexander Duff stood unsuccessfully against the candidate of the prevailing Kintore-Grant alliance.
We have had a brisk campaign, which is likely to be prolonged, but I hope from the complexion of things to be some time in the North as it is thought it will not come to our ... House to meet for the queen’s business for some time. The ferment seems to be less, although all means are tried to keep up the heat and increase the progress.
Aberdeen Univ. Lib. Duff of Braco mss 2727/2/82.
He did not vote in the division on the opposition motion censuring ministers’ conduct towards Caroline, 6 Feb. 1821. He was absent from the division on Catholic relief, 28 Feb. It became common knowledge that when he asked the king how he wished him to vote on the relief bill he was told to do as he pleased. He apparently divided against the second reading, 16 Mar., but stayed away from the division on the first clause, 23 Mar. 1821.
By then he had drawn attention to himself by voting in a majority against government on the 21st for repeal of the additional malt duty, on which feelings were strong in Scotland. He was immediately dismissed from the bedchamber. According to Mrs. Arbuthnot, the king’s subsequent claim that he had ‘sacrificed his own feelings’ in agreeing to Fife’s removal was deemed to be a lie by the duke of Wellington, who
happened to know that the king wanted to get rid of him, that he was delighted with the opportunity offered, and that the ministers had interposed to induce him to do it in the regular way and not to wound Lord Fife’s feelings by doing it at the drawing room as he had wished.
Grey Bennet diary, 44; Greville Mems. i. 116; Lady Holland to Son, 5; Buckingham, i. 143; Arbuthnot Jnl. i. 98.
On 3 Apr., when ministers mustered their forces to reverse the vote, Fife, responding to an allusion to his dismissal for showing ‘independence’ by the Whig Lord Archibald Hamilton, stated that he had ‘a considerable time ago’ informed the king of his wish to retire, ‘from his inability to attend regularly to the duties, and to ... prevent disagreement - owing much to being obliged to watch over the interests of a vast number of people, under circumstances cruel and vexatious’. He claimed that it was ‘a satisfaction to be released ... for various reasons’, but revealed that he ‘had it from authority’ that ‘the resolution communicated so abruptly was considered as a reprimand’ for his vote, though no notice had been taken of his minority vote on the same issue on 5 July 1820. He ‘did not repent’, believing the tax to be ‘impolitic and unjust in principle, baneful in the effects, dangerous in the result’, and he again divided against government. The radical Whig Hobhouse later claimed that when Fife ‘begged’ him not to make capital of his statement of the official reason for his removal he complied. On 6 Apr., however, he, Creevey and Grey Bennet pressed for inquiry, but they failed to secure the support of the Whig leadership.
In April 1821 it was reported that Fife, a widower since 1805, was ‘going to console himself for the loss of entrée to the king’s bedchamber by taking Lady Georgina Bathurst into his own’. Lady Bathurst was supposedly distraught at the prospect, but her daughter never married.
spent a fortune upon her; his presents in jewels, furniture, articles of dress, and money, exceeded £40,000. In return for all this generosity, Lord Fife asked nothing more than the lady’s flattery and professions of affection ... On his return to London, the old roué would amuse George IV with a minute description of the lady’s legs, and her skill in using them. Horses’ legs are frequently the cause of the ruin of members of our aristocracy, but in the case of Lord Fife, the beautiful shape of the supporters of Mlle Noblet had such an effect upon the perfervidum ingenium Scoti, that he from first to last spent nearly £80,000 on this fair daughter of Terpsichore.
Gronow Reminiscences, i. 121, 303.
In 1822 Fife introduced to the London stage the ‘very pretty’ sixteen-year-old opera dancer Maria Mercandotti, who was believed by some to be his daughter. She was pursued by many men, but in 1823 ran off with the wealthy wastrel Edward ‘Golden’ Ball Hughes, whom she married at Duff House, with Fife and her mother in attendance, on 22 Mar.
In April 1826 he was in line for a British peerage, on the king’s insistence to Lord Liverpool:
I am quite aware of some of the trifling objections to some of the fooleries of his past life, but who is exempt from some nonsense or other? I dismissed him from my household, and used him apparently ill to please my government and poor Lord Londonderry; but, notwithstanding this, my friend Fife never gave a vote against the government afterwards, and by his loyal example when I was in Scotland did the greatest good.
Geo. IV Letters, iii. 1237; Add. 38371, f. 165; 38576, f. 99.
It was expected that his promotion would make it unnecessary for him to stand for Banffshire, where he faced an opposition, at the general election in June; but in the event it was delayed, and he was returned after a contest. At a celebration dinner he alleged that the ‘clannish’ spirit which had been raised against him had been provoked by his vote on the malt duty and his restoration of his interest in Elgin burghs (where his brother now came in), but that ‘both these acts ... had increased the confidence which His Majesty and government reposed in him’.
Fife, who did not take up permanent residence at Duff House until 1833, but rarely left it after 1838, was a benefactor of the poor, a keen promoter of local public works and an enlightened, improving landlord. He created new settlements at Aberchirder and Dufftown. In 1847 he escaped injury in a knife attack by a drunken servant with a grievance.