Ferguson, a close friend of Henry Dundas, was, through the name of his estate, the subject of one of the 4th Duchess of Gordon’s riddles:
My first is found upon the banks of Tyne,
My second is scarce quite half of nine;
My whole a laird of Aberdeenshire race,
An honest fellow with an ugly face.A. and H. Tayler, Jacobite Exile, 202.
Returned for Aberdeenshire in 1790 in accordance with the compromise negotiated by Dundas, he was absent, supposed doubtful or hostile, from the division on the Test Act, 10 May 1791, but voted against the abolition of the slave trade, 15 Mar. 1796, and for the assessed taxes augmentation bill, 4 Jan. 1798. When Alexander Leith Hay solicited ministerial support in Aberdeenshire at the next election in May 1800, Dundas told him that ‘the long and uninterrupted personal friendship which has subsisted between Mr Ferguson and me, joined to his zealous support (under trying circumstances) of the government’ put it out of the question. Dundas subsequently told Ferguson that ‘I hold it to be the first of all political maxims never to abandon a tried friend, and Mr Pitt, with whom I have conversed on the subject, concurs with me’. For all this, a conversation with the lord advocate gave one of Hay’s supporters the impression that ministers were really ‘indifferent’ about Ferguson’s success and it was reported that even Dundas was ‘heartily’ amused by his anxiety over his seat. Yet on 2 Oct. 1801 Dundas pressed Addington to comply with Ferguson’s patronage requests, especially as he had received ‘a pretty hard knock lately’ over one such.
He comfortably beat Hay at the general election of 1802, when Charles Innes described him as ‘friendly to Mr Dundas, but altogether independent’ and forecast that he would support Addington even if Dundas went into opposition. In a contemporaneous list in the Melville papers he was placed among the ‘partisans’ of Pitt and Dundas and this judgment proved the more accurate, for he voted with Pitt for the orders of the day, 3 June 1803, joined in the combined attack on Addington in 1804 and supported Pitt’s second administration, voting against the censure of Melville, 8 Apr. 1805.
Ferguson, whom Lord Grenville described in 1806 as ‘one of Lord Melville’s most devoted adherents’,
Ferguson voted with government on the Scheldt inquiry, 5 and 30 Mar. 1810, when the Whigs classed him as ‘against the Opposition’. He opposed the release of John Gale Jones, 16 Apr., and parliamentary reform, 21 May 1810, and was in the government minorities on the Regency, 1 Jan. 1811, the sinecure bill, 4 May 1812, and the call for a remodelling of the administration, 21 May. He voted against Catholic relief, 22 June 1812, 2 Mar. and 24 May 1813 and 21 May 1816. In January 1811 he pressed Melville, to whom his prime loyalty still lay, to resume an active political role.
After Ferguson’s death, it was alleged that his only utterance in the House was a request for the window behind his seat to be repaired,
With the passage of time Ferguson, who was noted for his dry, self-mocking humour, became a popular figure at Westminster and the subject of many anecdotes. Addington recalled how
one day Pitfour, with several others, was taking his dinner in the coffee room of the House, when someone ran in to tell them that Mr Pitt was on his legs. Everybody prepared to leave the table except Ferguson, who remained quietly seated. ‘What!’ said they, ‘won’t you go to hear Mr Pitt?’ ‘No,’ he replied, ‘why should I? Do you think Mr Pitt would go to hear me?’ ‘But indeed I would,’ said Mr Pitt when the circumstance was related to him.
Pellew, Sidmouth, i. 153-4.
In 1817 Farington heard from Lord Lonsdale Ferguson’s story of his mother’s parting advice to him when he left Scotland:
Never expose yourself, James, to be tried for a rape, for your broad shoulders will cause a jury to think it probable that you made the attempt, and your face will make it manifest that it must have been against the will.
Farington, viii. 136.
Raikes, reflecting in 1837 that parliamentary eloquence could not sway people whose political views were fixed, recalled that Ferguson ‘was wont to say that he had heard very many fine speeches in his time on baith sides of the question, and on coming down to the House he had vary often changed his opinion, but naver his voted’.
