Cavendish continued to sit for Derby on the family interest until, on the death of his uncle John in 1796, he succeeded to the county seat. Until his sons came of age, he was the only Cavendish left in the House. Outside it he was ‘very good humoured’ and talkative, but he did not shine in Parliament. A member of Brooks’s and the Whig Club accustomed to act with the Foxite Whigs, he was listed a supporter of repeal of the Test Act in Scotland in 1791. He adhered to Fox, allegedly against his own sentiments, 13 Dec. 1792, and the Duke of Portland queried whether he could expect Cavendish to follow him in the schism then arising. Not the most reliable of attenders, he spoke against the traitorous correspondence bill, 26 Apr. 1793, and presented his constituents’ petition for reform, 6 May, though he did not vote for it next day. He voted with Fox against the war with France on 17 June. In January 1794 Thomas Pelham noted that although Cavendish’s brother the 5th Duke of Devonshire was very eager for the war, he was inclined to be against it, ‘but I rather think he will not vote against’.
Cavendish was opposed to the imperial loan, 8, 14 Dec. 1796, critical of the conduct of the Bank of England on the stoppage of cash payments, 22 Mar. 1797, and of the Admiralty over the naval mutiny, 10 May. Although he disliked the timing of Grey’s motion for parliamentary reform, 26 May 1797, he voted for it. During the Whig secession he took no interest in promoting county meetings, but attended the House when Fox and his lieutenants did.
Cavendish followed Fox’s line during Addington’s ministry. He was spokesman and teller against the Nottingham election bill, 3 May 1803, sat on the civil list committee and in March and April 1804 joined the onslaught against the ministry, though he spoke only once, 6 Mar., and abstained on Pitt’s naval motion of 15 Mar. He opposed Pitt’s second ministry, but irritated Fox’s allies the Grenvilles by promoting his son’s candidature at Aylesbury in 1804, against their interest. He was named to the select committee to consider the tenth naval report, 25 Apr. 1805. He shared Fox’s misgivings on the death of Pitt. When his friends took office thereupon, he refused a peerage.
When the Whig opposition had to find a new leader in the House late in 1807, Cavendish assured Lord Grey that he was tired of politics, out of touch and despondent: ‘I always act in concert with my brother and in deference to his superior judgment’. He concurred in the choice of George Ponsonby. It was hoped that Cavendish would f’te Ponsonby and thus make him better known among the Whig stalwarts. He attended regularly 1807-8, speaking in defence of Windham’s military plan, but seldom in 1809, when he voted in the minority of 25 Apr. against ministerial corruption, but with ministers against Madocks’s motion on the subject, as did other conservative Whigs, 11 May. He was reported to be disappointed at Lords Grenville and Grey’s refusal to parley with Perceval when the latter became premier in September 1809, but he himself assured Grey that he approved their conduct with one quibble only, that Grey should perhaps have agreed to come to town. At this time Tierney, dissatisfied with Ponsonby’s leadership, was inclined to replace him by Cavendish (or Lord Henry Petty), but the idea was not taken seriously.
Cavendish was reluctant even to attend in January 1810, though he mustered with opposition on the Scheldt question, against Burdett’s imprisonment, 5 Apr., for the release of Gale Jones the radical orator, 16 Apr., and for Tierney’s motion on the droits of Admiralty, 30 May. He did not favour a Whig junction with Canning and on 15 Nov. 1810 he was one of the Whigs who voted with ministers for an adjournment on the King’s illness, and on 13 Dec. one of the committee to examine the King’s physicians; but on 1 Jan. following he was in the opposition majority on the Regency bill. On 18 Jan. he proposed adding the Duke of York to the Queen’s council. He thwarted the Grand Southern Canal bill, 8 Apr. 1811. Although an invariable supporter of Catholic relief, he objected to the British and Irish militias interchange bill which he thought would make militia service unattractive, 23 May. On 14 Jan. 1812 George Ponsonby gave notice that Cavendish would move for an inquiry into the state of Ireland, but the death of his son William incapacitated him and it was undertaken by Lord Morpeth. He returned to the House to vote for Catholic relief, 24 Apr., and for a stronger administration, 21 May 1812. He had deplored the Prince Regent’s pusillanimity in deserting the Whigs on the expiry of the Regency restrictions and had to overcome a strong inclination to retire from the political arena. He was inconspicuous in the first two sessions of the Parliament of 1812 and Lord Holland reported, 18 Dec. 1813, that he was one of those who would actually defend ministers if their war effort were impeded in the House.
Cavendish rallied to opposition when hostilities were resumed against Buonaparte in 1815: he informed Fitzwilliam, 31 Mar., that he sided with Grey against Grenville on this issue, believing that Buonaparte should be overthrown by his subjects, not by exterior force. On 25 May he moved an amendment to the Regent’s address, critical of the allied war aims. It was defeated by 331 votes to 92. The party leadership was impressed by his eagerness to contribute towards the payment of Tierney’s debts in September 1815, when he had already paid half the price of Tierney’s seat in that Parliament.
On George Ponsonby’s death in July 1817, Henry Grey Bennet wrote to Creevey:
Who is to lead us now? God knows. Some talk of Lord George Cavendish, whom I resist, because I think his politics are abominable and his manners insolent and neglectful: but also because the Cavendish system, with the duke at the head is not the thing for the present day. They are timid, idle and haughty.
Cavendish was averse to taking the lead: in fact, he was one of those who thought it ‘foolish to talk of choosing any leader’.
Cavendish readily supported the requisition to Tierney to lead the opposition in July 1818. He himself was as lax as ever: he did not come up to town until 2 Feb. 1819, and then too late to take the oath and vote that day.
Cavendish accepted a coronation peerage in 1831, without consulting his nephew the Duke of Devonshire.