Luttrell continued to represent the family borough, giving an indifferent and silent support to administration. He showed no more interest in politics after 1790 than before, seeking only to maintain his hold on Minehead (where he lost one seat in 1796, but regained it in 1802) and expecting only the assurance of local patronage and preferment for members of his family.
Pitt could count on his support, but not on his attendance. He paired with ministers on the Oczakov question, 12 Apr. 1791. That month he was reckoned hostile to the repeal of the Test Act in Scotland. He took leaves of absence of a month on 9 Dec. 1795 and 15 Dec. 1796, and again, 10 Mar. 1803, for health reasons. His diaries showed that he was in the House in April 1805, but he left no mark upon its records. He was again being listed a supporter of Pitt. On 29 Mar. 1808 he obtained leave for illness. The Whigs listed him ‘doubtful’ in 1810 when he rallied to ministers on the address, 23 Jan., and on the Scheldt expedition, 26 Jan. and 5 Mar. He was in the government minority against a more efficient administration, 21 May 1812. He was listed as a government supporter by the Treasury in 1812, but wrote to Lord Liverpool, 10 Apr. 1813, to complain that his brother Francis had not been appointed chairman of the board of customs.