Howard was styled Viscount Andover from 1800 when, on the death of his elder brother in a shooting accident, he became heir to the earldom. His family and personal connexions were Whig and in December 1801 report had it that he would be returned for Carlisle at the next election by the Duke of Norfolk. His father would have liked him to come in for Cricklade Hundreds, on the family interest, but found that his prospects there were poor.
Andover’s only known vote against Addington’s ministry was on the resumption of hostilities with France, 24 May 1803. (He had taken a month’s leave for illness on 9 Mar., after serving on the Bridgwater election committee.) He was, however, ‘one of the first to offer his services to repel the foe’, becoming a commandant of volunteers. Listed ‘Fox’ in March and May 1804, he opposed Pitt’s additional force bill, 8 June, being locked out when intending to do so again on 11 June.
Andover contested Cricklade at his father’s instigation in the general election of 1806. He had made his intentions clear a year before. He was defeated by a fellow Whig, Lord Porchester. He was invited to stand again in 1807, but declined on his father’s advice. In June 1811, offered a favourable opening at the next election, he announced that he was no longer interested. His politics remained unobtrusively Whig. He was a ‘good man’ and ‘well liked’. Agriculture was his chief interest, and ‘his appearance and usual costume was that of an ordinary farmer’.