Harvey came from a leading family of Norwich corporators who, as a friend reminded the home secretary Peel in 1825 when soliciting a position for his distinguished elder brother, Lieutenant-Colonel John Harvey of Thorpe Lodge, Norfolk, had ‘always been attached to government’.
At the 1826 dissolution he made way for Charleville’s son Lord Tullamore. The following year he offered as the ‘county candidate’ for a vacancy at Maldon, professing hostility to Catholic relief, but he caused ‘a most extraordinary sensation’ by retiring on the ground that he had been lured into standing by ‘representations which he had subsequently discovered to be false’.
