Chaytor was the eldest son of Sir William Chaytor, who represented Sunderland as a Liberal, 1832-5.
Following the failure of his father to secure government backing to replace him at the 1832 general election, Chaytor offered again at Durham City.
In November 1834 Chaytor’s father wrote to Lord Durham, offering to transfer his Durham interest to him, stating that ‘my eldest son ... does not feel inclined to stand again for the City partly on account of ill health during his residence in London and his dislike of it when there’.
