Bury, whose family were of Dutch noble origin, had assumed the courtesy title on his elder brother’s death in 1804, ‘in consequence, as was believed in the family, of ill treatment at Harrow School’; this may explain why he was apparently educated privately. He followed in the military traditions of his family and was a midshipman aboard the Superb in 1809, before switching to the army in 1811 and serving in the Peninsula. According to his brother, ‘in one action a bullet passed through his boot near the ankle. In another the rosette which concealed the socket of his feather was carried away - "a feather in his cap" as his comrades used to say’. He was appointed aide-de-camp to the prince of Orange in 1815 and left the army the following year, having been awarded the Waterloo medal.
He was initially a fairly regular attender who, in keeping with his family’s politics, voted with the Whig opposition to Lord Liverpool’s ministry on all major issues, including parliamentary reform, 10 May 1821, 25 Apr. 1822, 26 Feb. 1824. In view of his background, his consistent support for military retrenchment is noteworthy. He divided for Catholic relief, 28 Feb. 1821, 1 Mar., 21 Apr., 10 May 1825. In February 1821 he read a letter from his father to the committee at Brooks’s for the management of the subscription for Queen Caroline, which ‘objected to the plan as doubting the chance of success, from the division of our friends’ on the subject.
Bury’s decision not to seek re-election, announced in October 1824,
I owe in France long-standing debts to the amount of £700 which I am liable to be arrested upon any day and in England about £1,000 ... but of course it is my debts in this country that require the most immediate attention; then the next thing to be considered will be to save me from starvation. I am sorry to say since I wrote to you last ... I have been arrested for 1,000 Fr. and not having one shilling in the world they were taking me off to prison when my servant ... went to his box and paid over 1,000 Fr. ... and thus saved me from the everlasting disgrace that awaited me.
With Albemarle’s own finances in a precarious state, it was left to another member of the family to send subsistence money. Bury had hoped that his wife’s family might assist him, but his fecklessness and lack of tact had soured relations with them. His reputation also hampered his father’s exertions to obtain an appointment for him from the Grey ministry, and the offer of a post in Ceylon was rejected as he could not pay his passage and feared for his health, recently impaired by serious illness.
