Kynaston continued to be returned unopposed for Shropshire and, when present, to give a general support to administration. Only one speech by him is known: on 23 Dec. 1790, favouring the continuation of the impeachment of Warren Hastings and suggesting a statutory regulation of future impeachments, since the House was so divided on the subject. On 3 Apr. 1797, after a division on it, he was excused attendance on account of militia duties. On 18 May 1798 he voted with opposition for the deferment of the land tax redemption bill and for Buxton’s clause in amendment of it. He had succeeded to his mother’s family estates and developed the ambition of securing the revival of the barony of Grey de Powis in his favour.
He was evidently not a friend of the Grenville ministry and voted against them on Ellenborough’s seat in the cabinet, 3 Mar. 1806, and on the Hampshire election petition, 13 Feb. 1807. Thereafter he took periodic leaves of absence for ill health. The Whigs were ‘doubtful’ of him in 1810, in which year he appeared in the government lobby on the address and the Scheldt expedition, 23 Jan., 5 and 30 Mar.; as also on the Regency, 1 Jan. 1811. He was listed a government supporter in 1812 and appeared in no minority list in that Parliament except on 2 Mar. 1813 when he was opposed to Catholic relief. He voted with ministers against Admiralty retrenchment, 25 Feb. 1817, and on their employment of informers, 5 Mar. 1818. He was encouraged and consoled in 1818 with a baronetcy.