Kaye was appointed
Bishop of Bristol in 1820, and remained there until his
translation in 1827 to
Lincoln. Kaye served as
Bishop of Lincoln for 26 years until his death in 1853. During his incumbency at Lincoln, he instigated widespread improvements to church buildings and
parsonages and established new
Church of England schools. As Bishop of Lincoln, Kaye resided at
Buckden Palace in
Buckden Huntingdonshire. In 1837, the county was transferred to the
Diocese of Ely and Kaye's official residence was transferred to
Riseholme Hall, the newly established episcopal palace at
Riseholme in
Lincolnshire. In 1848, Kaye was elected Visitor of
Balliol College, Oxford. At a time when the
Oxford Movement sought to revive some older Catholic traditions within the Church of England, although associated with the Hackney Phalanx, Kaye's theological inclinations displayed a certain sympathy for
Evangelicalism, and he regarded the
Tractarian movement with suspicion. He was opposed to the revival of the
Convocations of Canterbury and York and was sympathetic to
George Cornelius Gorham's views on
baptism. He reformed the educational requirements for the Anglican clergy and attacked the
Tractarians for betraying the
English Reformation. ==Personal life==