On 18 October 1744 Carteret became 2nd
Earl Granville on the death of his mother. His first wife,
Lady Frances Worsley, died on 20 June 1743 at
Hanover, and in April 1744, he married Lady Sophia Fermor, a daughter of
Thomas Fermor, 1st Earl of Pomfret and a fashionable beauty and "reigning toast" of London society who was younger than his daughters. Granville's ostentatious performance of the part of lover was ridiculed by
Horace Walpole as "The nuptials of our great
Quixote and the fair Sophia" and "My lord stayed with her there till four in the morning. They are all fondnesswalk together, and stop every five steps to kiss". The Countess Granville died on 7 October 1745, leaving one daughter, Sophia Carteret, who married
William Petty, 2nd Earl of Shelburne, later 1st
Marquess of Lansdowne. Granville's second marriage may have done something to increase his reputation for eccentricity. In February 1746, he allowed himself to be entrapped by the intrigues of the Pelhams into accepting the secretaryship but resigned within forty-eight hours. In June 1751, he became president of the council and was still liked and trusted by the King, but his share in government did not go beyond giving advice and endeavouring to forward ministerial arrangements. In 1756, he was asked by
Newcastle to become Prime Minister as the alternative to Pitt, but he perfectly understood why the offer was made and so he declined and supported Pitt. When, in October 1761, Pitt, who had information of the signing of the "
Family Compact", wished to declare war on
Spain and declared his intention to resign unless his advice was accepted, Granville replied that "the opinion of the majority (of the
Cabinet) must decide". He spoke in complimentary terms of Pitt but resisted his claim to be considered as a "sole minister" or
Prime Minister. Whether he used the words that were attributed to him in the
Annual Register for 1761 is more than doubtful, but the minutes of Council show that they express his meaning. ==Marriages and issue==