Williams married, firstly, in 1862 Fanny Florence Caulfeild, daughter of St George Francis Caulfeild of
Donamon Castle, Roscommon, who died in 1876. He married, secondly, Nina Mary Adelaide Sinclair, daughter of
Sir John Sinclair, 3rd Baronet. Temple House was demolished c.1922, or c.1930. A memorial to him was placed in Bisham Church.
The Fast Set Williams accompanied the Prince of Wales on his visit to India in 1875–6. For the purposes of the royal suite for this journey, he had the title of
equerry, with the courtiers
Arthur Ellis and
Dighton Probyn. He also had a role of aide-de-camp, with
Lord Charles Beresford. He was sent on a side trip from Bombay to
Hyderabad state, in November 1875, in response to an invitation from its Prime Minister
Salar Jung I, with
Charles Harbord, 5th Baron Suffield and
Francis Knollys.
William Howard Russell's entry for 3 February 1876, on a visit to an area from
Agra known for wild boar, records that Williams had "got his hand in" at
pig-sticking, as had
Arthur Ellis and
Augustus FitzGeorge. At the beginning of 1876 a potentially damaging scandal blew up, involving Williams's sister Edith, married to
Heneage Finch, 7th Earl of Aylesford: who was in India with the Prince's party. She wrote to her husband in February of that year from
Packington Hall, announcing her intention to run off with
George Spencer-Churchill, Marquess of Blandford. George's brother
Lord Randolph Churchill at the same time wrote to involve the Prince, whom he asked to moderate the reactions of Aylesford and Williams. Aylesford, however, sent a telegram to his mother, asking her to collect the children of the marriage; she had no idea of the circumstances, and
Alexandra of Denmark, who knew what Edith intended, sent her husband the Prince a cipher telegram again asking him to smooth things over. Matters became embroiled since the Prince took Aylesford's side, while Randolph tried to pressure him over the existence of certain letters. Williams wrote to
Charles Yorke, 5th Earl of Hardwicke, who was acting for the Prince in dealing with Randolph, suggesting Aylesford should challenge Blandford to a duel (by then illegal). Aylesford, by then returned from India, met Hardwicke in the House of Lords, and explained that instead he was divorcing Edith. In 1874
Sandown Park had come into Williams's possession, via his brother-in-law Lord Charles John Innes-Ker. The racecourse there was managed by Hwfa Williams, his brother. In 1885, the founding with the backing of
Leopold de Rothschild of the
Eclipse Stakes, offering prize money of £10,000, saw the Williams brothers and their partner Wilford Brett make the reputation of the Sandown Park racecourse. Sandown Park was one of a group of "gate-money" racecourses, within easy reach of London, that included also
Hurst Park,
Kempton Park and
Lingfield Park, that towards the end of the 19th century broke the mould of louche venues with poor security. It was run at a good profit, and had subscribers. Williams was also a racehorse owner, using the trainer John Dawson. He became a member of the
Jockey Club in 1881. He won the
Portland Plate in 1887 with Lisbon. In 1877 Williams bought the yacht
Enchantress from
Joseph Florimond Loubat. He was well known at
Cowes Week regatta, and celebrated for the occasion when in a gale he told his skipper "We will not shorten sail", resulting in considerable damage. Williams was prominent in the
Royal baccarat scandal of 1890–1, centred on allegations of cheating during two evening sessions at a private house party of the card game
baccarat made against
Sir William Gordon-Cumming. In the subsequent slander case, it came out that Williams had acted as a
croupier during the sessions. His major role in the affair was, with
Lord Coventry, to defend the reputation of the Prince of Wales by closing down the allegations, and to that end requiring Gordon-Cumming to pledge that he would not again play cards. As a cover-up, it failed. ==References==