Williams was born in South Australia, a son of
Thomas Williams (c. 1794–1881) and his second wife, Catherine née Codd, who arrived on the
Platina in February 1839. His father was a banker and farmer and was appointed to the
Legislative Council. He was jailed for six months for false pretences and died in England. Williams was educated at
St Peter's College, Adelaide, and entered St Mary's Hall,
Oxford University, but returned to Adelaide without a degree. In 1871 he was ordained by
Bishop Short as a deacon, and as a priest in 1874. In 1874 he was appointed curate of St Peter's Church,
Glenelg, under
Canon Field, remaining there for two years. He was at Melrose for five years 1875 to 1878?, servicing the townships of Laura, Jamestown, Gladstone, Georgetown and Port Pirie. Port Augusta was later added to his responsibilities. He resigned the incumbency in 1879 and was assigned to
St Jude's, Brighton. He resigned that cure in 1881, expecting a transfer to Melbourne but was sent instead to
Hobart,
Tasmania, where he relieved Canon Bailey at the St. John the Baptist Church on Goulburn Street. While there, in 1882, a sermon he gave criticising an article in
The Mercury brought an intemperate response from that newspaper. Williams returned to South Australia shortly afterwards, accepting an invitation to serve as the first incumbent of the Church of the Holy Cross in
Mount Gambier. In 1884 he founded a high school for boys in Doughty Terrace, Mount Gambier, at which many of the town's future leaders were educated. The existing grammar school closed soon after, and its principal, Richard Newstead Hobart (c. 1835–1898), joined with Williams as second master. The Rev. Donald Kerr was another recruit to the school's teaching staff. One of Hartley's students at this time was
Oswald Rishbeth, who would go on to be an academic classicist and pioneer of academic geography. Holy Cross Church closed in 1888 and Williams resigned the ministry around the same time. His exodus from Holy Cross Church, and perhaps from the Church of England, may have been a consequence of friction between himself and church authorities. He was of the
High Church tradition, dogmatic and outspoken in his views. He no doubt took the classes in Latin, of which he was a considerable scholar. and he left Mount Gambier a few years later. The building was for a time used as a private hospital, then may have been used as a girls' school. He moved to
Naracoorte or
Bordertown and may have been involved with a political organisation. He died at
the Grange. ==Other interests==