When Henry Hall died in 1875 he left the house, known then as Lansdowne Villa, and its contents to his youngest sister, Clara Ellen. Clara was the widow of architect, Charles Swyer and she renamed the house Worsley after the place in England where he had died. Upon her death Clara left the house to her unmarried daughter, also Clara (Swyer), who lived there until 1922 when she sold it to Mrs. James Wood. In late 1929 the Sisters of Mercy bought Worsley for 4,000 pounds and it was demolished four years later to make way for the Mercy Private Hospital.

Henry Hall was a chemist with premises in Bourke Street near the corner of Russell Street. He was one of Melbourne's early pioneers having arrived here in 1840 at the age of twenty-two.