The sisters say the men enjoyed having young girls around and their mother, considered a left wing radical and admired feminist, encouraged their joining in the libertine sex scene of the times.
"We were these nubile girls … jailbait," Kate Lilley said, describing her mother's house as "a brothel without payment".
Other men included a local film producer who is still alive and a poet who raped Kate Lilley when she was underage, plus renowned British erotic photographer, David Hamilton.
Known for his soft-lit images of prepubescent young girls, 83-year-old Hamilton took his own life in 2016 after a former model accused him of raping her in 1987 when she was 13 years old.
In the Woollahra terrace Hewett shared with her husband, writer Merv Lilley, a queue of famous men attended parties in the salon-like atmosphere where the sisters spent their early teens.
Sex was facilitated by Hewett who also slept with some of the men, while her husband had sex with other women.
When she turned 16 years old, Kate Lilley had already slept with six men and her younger sister Rozanna had slept with a dozen men by the time she reached the age of consent.
The men were older artists and writers, including Bob Ellis and Martin Sharp, and Hamilton who photographed pornographic images of Rozanna.
Rozanna said she had sex once, when she was 15, with Martin Sharp, who decades later apologised to her.
David Hamilton took close photographs of her genitals after showing her pornographic images of girls to prove others had consented.
"It was unbearable at home," Kate Lilley said, "I used to have sex with men to prevent them having sex with Rosie.
"But then I would find out they did have sex with Rosie."
Kate Lilley's teenage years are referred to in poems in her new book, Tilt, which the 57-year-old pet and university English professor has published.
Rozanna, 55, an anthropologist and autism researcher also writes poems about sexual abuse by (unnamed) older men in her book, Do Oysters Get Bored? A Curious Life.
Kate Lilley told The Australian she had sex four times with Bob Ellis when she was aged 15 and 16, "whenever he turned up".
Rozanna has written a poem about her sexual encounter with Ellis as a child.
At the age of 13, Rozanna Lilley was cast in the lesbian-themed film Journey Among Women about female convicts living in the bush.
Her mother, Dorothy Hewett, wrote the script.
The film is still available on DVD.
Tilt by Kate Lilley is published by Vagabond Press, $24.95.
Do Oysters Get Bored? A Curious Life by Rozanna Lilley, by UWA Publishing, $29.99.