There are about 80 owners who have property at the site and all are shareholders in HVML, which maintains the village for the benefit of all owners.
But some owners say they are being treated unfairly by directors of the management company and say a vote of no confidence is needed.
The HVML board of directors said there had been negative attention brought to the village recently by a minority group and HVML would not be partaking in any media attention that could further escalate damage already done.
Shane Ayers has owned about two acres (8000sq m) at the site for about 16 years and has recently built a new home there.
He said even before building began, HVML managing director Kylie Scott got into a yelling match with a digger driver who was attempting to begin excavation work by knocking out a gutter where the driveway would be going.
''She started coming out screaming at the digger driver.
''That is just the start of it. The rest of it is far out — everything from going to the council behind my back to a petition to get me voted off the board.''
Ayers said he spent about five months on the board but was voted off after a secret petition was sent to shareholders saying he was not a fit and proper person to represent the shareholders.
''It is just all about power,'' he said.
Jennifer Roger owns about an acre (4000sq m) and with her husband Peter Roger rents a top floor flat in what was one of the former Cherry Farm buildings.
She said the directors had engaged in harassing and bullying tactics, and were quick to issue charges for perceived infringements while dragging their feet on repairs such as removing tree roots that would cause problems for pavements.
Shahma Smith lives in one of the body corporate units on the property.
She said one of the directors searched her property without asking, and when she attempted to put herself forward as a candidate to be an HVML board member her campaign was undermined by a letter circulated to other shareholders discouraging her application.
Kathy Wintrup is another owner who says a vote of no confidence in the board is needed.
She said the board of directors' leadership style was based on intimidation, corruption and harassment.
''For me personally, it has reached the stage where I no longer feel safe or confident to walk about the village,'' Wintrup said.
Other shareholders said a small number of owners made life difficult for the rest.
The Star has been contacted by other shareholders who are concerned about the behaviour of Mr Roger, whose wife is a shareholder.
When asked by The Star for a response, Mr Roger said he could be confrontational, but he was defending his patch.
The HVML board was ''stealing money'' from his wife by issuing charges for perceived infringements that were out of proportion, he said.
The HVML board of directors said the annexure schedule allowed shareholders to pursue arbitration should they wish, but this had not been requested by any shareholders.
Requests by HVML to shareholders were very basic and easily comprehensible and they had compliance by 95 per cent of their shareholders, they said.
Requirements for the benefit of all shareholders included that levies be paid on time; no damage be done to HVML infrastructure or land; no personal items or rubbish be left on HVML land; all accidental damage be repaired; and vehicle crossings through HVML land be completed in a timely manner.