In fact, they're all around us - but not as hidden as you might think.
There are many famous freemasons, such as astronaut Buzz Aldrin, actor Michael Caine and even early Wairarapa settlers, including Joseph Masters.
Freemasonry provides rich pickings for conspiracy theorists - perceptions range from an occult organisation to a group conspiring with aliens to take over the world.
These perceptions have been boosted by pop-culture depictions.
Dan Brown's books The Da Vinci Code and The Lost Symbol, for example, pushed Freemasonry into the light and with it, tales of strange rituals and hidden agendas.
"There are no albino dwarfs in our lodge," laughs Mark, referring to one of the characters in the books.
In the books, there are some sinister perceptions of Freemasons running the US Government.
Hardly sinister however, is one of their fundraising activities in Wairarapa - selling berries and icecream at a fair at St Luke's Church. This is primarily what they do - they are a charitable organisation. The perception also isn't helped by a past policy which required members to be discreet about the organisation and their involvement.
But times have changed.
"There's nothing that says you can't talk about Freemasonry and all of us are quite happy for people to know (we are members)," says Julian. "There are aspects of it that we can't tell people but that is a very, very small portion."
This part is to do with the members' secrecy around the ceremonies which are based on its medieval roots.
Maintaining the secrecy of the ceremonies is basically to test the good character of those who join. They also have secret handshakes between members which identify what degree they have.
The traditions originated in the Middle Ages, when people were predominantly illiterate.
Symbols such as the square and compass, modes of dress, handshakes and signs were used to distinguish a stonemason from a non-mason to protect the highly prized technical know-how of their trade.
The masons formed guilds to work on large projects and naturally these became social places.
These days, the handshakes, outfits and signs are used solely for ceremonial means at lodge meetings.
Even members don't know all the ceremonial secrets.
Mark, who has the degree of Master Mason and position of Junior Deacon, will never get to know what Julian's ceremony was like. And he says he wouldn't want to know - it would ruin the experience if he ever became Master.
He says the organisation doesn't really help itself when it comes to the secret society perception because they don't publicise the charitable work they do, a principle of being a Freemason.
They say becoming one is not dictated by bloodlines or money - it's not an old boys' club for rich white businessmen.
"I've had times where people thought that they wouldn't be able to be a Freemason because they don't own a business and I'm standing there, look at me," says Julian, who is also a tradesman.
He says many people believe it is an elitist organisation, but there is no judgment, another principle of being a Freemason.
If they want to join, all they have to do is ask, he says.
Julian did it in this manner, joining seven years ago when he met a Freemason and expressed an interest.
Mark works for South Wairarapa District Council. He has been a member for three years and his great-grandfather, grandfather and father were members.
Both men say they were attracted by the sense of camaraderie and wanting to be part of a group based on principles of being a good, upright, moral person.
Mark says this aspect of the brotherhood can be seen in an episode of The Simpsons, where Homer joins The Stonecutters Guild, a satirical take on the organisation.
Mr Burns, the malevolent millionaire who owns Springfield's nuclear powerplant, is seated among everyone, eating, drinking and singing - not at the head of the table.
And this rings true of Freemasons - you leave your job at the door, says Mark.
Says Julian: "Everyone's on the same level, you walk in that door it doesn't matter who you are."
Religion and politics are left out although the Freemasons NZ code says members must believe in a "supreme being".
"With all things it's based around principles of how you live your life, but it's not any specific religion," says Mark, who has been to lodges that have Islamic, Christian and Sikh members.
Because it is an international organisation, members are welcome at lodges around the world. "Wherever you go in the world it's like a global club."
Members meet monthly to socialise and discuss fundraising. They try to help their communities, particularly the young and elderly, raising money through raffles, auctions, donations and other activities. Julian says no one is expected to make donations - men of any means are welcome.
Over the years, Wairarapa lodges have provided university and other kinds of scholarships to young people, including women. The overarching Grand Lodge often matches regional lodges' donations and, collectively, the organisation has provided opportunities to many. They have just given a scholarship together with NZ Opera to emerging 24-year-old singer Emily Scott.
In Wairarapa, Freemasons spent several years lobbying for funding for accommodation for the disabled and pensioner housing before taking it into their own hands and creating the Glenwood Masonic Hospital and Resthome. It is not only about giving money, they also do things for no other reason than to help - true charity, Mark says.
"We don't seek glory for what we do, it's just what you do," says Julian.
They admit they do get something out of it - feeling good. Helping and supporting members, members' widows and families is a key part of being in the brotherhood. For example, someone might offer to chop and stack firewood for a member's widow, or a member who is a farmer might give families surplus produce.
Mark says sometimes the idea of charity is met with scepticism. "It's never a reciprocal thing, if you get something for free there's normally a catch to it."
But by being in a group that values this, he says they contribute to keeping it alive in society. "You wish people acted like you're supposed to do anyway, helping people in distress, not judging people, these are all values that everyone should have."
Both men say being a Freemason has helped them grow as people and they look up to their fellow members. Despite the fraternal nature, "family comes first, work second, and Freemasonry commitment to your lodge comes last", says Julian.
So will women ever be able to join? Julian is frank - no. It would be difficult as it would require every lodge in the world to agree to it.
The original constitutions were published in 1723 and are fundamentally what Freemasons all over the world are united by and adhere to. Julian says there are female equivalents such as the The Order of the Eastern Star.
With five million Freemasons worldwide and 10,000 in New Zealand, the 60 members of the Martinborough-based lodge represent just a sample of the "secret society" but from meeting these members, they don't appear to be plotting to take over the Beehive.
