How you navigate those two issues makes a substantial difference over what is a long-term investment.
"A modest difference in after-fee returns can generate a large impact for a client's retirement goals. If you think about our performance at Milford as against set-and-forget funds, then our performance over the past 10 years on an after-fee basis can be as much as 5 per cent a year higher. Over many years the effect is substantial," he says.
While historic performance is not necessarily a guide to the future, Swann says you need to ask whether the provider you choose has set themselves up to be near the top of the pack over the long term.
He says Milford has a simple and clear strategy to do this. "We have a formula for navigating the complex universe of multiple markets with a need for diversification. We've got one of the largest and most globally experienced investment teams in New Zealand. There are now 23 people with global experience. We're an active manager. That means we do deep research on where we can find the best companies, the best industries and the best markets."
Swann says his company has an "absolute return" investing style. What this means is that while it looks for growth, it does so in a way that protects a client's capital. In other words, "we're looking for attractive returns at a reasonable level of risk".
At the same time, Milford insists on its team having skin in the game. "We put our clients' and our own money into these investments. All the team at Milford invest in the company's funds. This means we are entirely aligned to our clients' outcomes. We are professionals managing other people's money in a way finance professionals want their money managed."
But whatever the strategy, risk is part of growth investment. "Markets experience volatility," says Swann.
"They can go down substantially. We saw that during the global financial crisis. However, it's about matching your investment strategy to your time horizon. When you've got a time horizon stretching over 20 to 30 or more years, concerns over short-term volatility should not be a driving factor for most investors.
"The tendency for many New Zealanders to not move out of default conservative funds is going to have a huge impact on their lifestyle in retirement."
During the global financial crisis, the markets suffered some substantial falls. Yet Swann says the clients riding through that were still better off in growth funds.
He argues that the Government's approach of using conservative funds as the default for new KiwiSaver members could be changed. Instead, the default fund should be matched to the age of clients going into KiwiSaver.
"In effect, they are giving advice to the public by putting them into conservative funds; it's akin to poor advice.
"You tend to find there's a lot of inertia in default schemes. This is not restricted to KiwiSaver but applies around the world. Yet overseas default funds tend to be set by the provider, not by government, and they tend to opt for balanced funds, not conservative ones."
At the start, many KiwiSaver members don't take much notice of their investment. But Swann says that as balances grow, people start to pay more attention to their retirement savings.
This then becomes a trigger for many people to take another look at their choice of fund, and maybe find a new provider.
The various providers already work hard to encourage people to look at their outcomes.
Now officialdom is joining the education programme with a subtle prod. Swann says the new style KiwiSaver statements, which report fees and returns in dollars, are a step by the industry regulator towards encouraging greater engagement.
He says: "It would be ideal if everyone paid attention to how they were set up for long term savings. The younger you start, the greater the impact of compound interest".