"I'm out on the farm myself. That's the thing that will continue to drive me. I was the only farmer in the last government, outside of Damien O'Connor [Agriculture Minister], who was a farmer but had been in Parliament for 27 years. It had been a long time since he'd put the boots on.
"But now I think it's even more important that I help provide that voice. New Zealand is becoming more urban, more liberal. I think there needs to be people who understand farming, for the economy. The number of farmers is not great, it's massively disproportionate, but farming is vital, and it's vital that people understand the issues."
He supported the intent of the freshwater reform "because some things do need to change, but not the impracticality of it" in its application right across New Zealand.
Patterson arrived in Parliament with dirt under his nails.
He is a descendant of the Patterson, Jones, McIlraith and Greenwood families who arrived in the Ellesmere district of Canterbury and settled at Leeston and Southbridge in the 1860s.
The Pattersons took up Springbrook Farm in Lakeside, near the shores of Lake Ellesmere in 1905, of which Patterson was the fourth generation to farm.
It was a tough decision to leave Canterbury 13 years ago and move to Greenbank Farm in Lawrence, but not one he had regretted.
He first became politically aware at high school when the fourth Labour government came to power ushering in the Rogernomics reforms of the mid-1980s.
In his maiden speech in Parliament he said the fallout for rural New Zealand still resonated today, and this was what had got him into politics, first with the National Party, and then New Zealand First in 2014, because of what he saw as the "neglect of New Zealand's regions".
While the father of two was disappointed that the roller-coaster ride of the past three years appeared to have come to an end, he was going to make the best use of the strong contacts he had made in government.
"I've already had farmers ring me [about some of the issues facing farmers] about how they can engage."
He said his low-key, non-adversarial style could have been to his detriment because it had not translated into votes in the newly formed Taieri electorate. But by the same token he had developed good relationships across the political parties.
"I've almost got an obligation to use those contacts because I've got experience that not too many people have got."
On the whole, he had enjoyed the past three years as the only New Zealand First MP in the South Island.
"Being an MP is pretty relentless. People want stuff all the time, so it releases me from that. Everyone wants the chance to be in control of their own destiny. But I'm not going to be quiet at the farm gate. I like the big picture stuff. Local government doesn't appeal at this stage. It's really agriculture that I want to be involved with."
He said politics was not too dissimilar to farming.
"You can never switch off. With politics you never switch off. The only difference is the animals and the people."
One of the biggest advantages of not being at home much in the past three years, and leaving the running of the farm to his wife, Jude Howell Patterson, and a manager, was that he had begun to see the farm from a fresh perspective.
"I'd missed the farm work. For 30 years I'd been on the farm before I got into politics, and I think I'd got a bit stale on it, but now I'm seeing it with fresh eyes."
So many MPs who had been in politics for a long time were not so lucky when they lost their seats.
"If the worst of it is that I'll have to come back here, then I'll be pretty lucky."