He'd also noticed there weren't as many people fishing at the popular spots, like Lake Ferry.
"There's a little bit less people than usual, compared to last year I remember 40 to 50 people at the lake on some days.
Mr Silbery said conditions for the native fish were not ideal this season.
"We did have a pretty tough spring, a lot of rain around and the rivers are muddy."
Whitebait prefer clean, clear water.
One Carterton whitebaiter struck white gold however.
Diane Laing, who has been whitebaiting for about 50 years, had only been out once but it was her lucky day.
She wouldn't reveal how much she caught but it was "enough for a few whitebait fritters".
"We had a very good day, no muck fish, it was probably the luck of the draw, everybody's got different theories."
Mrs Laing believed there were still quite a few whitebait around the region, even though the mouth of the lake had been closed a few times.
"They say whitebait are getting less but it depends on the season ... they got an awful lot in the Waikato and on the West Coast. It hasn't been a good season on the river though."
Mr Silbery said he hadn't handed out any fines so far.
"Everybody's behaving, it's generally been good."
He said the season ends on November 30.
"There are still a few days left in the season for the keen and dedicated to get out there."
All whitebait species spawn in autumn, and around one month later, newly hatched larvae are washed to sea where they spend their first six months.
The annual spring migration, so eagerly anticipated by fishers, is when the juvenile fish leave the sea to return to freshwater rivers and streams.
