That was how Smolda beat him in the Ballarat Cup, Lennytheshark in the Miracle Mile and he even had a speed wobble three wide on the bend in an easy kill four-year-old race at Melton last February.
He has allayed those fears this season, with all three lead-up victories being when he has come from behind or sat parked in his last-start Kaikoura Cup stunner.
But that last win, in unbelievable time, had the extra bonus of terrifying his rivals.
It proved to them Lazarus, after some average workouts to start his season when he was too fat, has now a trimmed-down racing machine similar or even better to last year.
So when trainer-driver Mark Purdon almost certainly goes asking for the lead today, it is impossible to think any of his rival drivers wouldn't be happy to trail him and hope for a tow into a lucrative payday for second.
The horses drawn inside Lazarus wouldn't entertain a challenge while Titan Banner tried that and lost at Kaikoura so would surely trail.
Aussie raider Tiger Tara's trainer Kevin Pizutto has somewhat surprisingly suggested he wouldn't mind trailing the champ.
Meanwhile, Lazarus' Auckland Cup-winning stablemate Dream About Me has been cruelled by a second line draw.
Which leaves the riddle that is second favourite Heaven Rocks.
Their trainers Purdon and Natalie Rasmussen believe Heaven Rocks has a comparable motor to Lazarus but his erratic form this campaign and a roller-coaster week of health make it increasingly hard for Purdon to consider trailing Heaven Rocks should he challenge mid-race.
"Maybe a month ago that looked likely but the way Lazarus has raced since I think it would be hard to trail anything now," admits Purdon.
"Maybe if Heaven Rocks got there early enough we could trail but I think Lazarus could come off his back and beat him anyway."
Although that should cement punters' confidence today at $1.30 fixed, many will struggle to see the point in a win bet so will be keen to find place or exotics value.
Heaven Rocks is an obvious danger but his manners make him hardly an appealing place option so Tiger Tara and Titan Banner may be better options, and Dream About Me can still place if they go hard and luck goes her way.
Jack's Legend has improved with every public outing this season and is a serious place prospect while No Doctor Needed has the draw and driver to sneak into the money or at least boost the First4.
And in case you are considering going shopping for value in the huge win prices around any of his rivals, remember this: Only six horses have beaten Lazarus home in a race and none of them are in New Zealand.
Don't expect that number to change today.
NZ Trotting Cup picks
Lazarus.
Heaven Rocks.
Tiger Tara.
Jack's Legend.
Cup Day
Where: Addington, Christchurch.
When: First race 12.02pm today.
What: 12 races headlined by $800,000 New Zealand Cup; $170,000 Sires' Stakes Final; $100,000 NZ Trotting Free-For-All.
Punt: $40,000 Pick6; final field betting all races; Lazarus out market for NZ Cup; $30,000 starting pool for Cup First4.
Watch: Trackside previews big day from 11am; replay of Box Seat preview show from 9.30am.