New Zealand's spell without a victory in England now extends back 14 years. The visitors hopes of any similar reward turned on their heel and exited through the picket fence on the Sunday at Lord's when they were routed for 68.
"That's a fair assessment," McCullum said. "Until that point our self-belief was high. It ripped our hearts out and started to create self-doubt which is a horrible thing in this game. That's what unfolded in this test. We had periods where we dominated but they didn't last long enough."
Grey skies enveloped Headingley all day and drizzle came and went. Ground staff worked urgently to get the match started with England needing just four wickets.
McCullum said they lacked the batting ability to counter Swann's off spin or the swing, seam and bounce of his pace bowling amigos. He agreed the team was mentally worn down.
"There was an element of that. England grew in confidence after that last innings at Lord's and we started to doubt ourselves as a batting group. Once that creeps in it's hard to stop.
"It's definitely a step backwards. We saw England flex their muscle and we failed to respond."
Regardless, McCullum wants to persevere with the top order.
"It's early days but I probably want the status quo. The guys have performed well in patches throughout the summer. If you fast forward 18 months we've got the makings of a good team. I'm a fan of protecting people who have performed for a period and that's what we've seen from this group.
"It is now a big gap for the test side [until Bangladesh in October]. We'd love to play more but we've got to earn the right. It is a touch frustrating we're not donning the whites for a few months."
England captain Alastair Cook said the result vindicated his decision to bat on in the second innings.
"Clearly I woke up and looked out the window first thing but the ground staff did a good job to get us out there.
"Tactically we didn't enforce the follow-on because we thought the wicket was only going to get worse. We wanted time for the footholes to get deeper for Swanny. We wanted to bat New Zealand out of the game so they didn't have a sniff in the series. It is easy when you are sitting behind the rope saying 'I would have pulled out with 350 on the board'."