Rolando extended Inter's lead 12 minutes after the break, bundling the ball home from practically on the line for his first goal for the club since joining on loan from Porto.
Romula fired Verona back into the game in the 71st minute and Toni thought he reduced the deficit again five minutes later but his effort was harshly ruled out for a perceived foul on Inter defender Andrea Ranocchia.
"We were naive at times, which we shouldn't have been," Mazzarri said. "We should never let our guard down in football and never think we have already won.
"We had a great game up to a certain point but if we want to be one of the top teams we should never take our foot off the gas."
Inter forward Ishak Belfodil and Verona defender Vangelis Moras were shown red cards after the final whistle for an off-the-ball scuffle.
"We made some silly mistakes and paid a heavy price for them," said Verona coach Andrea Mandorlini, who spent seven years at Inter as a player. "We conceded three goals on set plays and were too soft in those defensive situations. You can be unlucky if it's one, but that many in one match means you aren't focused enough.
"Aside from how well we played otherwise, the fact remains we can only blame ourselves for conceding those goals. We can do better, but above all must learn we can't keep making mistakes against the top clubs. We paid for those against Roma, Juventus and now Inter."
Inter moved provisionally six points behind leader Roma, which has won its opening eight matches and plays Udinese on Sunday.
Earlier, Sampdoria picked up its second victory of the season with a 1-0 win over 10-man Atalanta.
Atalanta, which had won three matches in a row, should have taken the lead on the stroke of halftime but Cristian Raimondi headed off the crossbar.
Shkodran Mustafi headed Sampdoria in front 10 minutes after the break. Atalanta midfielder Luca Cigarini tried to hit the ball away from goal but couldn't prevent it from going in. The referee initially showed him a red card and awarded a penalty but then opted to allow the goal and keep Cigarini on the pitch.
Atalanta was reduced to 10 men seven minutes later when substitute Constantin Nica was sent off in only his second Serie A appearance after bringing down Eder.