Pandev doubled his tally 10 minutes later as Lorenzo Insigne lifted a superb ball into the area for the Macedonia striker to smash across goal into the bottom left corner, with his weaker right foot.
Napoli, which lost its perfect record in Wednesday's 1-1 draw at bottom club Sassuolo, moved a point ahead of Roma, which hosts Bologna on Sunday.
"We really wanted these three points after the awful night against Sassuolo," Pandev said. "We got over it really well."
Coach Rafa Benitez rotated his squad, with key players Gonzalo Higuain, Marek Hamsik and captain Paolo Cannavaro starting on the bench.
Genoa has lost four of its opening six games, raising speculation its young coach Fabio Liverani could be the first Serie A manager to be fired this season.
It has had only one worst start to the season, 30 years ago.
There were signs of improvement in the second half, however, and Genoa almost got one back in the 64th when Alberto Gilardino's header flashed just past the left post.
It also had strong claims for a penalty turned down three minutes later when Cannavaro brought down Gilardino in the area, but the Genoa striker was harshly flagged offside much to the displeasure of the home fans.
Milan coach Massimiliano Allegri had castigated his team for wasting too many chances after it again needed two late goals to rescue a 3-3 draw at Bologna midweek.
Forward Mario Balotelli sat out the second of a three-match ban, while Stephan El Shaarawy, Riccardo Montolivo, Mattia De Sciglio, Kaka, Giampaolo Pazzini and Matias Silvestre are still out injured.
Milan had been ordered to keep one end of San Siro the Curva Sud closed as punishment for discriminatory chants during the loss to Napoli. Fans responded by holding a protest outside the stadium with banners further insulting the Southern city of Naples.
There were also more anti-Neopolitan chants inside the stadium and a warning announcement had to be made by stadium officials. Milan could face further punishment.
Milan's profligacy in front of goal continued against Sampdoria, which had only two points heading into the match.
Kevin Constant was close to giving Milan the lead in the 17th minute but he volleyed Robinho's cross just past the left post. Phillipe Mexes was also inches away from connecting with a cross right in front of goal which would have almost certainly put Milan in front.
It was Samp which had the best chance of the first half, however, as Andrea Costa headed a corner just the wrong side of the right upright.
Milan was booed off the pitch at halftime but broke the deadlock 40 seconds after the restart when Robinho found Birsa for a fine low strike into the bottom right corner.
And it should have doubled its advantaged moments later. Birsa's effort was parried by Samp goalkeeper Junior da Costa but the ball eventually came to Robinho, who incredibly missed an open goal from a meter out.
Forward Alessandro Matri has yet to score since signing from Juventus in the offseason and he had a shot saved by Da Costa when one on one with the Samp goalie.
M'baye Niang should have secured the points in stoppage time with a low, angled drive which Da Costa managed to get a hand to, to deflect just past his right upright.
Both Napoli and Milan are in Champions League action next week, against Arsenal and Ajax respectively.
