Those who left early yesterday were from Pakistan, it said.
Around two dozen uniformed Turkish police officers lined the boarding plank of the ferry after it docked in the Turkish town of Dikili, accompanied by two Turkish coast guard vessels.
All Pakistanis on board were men, a Turkish official said.
Each deportee was accompanied by a guard from the EU's border agency, Frontex, with a doctor and interpreters also aboard.
"Nobody indicated to our escorts last-minute that they would like to apply for international protection," Frontex spokesperson Ewa Moncure told reporters in Greece.
One man, who called himself Ali, told AFP news agency, "We risked our lives to come here, we don't want to go back to Turkey because they are going to send us back to Pakistan. We don't want to apply for asylum in Greece, we want to go to Germany."
Before the boat left Lesbos, at least two rights activists plunged into the water close to the small ferry. They dangled from the heavy chain of the anchor and flashed the "V" for victory sign in an attempt to prevent the vessel from sailing from the port of Mytilene back to Turkey. They were plucked from the water by the Greek coastguard.
Activists, who say the EU-Turkey deal runs roughshod over human rights, stood at the gates of the port blowing whistles and banging on metal barriers in protest.
A first group of 202 migrants, most from Pakistan and Afghanistan, were sent back to Turkey on Monday.
Turkey's Parliament approved overnight an agreement enabling Ankara to repatriate Pakistani migrants.
Greek authorities said 149 migrants had arrived in the past 24 hours on three Greek islands: Lesbos, Samos and Chios, up from 76 the previous day.
Under the EU-Turkey agreement, Ankara will take back all migrants and refugees, including Syrians, who enter Greece through irregular routes in return for the EU taking in thousands of Syrian refugees directly from Turkey and rewarding it with more money, early visa-free travel for its citizens and progress in its EU membership negotiations. Reuters
