Leaders of other member countries said the real fight was not with the EU but within May's fractured Conservative Party.
It is "not up to the European Union to make concessions to deal with an internal British political matter", French President Emmanuel Macron said. "It's a matter concerning Britain's political ability to find a presentable agreement. That's all."
At the end of the summit, May said there would be "tough negotiations", but she remained confident Britain and the bloc could strike a divorce deal that worked for both sides.
EU Council President Donald Tusk said the summit also left him more optimistic than he'd been after a bad-tempered meeting last month and a negotiating crisis on Sunday.
"What I feel today is that we are closer to the final solutions and the deal," Tusk said. He acknowledged the feeling "may be a more emotional impression than a rational one". "But emotions matter, also in politics."
But May was under attack from across Britain's political spectrum after saying she was considering the EU proposal for a longer post-Brexit transition period.
Seeking to unblock the stalled divorce talks, May said the initial 21-month transition could be extended by "a matter of months". At present, the two sides say Britain will remain inside the EU single market, and subject to the bloc's regulations, from the day it leaves on March 29 until December 2020, to give time for new trade relations to be set up.
The EU has said an extension would give more time to strike a trade deal that ensures the border between Northern Ireland and the Republic of Ireland remains friction-free. The future Britain-EU border in Ireland has been the main sticking point in the talks.
The Irish border will be Britain's only land frontier with the EU after Brexit. Both sides agree there must be no hard border that could disrupt businesses and residents on both sides and undermine Northern Ireland's hard-won peace process. But each has rejected the other side's solution.
The EU says the solution is to keep Northern Ireland inside a customs union with the bloc, but Britain rejects that because it would mean customs checks between Northern Ireland and the rest of Britain.
Britain has proposed instead that all of Britain could stay in a customs union - but only temporarily. The EU insists there must be no time limit.
This week's summit, which had been billed as a make-or-break moment, turned simply into a chance for Britain and the EU to give themselves more time - perhaps until the end of the year - to break the logjam.
EU Commission president Jean-Claude Juncker welcomed the opening May offered, saying "this prolongation of the transition period probably will happen".
German Chancellor Angela Merkel described finding a solution as "squaring the circle". But she added: "Where there's a will, there should be a way, and normally there is a way."
The lack of progress scrapped plans for a special EU summit on Brexit that had been pencilled in for next month, though EU leaders said they would assess the situation later.
The next official EU summit is scheduled for December, just under four months before Britain ceases to be an EU member. Any deal that is struck needs to be approved by the British and European Parliaments.
- AP