More than 200 people protested outside City Hall.
"Hey hey! Ho ho! Rob Ford has got to go!" they chanted.
City Councilor James Pasternak said the controversy consuming Canada's largest city cannot go on day after day. He said several city councilors asked Deputy Mayor Norm Kelly to approach Ford and "orchestrate a dignified exit from city hall."
Kelly met with Ford and suggested he take a temporary leave until later this year or early next year, but Ford rejected that idea. Councilor Frances Nunziata, also a Ford ally, said they are all frustrated Ford won't step aside temporarily.
"There's nothing we can do. He's the only one who can make the decision," Nunziata said. "He needs to take some time off and get the help he needs."
Kelly earlier said Ford didn't tell anyone he would admit to smoking crack before he did so Tuesday.
"It came right out of the blue," said Kelly, who learned about it from a member of Ford's staff after the mayor stopped on his way to his office to tell reporters. "I was like, 'What? What have you been smoking?'"
There is no clear legal path for Ford's critics to force him out. Municipal law makes no provision for the mayor's forced removal from office unless he is convicted and jailed for a criminal offense. Police have not charged Ford.
"He has stubbornly refused to listen to everyone across the city to step down," Councilor Janet Davis said.
Nelson Wiseman, a professor at the University of Toronto, said the province of Ontario could conceivably step in and put Toronto under trusteeship because municipalities are under provincial jurisdiction. He said the chances of that happening are "slim to none," though it might be possible if Ford is charged with a crime and almost all the city councilors plead for Ford to step aside in a motion.
Ontario Premier Kathleen Wynne has said she's concerned that Ford's personal issues were making it difficult for the city to carry on normally. But she said it was up to police, the courts or the mayor to take action.
City Councilor Denzil Minnan-Wong, a member of Ford's executive committee, is putting forward a motion that would ask Ford to take a leave of absence, a measure that could be voted on next Wednesday.
Another councilor is putting forward a motion that could strip some of his powers.
"The right thing to do is for council to take a clear position," Minnan-Wong said. "I remain concerned that there's more information that's going to come out. I'm troubled by that and that it will hurt this city even further."
Voters may have the final word on Ford's future. He has said he plans to run in the October 2014 mayoral election.
Police said they obtained the video in the course of a drug investigation into Ford's friend and occasional driver, Alexander Lisi.
The mayor has called on police to release the tape, but police said they are prohibited from doing so because it is evidence before the courts. Police said the video will come out when Lisi goes to trial on drug and extortion charges. Lisi is accused of threatening two gang members who had been trying to sell the video to the media.
A judge is expected to make a decision early next week on whether to allow the release of remaining portions of a document that revealed Ford's ties and covert meetings with Lisi, an alleged drug dealer.
Toronto Police Chief Bill Blair has also said police have a second tape, but he has declined to discuss what's on it.
___
Follow Rob Gillies on Twitter at http://twitter.com/rgilliescanada
