"I want to be worthy of standing shoulder to shoulder with the brave artists who are fighting for all people to be treated with the respect and dignity they deserve," the statement read, posted on Chalamet's social media accounts ahead of the 2018 Oscars.
He was nominated for best actor for his role in Call Me By Your Name that year.
Allen's daughter Dylan Farrow had spoken out against the director in a television interview, and recounted details of the alleged sexual abuse she suffered from him when she was 7.
The interview aired on CBS This Morning, and in response filmmakers and actors denounced Allen, including Greta Gerwig, Colin Firth, Jeff Daniels, and Natalie Portman.
In his book Apropos of Nothing, Allen writes about Rainy Day in New York and how he felt about Chalamet's statement, the Independent reports.
"All the three leads in Rainy Day were excellent and a pleasure to work with," Allen writes.
"Timothee afterward publicly stated he regretted working with me and was giving the money to charity, but he swore to my sister he needed to do that as he was up for an Oscar for Call Me By Your Name, and he and his agent felt he had a better chance of winning if he denounced me, so he did.
"Anywho, I didn't regret working with him and I'm not giving any of my money back."
Chalamet's co-star in A Rainy Day In New York also publicly said she regretted deciding to work with Allen.
The film also starred Elle Fanning, Selena Gomez, and Jude Law.
The release of the movie by Amazon Studios was stopped following the controversy regarding Allen, but it was released in Poland in July 2019.
Chalamet lost out for the award for best actor to Gary Oldman for his role in the Darkest Hour.