"She is immune to anyone other than the powerful and famous people who populate the pages of Vogue," he writes in his upcoming memoir, The Chiffon Trenches, according to a copy seen by Mail Online.
Talley was a firm fixture on the international fashion circuit for several decades, counting the likes of Andy Warhol and Karl Lagerfeld among his close friends.
He left Vogue in 2013 but continued to occasionally work for the fashion bible, hosting red carpet interviews at Wintour's annual Met Gala.
That changed in 2018, when Talley was replaced by Liza Koshy, a then 22-year-old YouTube star.
Clearly still stung by the decision, Talley writes in his memoir: "This was clearly a stone-cold business decision. I had suddenly become too old, too overweight, too uncool, I imagined, for Anna Wintour".
He goes on to say: "After decades of loyalty and friendship. Anna should have had the decency and kindness to call me or send me an e-mail saying: 'Andre, I think we have had a wonderful run with your interviews, but we are going to try something new.' I would have accepted that … I understand; nothing lasts forever. Simple human kindness. No, she is not capable."
He added: "I wonder, when she goes home alone at night, is she miserable? Does she feel alone?"
Talley suggested that others within the industry shared his views.
"She has dashed so many on a frayed and tattered heap during her powerful rule," he added.
Wintour, Vogue's editor-in-chief for the last three decades, has long been known for her tough management style. Said to be the icy inspiration behind the lead character in the book and subsequent film "The Devil Wears Prada", Wintour has defended herself and suggested there is an element of sexism in the way she is viewed.
"I don't believe in wasting anybody's time," she told the New York Times in 2016. "I like to be honest. I like to be clear. In my own personal career, I have felt almost the most difficult thing to deal with is someone who doesn't tell you what they are thinking."
Despite her tough skin, sources at Conde Nast, Vogue's publisher, suggested Wintour would be extremely hurt by Talley's memoir.
Sources at the company told Page Six that many of Wintour's staff have been with her for several decades and "think very differently" about her.
Talley, whose memoir is out in September, has previously hinted at a difficult relationship with Wintour, telling the New York Times in 2018: "Most days, she treats me like family... I know she cares for me deeply. But other days, she treats me like the proverbial black sheep, that family member who is left out, shut out, to be avoided."
But he has also spoken warmly of Wintour's infamous 2005 intervention when his weight ballooned. "She intervened because she was alarmed and cared, and she had the minister from my church there, and the De la Rentas," he told the Guardian.
"She's loyal, a loyal friend. One sees the glacial sunglasses and impeccable dresses. But she cares."
Reflecting on his current position, Talley writes in his memoir he is now one of "the dinosaurs of Vogue, an endangered species".