Durant's teammate Stephen Curry has already said he would be unlikely to attend while Warriors coach Steve Kerr has been a vocal critic of Trump.
Durant was speaking amid an outcry over Trump's response to violence in Charlottesville, Virginia, last weekend, which saw neo-Nazi and white supremacist groups clash with counter-protesters.
Durant said he believed Trump bore responsibility for stoking racial divisions in the United States.
"He's definitely driving it," Durant said. "I feel ever since he's got into office or since he ran for the presidency, our country has been so divided and it's not a coincidence.
"When [Barack] Obama was in office, things were looking up. We had so much hope in our communities where I come from, because we had a black president and that was a first.
"So, to see that and to be where we are now, it just felt like we took a turn for the worse, man. It all comes from who is in the administration.
"It comes from the top. Leadership trickles down to the rest of us.
"So, you know, if we have someone in office that doesn't care about all people, then we won't go anywhere as a country. In my opinion, until we get him out of here, we won't see any progress."
Durant meanwhile saluted athletes who have spoken out about race in America, citing former San Francisco 49ers quarterback Colin Kaepernick and NBA superstar LeBron James, among others.
"As far as what's going on in our country, for one as an athlete, you have to commend Colin Kaepernick, LeBron James, Carmelo Anthony, CP3 [Chris Paul], Dwyane Wade for starting that conversation last year.
"For us to move forward, we need more athletes and people of power and influence to come out and speak," Durant added.
"It's great to see a lot of athletes coming together, and trying to direct a positive path for a lot of kids and a lot of people in this country who look up to us.
"It's huge for us, it's huge for sports. It's huge for the influence we have, because we're leaders at the end of the day.
"It feels good to see my brothers in the NBA and across sports speaking out."