In a press conference today, Mr Trump said the violence had no place in the United States.
"We condemn in the strongest possible terms this egregious display of hatred, bigotry and violence on many sides, on many sides," he said.
"It's been going on for a long time in our country. Not Donald Trump. Not Barack Obama.
"It's been going on for a long, long time. It has no place in America. What is vital now is a swift restoration of law and order and the protection of innocent lives."
Nationalists were holding the rally to protest plans by the city of Charlottesville to remove a statue of Confederate General Robert E Lee when the violence took a deadly turn.
However, what the President said next raised eyebrows as instead of condemning far right groups, he highlighted the country's economic achievements.
"Our country is doing very well in so many ways. We have record, just absolute record employment. We have unemployment the lowest it's been in almost seventeen years.
"We have companies pouring into our country. Foxconn and car companies and so many others, they're coming back to our country. We're renegotiating trade deals to make them great for our country and great for the American worker," he said.
The President earlier condemned the event in a series of tweets and also called for unity.
However, critics said his response didn't go far enough and that he needed to condemn the white supremacist movement specifically.
Charlottesville Mayor and Democrat Michael Signer directly blamed Mr Trump over the tragedy.
"I'm not going to make any bones about it," he said.
"I place the blame for a lot of what you're seeing in America today right at the doorstep of the White House and the people around the president."
Republican Senator Cory Gardner said the event was terrorism and needed to be called exactly that.
Fellow Republican Marco Rubio tweeted those responsible for the tragedy were "agitators in search of relevance and publicity who must be stopped".
He also called on Trump to call the event for what it was - a terror attack.
Others also condemned the violence and blamed it on white supremacists.
Virginia Attorney-General Mark Herring, a Democrat, also tweeted it was incorrect to say there was hatred on all sides.
New Jersey Governor Chris Christie, a Republican and Trump supporter said white supremacists had no place in the US.
"We reject the racism and violence of white nationalists like the ones acting out in Charlottesville. Everyone in leadership must speak out," he said.
However the Daily Stormer, a white supremacist website promoting the Charlottesville demonstration on its Summer of Hate edition, praised the President's response.
"Trump comments were good. He didn't attack us. He just said the nation should come together. Nothing specific against us. ... No condemnation at all. When asked to condemn, he just walked out of the room. Really, really good. God bless him," it read.
Controversial filmmaker and strong Trump critic Michael Moore went even further and accused the President of inciting the violence in the first place.
In a stinging Facebook post, Moore wrote: "To the White Leader in the White House.
"There is not bigotry 'all on sides'. There is only one side where the hate lies - yours.
"You are the one who ran a campaign of hate speech and racism. You are the inciter of this violence. And we are the ones who are going to peacefully and nonviolently remove you from power. Mark my words."
Harry Potter author and fellow Trump critic JK Rowling also slammed Trump's response, and said his condemnation didn't go far enough.
Others were quick to join the criticism.
