"The three people were pronounced dead at the scene, with a fourth taken to hospital," the statement said.
It added that in a separate incident earlier in the afternoon a rally car collided with five people.
Four people were treated at the scene after the first incident and a fifth was taken to hospital.
"Emergency services were in attendance. Investigations into both incidents are ongoing," police said.
Tony Cowan, who witnessed the second, fatal accident, said: "It was just one car that lost control. It went sideways one way and then to the other side of the road and ploughed into four people.
"It was terrible ... absolutely terrible. I ran to help but there was little I could do. The air ambulance arrived after about three quarters of an hour. There were police cars and ambulances. It was chaos, just chaos."
The annual three-day rally is named after the Scottish Formula One driver, who grew up in the area. Jim Clark was killed in a motor racing accident in Hockenheim, Germany, in 1968.
Peter Cunningham, 40, a care worker from Bolton, was about 60ft from the crash, along with his eight-year-old son Callum. He said that about 20 minutes before the cars set off officials warned a group of about a dozen spectators to move because they were in an "unsafe place" just beyond a small bridge.
He saw one car race over the bridge, followed by another about 10 seconds later.
The second car appeared to lose control after hurtling over the "bump" of the bridge.
"The back end wobbled and the guy just lost control of the car and went into spectators sideways," he said.
John Lamont, the local Tory MSP, who lives about five miles from the crash site in Coldstream, said: "There's shock and disbelief that something as tragic as this could happen.
"Clearly there's an element of risk in any rally of this nature but the organisers go to great lengths to ensure that safety is paramount at all times."
