Dominique Dowding, Alexandra Park chief executive, confirmed the builder's departure.
"Basically, we have replaced the contractor on the East Gate site, Canam Construction. We now have a new contractor in place, CMP. We believe the project and construction are in good hands and it will be completed as we wanted."
Asked why Canam had left, Dowding said: "For legal reasons, I can't get into that."
Canam was building between 112 and 118 units at the site near the ASB Showgrounds.
Stage one of the project is planned to build 246 "high-end apartments" and gross sales by last February had reached nearly $300m, Alexandra Park said last year.
The apartments, in 2016, were being advertised at between $585,000 and $2 million.
The project is at 223 Green Lane West on part of Alexandra Park's big carpark which the business says is being "transformed into an urban village."
The first 750 residents were due to move in alongside the trotting track this year but those close to the project said there had been delays.
The project was launched in 2015 with apartments, townhouses and penthouses in high-rise formats.
Dowding said last year a FreshChoice grocery store and other shops would be developed as part of the project.
Colliers International has been marketing the project and the lead architect is RTA Studio.
Colliers has said the Alexandra Park project will be "a transformation, truly unique, combining exciting retail with exquisite design, inherent quality and luxuriously appointed freehold one, two and three bedroom apartments, penthouses and terrace homes, located in the heart of Epsom with the fabulous Cornwall Park over the road and close to the best schools including full grammar zones, public transport."