"England players will be focusing on preparing for our matches ahead of the Old Mutual Wealth Series and will not be released for the Barbarians fixture," a Rugby Football Union spokesman said.
The Barbarians had previously stated that they were enthusiastic about opening discussions with the four unions - the RFU, Irish Rugby Football Union, Scottish Rugby Union and Welsh Rugby Union - over releasing the players.
"We are going to assemble an extremely strong team for this historic game at Twickenham," a Barbarians spokesman said.
"We hope it will include players involved in the exceptional Lions series with the All Blacks. The players will get the chance to face New Zealand for a fourth time this year.
"We have reached an agreement with Premiership Rugby, who have been very supportive about player release. We will talk with the home unions about access to players."
Regardless of the agreement with Premiership Rugby, release of the England internationals involved with the Lions including Maro Itoje, Owen Farrell and Mako Vunipola always required the consent of the RFU, with the Barbarians fixture falling within the EPS window.
All of the England's 17 players involved with the Lions in New Zealand are expected to be named in England's Elite Player Squad (EPS) for the new season.
Obtaining permission from the four unions to release their top players a week before their respective November international periods always appeared to be a tall order, with the other unions now likely to follow the RFU's lead.
Interest in the fixture would have been sent through the roof had the unions approved the concept of combining the Lions players with other leading internationals from around the world under the guise of the Barbarians, including Australia flanker David Pocock who is returning from his sabbatical.
The match between the Barbarians and New Zealand, the first since 2009, has been arranged as part of the All Blacks' 125th anniversary celebrations.
The RFU had previously hoped to use the November 4 date to bring forward the much anticipated meeting between England and New Zealand, with no match between the two sides on the schedule until November 2018.
Negotiations however fell through, with the match falling outside the international window and a number of Premiership clubs opposed to releasing their players early to England for an extra Test.