It considered a warning by the newsreader - who said the item contained "disturbing images" - was sufficient.
However, the BSA found inadequate warning was given to viewers.
It said the footage did not explicitly show the man being shot, as his head and part of his body were behind a counter. However, viewers did see the man's legs and body convulse as the bullet hit.
"This was horrific, highly disturbing content, and we think that many viewers would have been very upset given they were not prepared for it," the BSA said.
The authority said broadcasting standards recognised disturbing material was often shown in news programmes, to reflect a world in which violence occurred, and there was very high public interest in the report.
"We should not be shielded from such events, which unfortunately do take place."
Nevertheless, the standards required viewers to be adequately warned of potentially disturbing, alarming or graphic content, the BSA said.
"While recognising the importance of the item, we find that the warning given in this instance was insufficient to prepare viewers for the graphic execution they were about to see."
The BSA found the inadequate warning, and the fact the footage screened some 30 seconds after the warnings, meant viewers were not given a reasonable opportunity to exercise discretion over their children's or their own viewing.
It upheld the complaint under the good taste and decency, children's interests and violence standards.
The BSA did not make any order as it said the decision would give sufficient guidance to broadcasters around showing graphic content in news items.
More than 175 people were injured and at least 67 people were killed in the Kenyan mall massacre.
