Smither said her daughter had told her, "Mum, it's an alien coming to get us,'' and had been unable to sleep on Thursday night.
Her son was familiar with drones, having earlier researched them, and was concerned as he understood some models provided a live video feed back to the operator, Smither said.
He had asked, "What if it's someone who's not a nice person?''
Smither had since lodged a complaint with the Civil Aviation Authority (CAA).
She said she would attempt to shoot the drone or throw something at it if it returned to the airspace above her home.
A CAA spokeswoman confirmed it was illegal in most cases to fly a drone over a homeowner's property without the permission of the property owner.
She said the CAA was following up Smither's complaint, but cautioned against people taking matters into their own hands and shooting down a drone.
We would strongly advise against this as they could be charged with endangering an aircraft, which is a serious offence.''
Smither was not the only St Kilda mother to report allegedly inappropriate drone usage the same night.
Richardson St resident Audrey Ross described a similar incident about 5.45pm as her 3-year-old son played outside
However, this time it was a black quadcopter (a drone with four rotors).
She had also gone outside to investigate after hearing a buzzing noise, before she looked straight up and saw a drone directly above her.
The unmanned aerial vehicle had also flown off as soon as she spotted it.
The incident had been "really upsetting'' and she hoped it was "just kids'' flying the drone, and not someone with sinister intentions.
"Even if it's just kids playing around being silly, they need to learn that it's not appropriate,'' Ross said.
She was now hesitant about allowing her son to play outside.
"We've got quite a private backyard and this has taken that privacy away.''
