Castro, 23, immediately picked it up and handed it to Pirates third-base coach Mike Rebelo who looked bemused by what was being given to him.
Professional sports tend to have strict codes surrounding electronic devices and mobile phones. Major League Baseball doesn't allow players or coaches to have their mobile phones on them while they're in the dugout.
An embarrassed Castro spoke about the incident to the media after the game which the Pirates lost 6-4.
"I don't think there's any professional ballplayer that would ever go out there with any intentions of taking a cellphone,'' Castro told media members through an interpreter. "It's horrible it happened to me. Obviously, it was very unintentional.''
Castro explained his sliding glove is generally placed in his back pocket and he believed this was all he had on him.
"My first day back, if I was to be the centre of attention, I would want it to be helping the team win, but never in this form,'' Castro said. "This is definitely something that was an accident, a mistake, something I'm going to learn from. But definitely something I didn't mean to happen.''
The never before seen moment sent social media into a spin with Castro trending on Twitter as the footage circulated.
Pittsburgh Post-Gazette journalist Mike Persak wrote: "I straight up can't believe this happened what the hell".
Bay Area radio producer Kyle Madson wrote: "Given this sport's problems with cheating via electronics over the last decade or so this seems like the kind of thing that shouldn't be allowed."