For four days, his sons and daughters were locked in heated discussions, unsure what was going on and pleading to see him.
DailyMail.com has now learned that following an emergency Jackson family meeting and desperate pleas to Joe's manager, Charles Coupet, family members were finally granted access to the octogenarian on Tuesday night.
Coupet has not responded to DailyMail.com's requests for comment.
But DailyMail.com has learned that Joe himself had given verbal instructions making clear he didn't want visitors or family members to see him or his medical records
It's understood Katherine, daughters Rebbie and Joh'Vonnie and granddaughter Yashi Brown, did visit with the patriarch in Las Vegas, Nevada on Tuesday.
The visit came following a tense stand off during which the Jackson family struggled to gain medical information about, and access to, Joe.
Joe's son Jermaine confirmed the odd situation when approached by DailyMail.com.
'No one knew what was going on - we shouldn't have to beg, plead, and argue to see our own father, especially at a time like this,' he said. 'We have been hurting.
'We were not being told where he was and couldn't get the full picture. Even from the doctor. My mother was worried sick.'
Jermaine, 63, added 'He's very very frail, he doesn't have long. The family needs to be by his bedside - that's our only intention in his final days.
'It's what any family would want, but some people around him think they know better...and they don't.'
Talent manager Joe, who guided his children to unprecedented showbiz success as The Jackson 5, is understood to suffer from a form of dementia and has suffered several strokes in recent years.
A source close to Joe's daughter, superstar singer Janet Jackson, confirmed that she too had been denied access to see her father in recent days.
And Joe's other daughter Joh'Vonnie Jackson - the product of an extra-marital affair - had also been barred from seeing her dad despite being very close to him and living nearby in Vegas, the source claims.
It's understood Katherine and daughter Rebbie had last visited Joe at his Vegas home last week after learning his health had deteriorated.
But after the brief visit the women spent a night in a nearby hotel before returning to discover Joe had been moved from his home.
DailyMail.com has learned that Joe himself had given verbal instructions - which exist on a phone recording - making clear he didn't want visitors or anyone from the family to see him or his medical records.
Part of the recording was overheard by some family members.
How that recording was obtained is not clear, but one source close to the camp said: 'The people around Joe Jackson are the people who have been there for him in his everyday life.
'They are acting in his interests and no one else's.
'The Jackson family doesn't have the capacity to deal with this on its own, so certain people are taking charge of the situation.
'It's delicate... and not as black and white as people are making out.
'The Jacksons are now in dialogue with the people looking after Joe, and they are being kept informed.'
One family member said: 'If it's true there was verbal instructions from Joseph, we'd like to know how that instruction was obtained.
'He's a proud man who is all about image - was he saying he doesn't want anyone to see him like that, or was he saying he doesn't want family near him?
'Something is not right about this, and the family wants to have guarantees that they will continue to have access. I can't believe we're even having to say that.'
