Their bodies, each shot once, were left on the floor of their home in San Jose, next to a chilling note that read: "Sorry, my first kill was clumsy".
They were discovered by concerned friends, who opened an unlocked sliding glass door at around 2pm on Sunday and found them lifeless.
It was unknown how long the couple, who were Muslim, had been dead. Police haven't released a motive in the killings or any other details.
The husband and wife often hosted relatives at their home since moving from Bangladesh to the United States.
Mr Rabbi, who was an engineer, liked to hunt and had several guns in the house, friends told KTVU.
A source told ABC7 that aside from the ominous message on the floor, another rambling message was written across the wall.
It said in part: "I can't be like you, telling a lie. I can't love someone without telling them."
It also said the wife begged for her life before she was killed, NBC Bay Areareported.
Evergreen Islamic Center in San Jose said the bodies of the parents were released Wednesday afternoon to relatives and the funeral is being planned for Friday at the Five Pillars Farm Cemetery in Livermore, KNTV reports.
Mosque spokesman Faisal Yazadi did not know about the arrest of the younger son until notified by a reporter.
"Needless to say," he said. "This is one of those tragedies nobody ever wants to be in. All we can do is pray. This is a family matter and a police investigation."
It was not immediately clear Thursday morning if the two sons have attorneys.
Hasan Rahim, from the Evergreen Islamic Center, said the Rabbis often attended services at the mosque.
But he said he had warned the couple of the dangers of keeping guns at home.
"I just said be careful. If you have weapons in the house be careful," Mr Rahim said.
"I've known him for 30 plus years, so it's a loss," he added.
"People were in disbelief, in shock, they were stunned and the loss is so enormous that we are still finding it difficult to come to grips with the truth.
"I still cannot reconcile the tragedy of this whole situation. It is just unbelievable. It is just too much too accept."
Friends described the the husband and his accountant wife as gentle and kind.
"We are surprised that they are no longer with us and especially going away in such a manner is something we just can't come to grips with," Evergreen Islamic Center member Faisal Yazadi said.
"He always asked me about my family, now I can't answer him back. I can't ask him about his family," he added.
-Daily Mail
-AP
