Some news reports stated that Dunn has other children with the boy's mother.
Police said the incident occurred while the boy's mother was running errands.
A Flagler County Sheriff's deputy wrote in his report that he noticed the boy's face was red and that "the impression left behind was similar to marks left behind by fingers."
"I saw loop marks on his right arm and left hamstring area," he wrote in the report.
Deputies said a belt discovered in the home appeared to match the injuries on his leg.
Child protective services officials examined the boy and found signs of physical abuse, including injuries to the area around his right eye and marks on his arms, legs and buttocks, authorities said.
Dunn denied the allegations, telling deputies that he had not hit the boy or held his head to the ground. However, Williams said that after interviewing Dunn, authorities determined a crime had occurred.
The man admitted he took the boy out to the pond and said that, once they were out there, he picked him up and told him he was going to feed him to the alligators.
But he was just teasing the boy, Dunn told authorities.
Dunn was arrested on a child abuse charge. He has been instructed not to have any contact with the child, police said.
His lawyer, William Bookhammer, could not immediately be reached for comment.
According to Florida law, child abuse is defined as "intentional infliction of physical or mental injury upon a child; an intentional act that could reasonably be expected to result in physical or mental injury to a child; or active encouragement of any person to commit an act that results or could reasonably be expected to result in physical or mental injury to a child".
Florida hosts the largest alligator population in the United States and made the creature its official state reptile in 1987, according to the National Zoo. An estimated 1.3 million to 2 million gators live across all 67 counties in Florida, the Orlando Sentinel reported in 2013, and inhabit fresh water marshes, swamps, rivers and lakes.