The woman was 13 when the assault happened and had the child at 14 - below the age of consent.
Work and Income had since admitted it was wrong and is repaying about $4500 to the woman.
Asked if she would apologise to the woman by Green MP Jan Logie, Tolley said she would.
"As Work and Income have said themselves, they got it very wrong in this case ... It is very clear that in this situation Work and Income did not apply the law correctly and I appreciate this was extremely distressing and upsetting for the young woman and I sincerely apologise for that."
She said the reason there was an obligation to name a father, except in cases such as rape, was to try to ensure the father paid a fair share of child support.
She said the Government had no intention of changing the obligation, which was put in place under National in 1990 and the penalties were increased by Labour in the 2000s. However, she had asked officials to assess its effectiveness.
"That is what I have asked MSD (Ministry of Social Development] to do some research on, to see if the intent of the policy is actually reaping the rewards that we want to ensure that it has."
The law requires women to name the father to apply for child support or face sanctions of up to $28 a week per child off their benefit.
Logie said about 15,000 women had their benefits docked for refusing to do so.
"Is it appropriate to deprive women of essential income when the reasons people don't name a father are personal, private and, frankly, none of the state's business?"
Beneficiary groups have called for the clause to be dropped altogether.