It is not yet known whether Kaye's cancer was caught early. She will undergo further tests this week.
"We'll have better information on that in the next few days," Key said.
"She's pretty young to have such a diagnosis but let's see."
Kaye was given the diagnosis on Friday, as she was dealing with the aftermath of the massive earthquake in the East Cape in her role as Civil Defence Minister.
She is the MP for Auckland Central and also holds the ACC, Youth Affairs and Associate Education portfolios.
During her absence, Gerry Brownlee will take over Civil Defence, Nathan Guy will be acting ACC minister, Anne Tolley will take over Youth Affairs and Hekia Parata will take over Kaye's Associate Education responsibilities.
In a post on Facebook this afternoon, Kaye said the diagnosis was "devastating news for me and my family".
"Having the opportunity to serve NZ as the MP for Auckland central and a cabinet minister continues to be a huge privilege," she said.
"I have always worked hard and given everything I have to both roles.
"I told the [Prime Minister John Key] on Friday and I took a leave of absence from my ministerial portfolios. The PM has appointed acting ministers to cover my portfolios."
She added: "I feel so lucky to have such an amazing family and group of friends who are giving me buckets of love and have been so strong helping me work through this".
Kaye said Key had been "hugely caring and supportive".
"During this time I would be really grateful to have a bit of space and privacy while I get treatment and focus on getting well."
On Friday I was told that I have breast cancer. It was devastating news for me and my family.
Having the opportunity...
Posted by Nikki Kaye on Sunday, 4 September 2016
MPs expressed shock and sympathy on social media today in response to Kaye's decision to stand down.
Labour MP Jacinda Ardern, who is Kaye's rival for the Auckland Central seat, tweeted: "This is gutting news Nikki. Thinking of you and wishing you the speediest recovery."
Cabinet minister Steven Joyce said: "Get well, Nikko. We're all thinking of you."
United Future leader Peter Dunne said: "Terrible news re @nikkikaye. Best wishes to her for a full and speedy recovery."
Justice minister Amy Adams posted: "Nikki has our full support and we're looking forward to welcoming her back soon."
Labour leader Andrew Little said he hoped Kaye made a swift recovery.
"Health is pretty crucial. You only get one chance in life, so we wish her well for addressing her health issue.
Little said Kaye was a "hard-working, reasonably new minister".
"But it doesn't matter who it is. You get a big health scare, you've got to be able to have the time to go and deal with it.
"It's the right thing and it's important that she is given the opportunity to get back on her feet."
Green Party co-leader James Shaw said the cancer diagnosis was "a real personal tragedy for Nikki Kaye and for her family".
"I just want to wish her all the best of luck and the best for her recovery... and I hope that she gets all the support she needs."
In 2011, 2894 women were diagnosed, of whom 96 were aged 35 to 39.
The cancer survival rate at five years after a breast-cancer diagnosis varies greatly by how early in the course of the disease it was detected - from 99 per cent if it was at the earliest stage, to around 15 per cent if it had advanced to stage four, which is not curable.
Treatment for breast cancer typically involves some combination of surgery, drugs such as chemotherapy, Herceptin and hormone therapy, and radiation therapy.
Breast Cancer Foundation research and communications manager Adele Gautier the first treatment for most women diagnosed with breast cancer is surgery, either a mastectomy to remove the full breast or a "lumpectomy" to remove just the tumour and the tissue around it, also called breast-conserving surgery. The clinical factors in this choice are the size and stage of the tumour as determined by a breast biopsy.
In some cases, if a tumour is particularly large or aggressive, the woman may have chemotherapy first, to shrink the cancer before surgery.
"Depending on whether it's oestrogen-receptor positive or HER-2 positive will determine what kind of drug treatment. With early-stage cancer many won't have chemotherapy. Some will be on Tamoxifen hormone therapy for five to 10 years but may not have chemo."
"For some patients, radiation therapy will be recommended to prevent recurrence."
The radiation decision is informed by a test done during surgery, called sentinel-node biopsy, in which a radioactive dye is used to identify the underarm lymph node most likely to have been affected by the cancer so it can be removed for cancer analysis.
Kaye's story so far
Nikki Kaye entered Parliament in 2008 as the National Party's first-ever Auckland Central MP, claiming an electorate which had been held by left-leaning parties for 90 years.
She held onto the electorate by very narrow margins against another rising star, Labour's Jacinda Ardern, in 2011 and 2014.
Kaye became a government minister in 2013, taking on the Food Safety, Civil Defence and Youth Affairs portfolios, and associate education and immigration roles.
After re-election in 2014, she was given greater responsibility, taking on the ACC portfolio.
She is part of National's socially liberal wing. She worked on a cross-party group to legalise same-sex marriage in 2013, and led a campaign within her party to keep the alcohol purchase age at 18 years old in 2012.
Kaye was born and raised in Auckland, growing up in Epsom and Kohimarama.
Her parents separated when she was seven, and she was one of nine siblings or half-siblings.
She was head girl at Corran School (now part of St Kentigerns) and studied genetics and law in Dunedin and Auckland.
She joined the National Party in 1998 and began working for then-Leader of the Opposition Bill English in 2002. After a stint in policy work in the United Kingdom, she returned to New Zealand in 2007 to run for Auckland Central.
Kaye is also a keen runner, and won the Auckland Women's 3000m event in 1997.
