"It doesn't seem like that to me," he says down the phone line from London in a voice that is so self-effacing that you can picture him looking at his feet when he talks. "I'm as busy as I want to be but I'm not working all the time. I get time off - as you should at my age."
It's a pleasingly relaxed attitude from someone whose peers often say they seize a work offer because they fear that it may be the last. "I turn stuff down," he admits, "but not acres of stuff. The stuff I get offered tends to be quite interesting, but I'm always quite picky. It has to interest me and I have to be right for it.
"I'm getting quite good at picking the ones that are right for me."
Leigh's way of working, in which actors build the character from the ground up - sometimes over as much as a year - by a process of sustained and complicated improvisation, is legendary. But in his newest role, in The Iron Lady opening in cinemas next week, Broadbent plays a real person: Denis Thatcher, the husband of Margaret Thatcher, the Prime Minister of Britain from 1979 to 1990.
It's not as if he's new to this kind of work. He had great sport with W.S. Gilbert, half of Gilbert and Sullivan, in Leigh's Topsy-Turvy in the 90s. He won a Golden Globe and an Oscar for playing the still-living writer John Bayley, the husband of Iris Murdoch in the 2001 film Iris, which charted her descent into dementia. His television incarnation of Lord Longford, who campaigned against homosexual law reform and agitated for the release from prison of Moors murderer Myra Hindley, was widely admired.
Bringing a real person to life on screen is plainly a different process from inventing one, but Broadbent says owing a duty to the reality does not limit artistic licence.
"I think it opens it up, really," he says. "It's slightly different with the Thatcher film because the role is not a naturalistic character arc - for much of the film he's a figment of her memory and imagination, so the normal rules don't quite apply.
"But it's really exciting to investigate all these complex real people, because they've had a lifetime to develop all sorts of conflicting idiosyncrasies so I don't find it limiting. But it's important to remember that at the same time you're doing a fiction. It's not a documentary. Yes, you owe a debt to the real thing but there are only so many known facts. You can't get too hung up about being totally fair in every respect."
The raw material in the case of the famously unobtrusive Denis Thatcher was thin on the ground. A 1999 documentary released after his death as Married to Maggie and the 1996 biography Below the Parapet, were both the work of his daughter Carol, so the perspective was scarcely broad.
"He was hard to know," says Broadbent. "He was a very private person. I looked at a lot of film of Margaret Thatcher where he was in the background or giving one-word answers to the press pack. But it was really Abi Morgan, the writer, who decided how she wanted him to be and it was up to me to inhabit the body."
Interestingly the Denis Thatcher the film delivers is far from submissive in private. And Broadbent says there is evidence for that.
"We have historical glimpses. We can tap into reminiscences by cabinet ministers, reporting on meetings at three in the morning when Denis would come into the room and say 'Thatcher! Bed!' and she would meekly get up and follow him."
Among the more intriguing trivia about Broadbent to be found online is that he turned down an OBE in 2002.
"I didn't think it would suit me, really," he says. "I have this sort of fanciful notion that actors - like all artists - should not be part of the establishment, that we should stand apart.
"I also thought that not accepting it would have pleased my father. I think he would have enjoyed that."
The Lowdown
Who: Jim Broadbent
What: Playing Denis Thatcher to Meryl Streep's Margaret in The Iron Lady
Where and when: Opens at cinemas Boxing Day.
- TimeOut