It is understood Evans, 55, and the man who accused him of inappropriate conduct continued to associate with each other afterwards.
Downing Street refused yesterday to say whether Prime Minister David Cameron was made aware of the allegations at the time.
Evans spoke of his "incredulity" about his arrest over an allegation of sexual assault and another of rape between July 2009 and March 2013, both of which he strongly denies.
The MP for Ribble Valley, Lancashire, said he could not understand why complaints have been made to police by "two people who are well known to each other and until yesterday, I regarded as friends".
He added: "The complaints are completely false and I cannot understand why they have been made, especially as I have continued to socialise with one as recently as last week. I appreciate the way the police have handled this in such a sensitive manner and I would like to thank my colleagues, friends and members of the public who have expressed their support and, like me, a sense of incredulity at these events."
Evans revealed in December 2010 that he was homosexual. In an interview at the time, he said: "With my background in South Wales it was hard enough being a Tory, let alone being gay."
Tory MPs rallied round the deputy speaker as he resisted calls to stand down, after being released on police bail. Evans is expected to return to the House of Commons as usual but has asked to be excused from his ceremonial duties during the Queen's Speech in the Commons on Thursday.
Number 10 has so far remained silent on Evans's arrest. However, Philip Hammond, a Cabinet minister, said he thought it would be "difficult" for the MP to carry on in his role as Deputy Speaker under such public scrutiny.
- Telegraph Group Ltd
Farage open to Tory pact
Nigel Farage has opened the door to an electoral pact between his UK Independence Party and the Conservatives but said Tory MPs would have to oust Prime Minister David Cameron as their leader first.
Some Tory MPs, worried about Ukip's strong performance in last week's local elections, called for a deal between the two parties at the 2015 general election.
It could see Ukip stand aside in seats fought by Eurosceptic Tories who pledged to vote to leave the EU in the referendum Cameron has promised by 2017.
The Tory leadership dismissed the idea and there is no sign of a move to trigger a vote of confidence in Cameron's leadership. However, that could happen if the party panicked after next year's European Parliament elections, when Ukip is expected to come first.
Calls for a 2015 pact may grow if Conservative and Ukip councillors form "local coalitions" after last week's elections to keep Labour and the Liberal Democrats out of power.
- Independent
