The unnamed MP, who cannot be named for legal reasons, was in custody on Tuesday after being held by the Metropolitan Police.
Scotland Yard said the man in his 50s was also detained on suspicion of abuse of position of trust and misconduct in public office.
Tory chief whip Chris Heaton-Harris urged the MP to stay away from Parliament but had not yet decided whether to suspend the whip, meaning the MP remains a member of the parliamentary party.
In a statement, the Metropolitan Police said : "A man, aged in his 50s, was arrested on suspicion of indecent assault, sexual assault, rape, abuse of position of trust and misconduct in a public office."
The Met said the alleged sexual offences were committed between 2002 and 2009 in London, and related to a report made in January 2020.
"An investigation is ongoing, led by officers from Central Specialist Crime,” the Met added.
A Conservative whips office spokeswoman said: "The Chief Whip has asked that the MP concerned does not attend the Parliamentary Estate while an investigation is ongoing.
"Until the conclusion of the investigation we will not be commenting further."
It is understood the Conservatives will not make a decision on whether the suspect has the whip suspended until after the police investigation is concluded.
The shock arrest comes after a string of sexual misconduct claims in Parliament.
Around 15 MPs are expected to have been investigated by the independent watchdog that examines allegations of bullying, harassment or sexual misconduct in the last year.
Earlier this month, Parliament’s Independent Complaints and Grievance Scheme (ICGS) wrote to MPs giving an “exceptional” preliminary update given the scale of interest in Westminster sleaze.
The message this week noted that there were 15 cases started against MPs between July 2020 and June 2021, adding: “We are seeing a similar trend in disclosure so far this year.”
The final figure will be published in the ICGS annual report in the coming months, but that alone does not detail how many parliamentarians have been complained about.