Slightly more than 100km from the end of Stage 9 on Sunday, the 33-year-old Spaniard stopped at the side of the road, hopped off his bike and got into the passenger seat of his team car.
"I could not keep going," Contador said.
"This morning I had a fever. And I did not feel good since I crashed on the first day."
Contador crashed in each of the opening stages, damaging his right shoulder, elbow and hip, and had already lost time on the other overall favourites.
He was in 20th position at the end of Saturday's eighth stage, three minutes, 12 seconds behind leader Chris Froome, who said Contador's departure from the race was a "big loss for the Tour de France."
"We are not going to have to chase his attacks 100 kilometres out anymore, it's one less thing for us to worry about, but it's maybe a shame for the race," Froome said.
During Sunday's stage, Contador dropped back to his team car four times before finally withdrawing.
He had started the stage in a bright fashion, though, attacking alongside fellow Spaniard Alejandro Valverde to catch a group of early breakaway riders on the first climb.
But Contador could not continue his effort and was quickly brought back by the pack before he pulled out in the Port del Canto climb. He then handed his bike to a mechanic and waved to TV cameras as he exited the race that made him famous.
#sbstdf Farewell El Pistolero! https://t.co/chzv6Ut2lm — CyclingCentral (@CyclingCentral) July 10, 2016
The withdrawal occurred on Contador's home roads near the town of Vilamur in Spain.
"It's bad news for me," said Tinkoff team owner Oleg Tinkov, who is planning to leave the sport.
"It was my last try to win the Tour de France. But who knows, maybe I'll come back in a few years."
Contador, who won cycling's showpiece race in 2007 and 2009, had made the Tour de France his main goal of the season, skipping the Giro to focus on the Tour.
If he recovers, Contador is still expected to compete in the road race at the Rio Olympics next month.
"I will undergo tests to find out what is going wrong," Contador said.
He was among the three pre-race favourites this year alongside defending champion Froome and Movistar's Nairo Quintana.
