Hayman (Orica-GreenEDGE) finished ahead of four-time winner Tom Boonen (Etixx-QuickStep) and Ian Stannard (Sky).
The finale saw the top three plus Sep Vanmarcke (LottoNL-Jumbo) and Edvald Boasson Hagen (Dimension Data) fighting for supremacy towards the end of the 257.5k from Compiègne to the velodrome in Roubaix.
The five men came together in the velodrome with the race decided in a sprint finish by the rider with the most left in his legs after 27 secteurs and 52.7km of cobbles.
The 37-year-old becomes the second Australian after Stuart O'Grady in 2007 to win the race called the Queen of the Classics.
Fellow Australian Heinrich Haussler (IAM Cycling) finished sixth, a minute behind Hayman.
Pre-race favourites Peter Sagan (Tinkoff), the world champion, and Fabian Cancellara (Trek-Segafredo), who was gunning for a fourth title, effectively lost the race when they found themselves on the wrong end of a bunch split after a massive pile-up some 115km from the line.
Cancellara later crashed on the wet cobbles and finished more than seven minutes behind Hayman on his last Paris-Roubaix before retiring at the end of the season.
"I can't believe it, I broke my arm five weeks ago and missed all the racing, I just raced a race in Spain last week," said Hayman, whose best previous result in the race was eighth place in 2012.
"This is my favourite race but this year I didn't even dare to dream."
Hayman was in the day's breakaway and when the big guns caught the group he managed to keep with them. Despite a late mechanical problem, he stayed among the five riders who went on to contest the victory.
"I knew I had to take it easy (in the breakaway) and I was waiting for the big guys to come, I saved all day," Hayman said.
Boonen attacked with 2.7km left but Hayman followed and countered, only for the 2005 world champion to rejoin him. The Australian, however, was the stronger on the final sprint.
"I wanted to be in front on the last bend because the one in front on the last bend in the velodrome always wins. But I was dead. We were all dead," said Boonen.
When the Etixx-QuickStep team realised that Sagan, who won the Tour of Flanders last Sunday, and Cancellara were trapped behind after a huge crash, Tony Martin pulled in front to help Boonen.
Boonen and company caught the fugitives with 63km left. Three kms later, as the group entered a cobbled sector, Cancellara accelerated brutally and was followed by Sagan as the duo tried to bridge the gap.
Sky, with four riders in the front group, upped the pace, only for Gianni Moscon to crash on a wet corner of the cobbled sector number 11, taking down team mate Luke Rowe in his fall.
Cancellara crashed on the cobbled sector of Mons en Pevele, losing all hope of a victory.
Sagan somehow managed to jump over the Swiss rider's bike to avoid crashing but he never made it back into the leading bunch.
Twenty kilometres from the line, Stannard's brutal acceleration cut that group down to five men.
Vanmarcke attacked in the Carrefour de l'Arbre, one of the most feared cobbled sectors, but Sky's Stannard worked hard to help rein in the Lotto-JumboNL rider and eventually Hayman was the freshest man.
