The Spirit Walker says he decided to turn his back and walk away from the Prime Minister after claims Malcolm Turnbull did not listen respectfully.
"I felt like our voices were falling on deaf ears. It felt like he was there to argue back with us," he told NITV News.
Mr Pryor said the Minister for Indigenous Affairs Nigel Scullion, who had visited the Aboriginal Tent Embassy earlier in the week, spoke over the top of elders during the exchange.
"Nigel Scullion was very rude. He was talking over us as we were telling the Prime Minister what we want," he said.
During the meeting, a spokesperson read from a statement outlining a list of demands, which included calls for treaty negotiations and an acknowledgement of Aboriginal sovereignty.
Mr Pryor also expressed disappointment that he and the Elders were made to enter Parliament House through the back of the building, instead of meeting at the front.
"We were disappointed that we had to walk around the back to meet him, he should've came out and seen me after I walked 6,000 kilometres to see him."
Mr Pryor says Malcolm Turnbull should have met him at the front of Parliament House like Opposition Leader Bill Shorten had done a day earlier.
A spokesperson from Mr Turnbull's office said the Prime Minister was interested to hear the perspectives of the group, but respectfully disagreed with several of the matters raised.
The group expressed concern over the cashless debit card trials in Ceduna, the East Kimberley and now the Goldfields region, and called for an end to the government's intervention in the Northern Territory.
Mr Turnbull's spokesperson said the card is designed to protect children and to ensure parents are better able to look after their children.
I'm proud that this has got people back onto their feet again and to continue fighting for our rights, and finishing this quest that we need for our people and get it across the line.
Mr Pryor said he felt frustrated from the meeting.
"I said straight to his face, I respect as the Prime Minister of this country, but I'm disappointed that I had to come all the way at the back to see you. It just goes to show that the Australian government do not care about our people," he said.
The meeting comes after Mr Pryor and a group of elders also met privately with the Governor General yesterday afternoon at Government House.
The list of demands were also presented to the Governor-General during the meeting.
"I've heard that the Governor-General took on our demands and he will address these issues on behalf of our people all the way to England to the Queen, and we start seeing something happening for our people," he said.
Mr Pryor also called out Opposition Leader Bill Shorten for using his walk for justice for Mr Shorten's own agenda.
"I was really annoyed that he turned around and I said that I support the Recognise campaign and reconciliation. I do not support that. I support what my people want, and for things to get better for my people" he said.
"You do not use other people's hard earned work to push for your own agenda."
Since the beginning of Mr Pryor's journey, his quest for justice has been widely followed from people - not just across the nation, but across the globe. Support continues to pour in from millions of people on social media, despite the outcome of today's meeting.
Mr Pryor remains hopeful that action will come out of his week in Canberra.
"The last couple of days meeting with Bill Shorten, the Governor-General, and the Prime Minister, these demands will hopefully be taken on, and will turn into action," he said.
"I'm proud that this has got people back onto their feet again and to continue fighting for our rights, and finishing this quest that we need for our people and get it across the line."
