Tomkins, who represented Australia at six Olympic Games between 1988 and 2008, winning three gold medals and one bronze, said the time was right to focus on the next stage in his life.
He has not competed since the Beijing Olympics.
Rowing Australia head coach Noel Donaldson led the tributes to Tomkins.
“The contribution he made to rowing and the excellence he achieved with his crew mates across two decades is second to none and he fully deserves his accolades as one of the greatest oarsman ever,” said Donaldson.
“The impact he has made in Australian rowing is underlined by the reaction of young guys who are involved in the team now.
“Some of them were not born when James started rowing, but they admire his achievements and aspire to reach his level.”
Tomkins won his first Olympic gold medal as part of the Oarsome Foursome in the men’s four in 1992 in Barcelona, before going back-to-back in 1996 in Atlanta.
The Victorian won his third and final gold medal in the men’s pair at the 2004 Athens Olympic Games, after picking up the bronze medal in Sydney in the same event four years earlier.
Tomkins is the also only rower in history to have won world championships in all sweep oar rowing events – men’s eight, men’s four, men’s coxed four, men’s coxed pair and men’s pair.
He won a total of seven world championships.
Tomkins, who rowed in the men’s eight in Beijing that finished in sixth place, was given the honour by the Australian Olympic Committee (AOC) of carrying the Australian flag at the opening ceremony.
During the Beijing Olympics, Tomkins was also elected chairman of the AOC Athletes Commission and AOC president John Coates paid tribute to the great rower.
“James has made and continues to make a unique contribution to the Australian Olympic Movement,” Coates said.
“He is a true champion in every sense of the word and I wish him and his family the very best of luck for the future.”