"Hitler must be smiling as he sees these modern-day guards of Auschwitz invading the streets of Ballarat trying to sow fear and intimidation." Anti-defamation Commission chair Dvir Abramovich said.

"This beyond-despicable display of depravity belongs in the Germany of the 1930s, not in our state."

Sunday's surprise march by about 40 men in black - took in the Eureka Stockade gardens, roads around Sovereign Hill, Main Road and Sturt Street - and was followed by slow moving police cars.

It came on the 169th anniversary of the Eureka Stockade - an event where troops killed 22 defenders including Edward Thonen, a 24-year-old lemonade seller who was also a German-speaking Jew. According to the Australian Jewish Genealogical Society, three Jews - including Charles Dyte - also helped to draft the resolutions at Eureka.

"Such obscene behaviour is spitting on the memory of the victims and the diggers who gave their lives to defeat the Third Reich," Dr Abramovich said.

"I have been contacted by many people in the Jewish community rightly asking why Victoria Police is allowing these hate-marches to take place without arresting these 'Final Solutionist' for breach of the peace or other breaches of the law."

Dr Abramovich said the commission began a campaign seven years ago to have the Nazi salute and Nazi swastikas banned nationally - and this had since become a reality in Victoria and Tasmania.

While the 40 or so men in Sunday's march did not wear or display Nazi symbols, police are investigating a gesture allegedly made by a 15-year-old boy not attached to the group.

"Imagine the grief a Holocaust survivor - or their children - would have felt confronted with this outrage, which they probably thought they would never see in their lifetime in Ballarat," Dr Abramvich said..

"These Hitler-worshippers revile anyone who does not fit into their Aryan worldview: indigenous Australians, Jews, Muslims, the disabled, members of the LGBTIQ+ community, and other ethnicities.

"(They) are the target of choice for this racist demonisation.

"These agents of evil, who dream of an Australian Hitler and a Fourth Reich, seek to mainstream and normalise their abhorrent beliefs and attract recruits to their warped cause."

He said the protest was driven by a bizarre belief that white people were "threatened by extinction".

"This atmosphere of vilification and dehumanisation can lead to murder," Dr Abramovich said.

"We've seen this in lone-wolf shootings around the world.

"I hope the state government considers outlawing the promotion and glorification of Nazism.

"This is a fight we have to win."

Dr Abramovich is a frequent visitor to Ballarat - and said he would be in the region in coming weeks.

Ballarat RSL President Alan Douglass said he was surprised to read about the march.

"I wish I'd been there," he said.

"I would have booed them to buggery.

"Australians went overseas and fought and sacrificed their lives for our freedom - and now we see these people in Australia glorifying what the Nazis stood for.

"That behaviour is disgusting."

According to the Bureau of Statistics, a total of 691,000 men and 35,800 women served in World War Two.

At the time, that represented one in every 10 of the nation's men, women and children.

Of those, 35,000 Australians died across all theatres of war.

In the war against Germany, 7289 Australians became prisoners of war. Of those, 234 died.

"The RSL is the largest ex-service community in Victoria and our organisation strands with the Ballarat community in condemnation of the weekend's protest," State President Robert Webster said.

"We don't condone this behaviour today - or any day."

Wendouree State Labor MP Julianna Addison said Sunday's protest was "abhorrent".

"The footage of masked men calling on people to attack others and to fight back is totally unacceptable in our community," she said.

"We're inclusive - and everybody is welcome.

"This is not Ballarat. This is not who we are.

"We're good people, we're welcoming people, we're inclusive people."

She said members of the African community had contacted her, expressing their concerns..

"I was very proud to speak in parliament against the Nazi salute," Ms Adison said.

"As a history teacher I have spent my life educating teenagers about genocide, about holocaust, so for me I know this is such an important issue and I do believe we have a real responsibility to continue to support our students to understand what happened in Nazi Germany and make sure it never happens again."

Health Minister and Macedon State Labor MP Mary Anne Thomas was in Ballarat on Monday and she was appalled.

"Ballarat (is about) Eureka - and is the home of the goldrush," she said.

"It was Australia's first big multicultural moment.

Ballarat always been a welcoming city for people from all backgrounds and will stay that way because as a government and as a community we are committed to enabling people to live in peace and harmony.

"Those gutless cowards with their faces masks on are not welcome here or anywhere else in the state of Victoria."

Defenders at the Eureka Stockade famously included Italians, Caribbeans, African-Americans and more.