Protesters may be banned from masking their faces in the wake of an ugly clash between anti-fascist and far-right groups in Coburg on Saturday.

Police Minister Lisa Neville will meet with Victoria Police Chief Commissioner Graham Ashton on Monday to discuss the response to the violence.

It comes as the leader of the far-right group United Patriots Front has vowed to continue organising protests and attending rallies.

Seven people were arrested and residents were scared to leave their houses as masked protesters fought outside a local primary school and library.

Ms Neville  said the use of masks was a disturbing trend among protesters, who wanted to avoid being identified by the police.

"It's really cowardly behaviour," she said.

Ms Neville said the government would consider whether a "legislative provision" or whether other means were necessary to "really prevent people using this sort of tactic".

Opposition attorney-general John Pesutto said "professional protesters" were to blame for the violence, and called for the reinstatement of "move on" laws to take action against recidivist protesters.

But Attorney-General Martin Pakula said move-on laws were still available to police and the legislation had only been amended in relation to peaceful and legal protests.

"These violent protesters don't need to be moved on to another place," he said. "They need to be arrested and that's what Victoria Police did on Saturday."

Acting Premier James Merlino said he had spoken with police about the violence and would consider any police request for additional laws.

"Those people who wear face masks, who carry weapons, who engage in violent behaviour, that is utterly unacceptable," he said.

"We will work closely with Victoria Police to give them the powers that they need to deal with this kind of behaviour."

A Victoria Police spokeswoman said the Chief Commissioner and Police Minister have weekly meetings, but would not be drawn on what would be discussed on Monday.

"These discussions are confidential and we won't be commenting further at this time," she said.

Senior police have already labelled those who covered their faces at the protests as "cowards".

"All they're looking for is a fight," said Commander Sharon Cowden, who announced a new taskforce to deal with the violent protesters.

"We saw inappropriate and often cowardly behaviour with people wearing masks and hiding their identity."

Sydney Road cafes locked their doors at the height of Saturday's violence.

Glass Den manager Alex O'Brien said it was a tense situation.

"Everybody sitting in the courtyard got up, screamed and ran inside," she said.

"And then everyone was on their phones videoing what was happening outside. It was a little bit scary, all we saw was the violence around us."

Resident Jacqueline Samec said she stayed inside her home until Saturday afternoon.

"Everyone has the right to protest, but no one has the right to act violent like they did and that's scary," she said.

The Coburg traders' group says it passed a vote of no confidence in one of the rally's organisers, Moreland councillor Sue Bolton, after she allegedly shrugged off their concerns about violence.

Coburg Traders' Association president Jason Sennitt​ said Cr Bolton had "dug her heels in" and said the rally was going ahead and thought the police were over-reacting when he asked her about the potential risks.

Cr Sue Bolton has been contacted for comment.

The rival groups have largely blamed each other for starting the violence.

But footage of one of the clashes appears to show the fight was mutual, with both anti-Islam and anti-racism protesters walking around police lines and bashing each other.

It is the latest outbreak of violence stemming from the extreme ends of politics and overshadowing what were meant to be peaceful but vocal protests over the past year.

On one side is the anti-fascist Antifa group, along with radical socialists, anarchists and anti-racists, who frequently battle far-right protesters at these events, some who support the United Patriots Front and a newer "pro-Australian" group who called themselves the True Blue Crew.

"We take great pride in protecting the everyday Australians, who only want to speak out in defence of our nation," the True Blue Crew posted on its Facebook page.

"This is a callout to patriots out there, who like what they saw today, who are willing to stand with us against the far left-wing scum."

The United Patriots Front said members were only standing their ground until their rivals approached them.

"The idiots marched right into us not knowing what they were walking into; our boys smashed them," the group said on its Facebook page.

"These kids have hit us with violence from the beginning – and we will continue to respond in kind."

But police said there were problems from both sides.

"People need to reflect on the message they are sending our communities by committing acts of violence and inciting hatred," Victoria Police said in a statement.

"Urging violence and hatred within our community is not a form of free speech and will not be tolerated."

The new police taskforce is combing through security camera footage to hunt down the rest of the troublemakers who attended the rival protests.

Leader of the far-right group United Patriots Front, Blair Cottrell, encouraged his followers to continue rallying.

"We have to get stronger and stronger. The only way to do that is to keep rallying," he said in a video posted on Facebook.

Anyone with information can contact Crime Stoppers on 1800 333 000.