About 3000 people congregated in small groups in the Domain on Tuesday, despite Police Minister David Elliott's threats of fines and imprisonment for breaching COVID rules.

They called for Australia Day to be abolished and for justice for First Nations people.

"They're out there celebrating this day like it's a birthday or Christmas," Paul Silva, nephew of David Dungay Jr who died in custody, said.

"(Today is) the day when our ancestors were murdered."

A last ditch effort by protesters to have the NSW Supreme Court approve the rally failed on Monday evening, with legal outdoor gatherings restricted to 500 people.

Police earlier warned a "highly visible and mobile ... operation" would be launched to ensure everyone complied with COVID restrictions.

Crime Commander Michael Willing said while the protest was technically unauthorised, police were mostly pleased with the protesters' conduct.

"With the exception of the few at the end, they were compliant, they were socially distant and very well-behaved," told reporters.

"Police facilitated as best they possibly could to ensure that people were in groups of less than 500 and socially distanced.

"At the end of the day the protester organisers abided by the agreement."

Four protesters were arrested as they marched from the rally after a scuffle with police.

Two were issued $1000 fines for failing to comply with COVID-19 directions. A 28-year-old man is accused of assaulting police and a 27-year-old woman of hindering police.

The annual event's usual march had been cancelled to comply with a deal made with police.

Another man, an 18-year-old counter-protester who allegedly yelled racist slurs at attendees, was arrested and charged with breaching the peace.

NSW Premier Gladys Berejiklian used her speech at the WugulOra ceremony at Barangaroo to calm tensions and recognise the hurt January 26 held for many Indigenous Australians.

"We must also recognise, as a mature and decent nation, that today is a day that causes pain for some of our First Nations people," she said.

"We cannot and should never deny any aspect of our history, or the key milestones that have made us the nation we are today."

Many of Sydney's traditional Australia Day events were scaled back or cancelled, with the celebrations and protests going ahead in sweltering heat.

The day began at dawn with the Sydney Opera House sails lit with Indigenous art in recognition of Australia's First Nations people.

Shortly after first light, the Aboriginal flag was raised alongside the Australian flag on the Sydney Harbour Bridge.

At midday two Royal Australian Air Force F-35A Lighting II aircraft flew over Sydney Harbour, but the traditional harbourside festivities of the ferry-a-thon and the tall ships parade were cancelled to avoid crowds gathering on the foreshore.

For the first time, the Salute to Australia was a private ceremony held at Government House and hosted by Governor Margaret Beazley to recognise the contributions of the frontline workers who helped with the pandemic, with new citizens also recognised.

Thousands descended on the state's beaches seeking relief from the heat, making for a hectic day for lifeguards.

NSW Surf Life Saving chief executive Steve Pearce said beaches along the state's coast were already packed by 10am on Tuesday, with many nearing capacity.

"We have all our assets out on the water - every jetski available, every rubber duck, we have our Westpac rescue helicopter in the sky, and we have our drones flying everywhere as well."

Australian Associated Press