Drew Douglas, a spokesman for major electricity suppliers Citipower and Powercor, said that 3800 Citipower customers were cut off from power after the storm struck about 3.30pm on Tuesday.  This number dropped to about 140 customers hours later.

Outages were also recorded at nearly 4000 Powercor customers' homes, but he expected far fewer of them related to the storm, he said.

A State Emergency Services Victoria spokeswoman said the group had received 875 calls for help across the state, with most — 660 — in metropolitan Melbourne.

Most of the calls had been made after 3pm, the spokeswoman said.

"The main problem has been the wind. We've had trees down and a fair bit of building damage, either as a result of the wind blowing tiles or loose objects, or trees or branches onto those structures," she said.

Amid the winds, a large metal sheet fell on to a public footpath at the city construction site where a brick wall collapsed last year, killing three pedestrians.

The metal sheet, which had strips of wood attached to it, fell over the top of a barricade separating the construction site from the footpath on Swanston Street.

Paramedics were not called to the scene on Tuesday afternoon, a spokeswoman for Ambulance Victoria said.

The Victorian WorkCover Authority is investigating the incident, which was at the same site where a brick wall collapsed in strong winds last March. French academic Marie-Faith Fiawoo, 33, and siblings Bridget, 18, and Alexander Jones, 19, were walking past Grocon's building site at the CUB Brewery in Carlton when the wall collapsed and died.

Several SES units were particularly busy during the windy weather, with volunteers at its Frankston unit responding to 94 calls and its Emerald unit responding to 59 calls throughout the afternoon.

Temperatures plummeted during the afternoon with the mercury dropping from 23 degrees at 3pm to 14 degrees in half an hour.

By 4pm the temperature was sitting at 11 degrees.

Earlier on Tuesday, the Bureau of Metorology issued a severe weather warning for large parts of Victoria, including Melbourne, Gippsland, the Wimmera and around Seymour and Warrnambool, predicting gusts of up to 100km/h.

Bureau duty forcaster Matthew Michael said the outer suburbs of Melbourne and elevated areas would be worst hit by the winds.

He said the winds would be at their strongest between 3pm and 5pm.

The Bureau recorded wind gusts of 109km/h in Frankston on Tuesday afternoon, the strongest winds in Melbourne.

VicRoads also reduced the speed limit on the West Gate Bridge to 60km/h due to the high winds.

A Melbourne Airport spokeswoman said some delays to flights were expected throughout the afternoon due to the weather and in particular strong winds.

She said travellers should go to the airport at the times they were expected and check with their airlines for any changes to their flights.

With Larissa Nicholson