Malcolm Fraser departs as something of a figure of reverence among those of the left of centre, respected for integrity, moderation, a pioneering role in multiculturalism and the removal of the stain of racism from Australian conservative statesmanship. He will be seen, particularly, as a champion of Aborigines and the rights of refugees and asylum seekers.

There was a time when just those constituencies despised, feared and hated him with a personal passion akin to the loathing, from the other side of politics, of Julia Gillard 35 years later. Even John Howard and Tony Abbott, disliked at the visceral and emotional level on the left of politics, have not been detested as Fraser was while prime minister.

For most, the rancour and bitterness acquired bite with Fraser's role in bringing down the Whitlam government, and his struggle for legitimacy and moral authority in the eight years after. To them, Fraser was ever a usurper, ruthless, a man who played his politics hard. When such people fall, as inevitably they do, they attract little sympathy. When a tear appeared in the corner of Fraser's eye as he conceded defeat in March 1983, after eight years as prime minister, hundreds jeered. Even in the National Tally Room.

Even some of his supporters had thought of him as a cruel and aloof bastard, who took no prisoners, even on his own side. Some, such as Reg Withers, himself a model of deviousness, would have added the word "treacherous" to the word "bastard."

He was tall, standoffish, caricatured as an Easter Island statue. A lonely boy who had been sent to boarding school after a polio and pneumonia fright at 10. He learnt self-reliance, but was hardly ever relaxed, of somewhat aristocratic mien. Some thought, wrongly, his discomfort with small talk and banter a sign of hauteur and disdain, and even his genuine interest in social problems a patrician matter of noblesse oblige.

The start of the humanisation of Fraser among such instinctive haters began with the tear. But it was much developed by his later public humiliation in Memphis, Tennessee when, after wandering alone at night, he woke in a motel room early one morning absolutely unable to remember the past 12 hours, and minus his trousers and his wallet. Most people assumed the worst, and thought it the more funny because he had once said that he would catch the Whitlam Government "with its trousers down".  He seemed human at last.

His image, among these detractors, improved as he took on leadership of Care Australia and Care International, and became involved with the Eminent Person's Group in pushing for a South African settlement (to add to his early work in the creation of a multi-racial Zimbabwe). It was burnished as it became clear over time that he had been a moderate and a liberal, not a modern market-oriented conservative of the type developing in the new Liberal Party, particularly under his old Treasurer, John Howard.

This new party had no time for him, or for his ideas. The fashioning of the modern image of Howard began with a dubious legend of Howard, as Treasurer, chafed and frustrated by Fraser's caution and lack of economic radicalism. In this story, the Fraser years in government had been largely ones of missed political and economic opportunities, particularly in reshaping the economy.  This account had no reference to social policy, but, it seemed clear, that, particularly multiculturalism, was regrettable too.

Fraser had been cautious, if only because the manner of his achieving power (though swingeingly endorsed by the electorate) had created widespread public division and anger in the community. Although naturally a divisive politician, he had thought to reconcile. In any event, he had been sympathetic to some Whitlam-era reforms (particularly in administrative law, and land rights) and had recognised that his "mandate" (as he saw it to rescue Australia from the economic trashing caused by the Whitlam government) did not extend to repudiation of many of the social ideas of Whitlamism. On matters such as this Fraser was ahead of his party, and somewhat more in tune than it with the times.

And if, on economic matters, the budget had needed adjustment and tightening, it was also a period of increasing unemployment, further oil shocks, and inflation out of control throughout the world. Options were limited, and a good deal of work was need to stimulate the economy. In retrospect even the Review of Commonwealth Functions, as the first "Razor Gang" report was called, was a fairly tame document, even by the standards of the structural cuts, on Fraser-era expenditure, imposed by the Hawke governments in the later 1980s. Fraser was hardly resisting Treasury advice in his failure to advance or implement fiscal, monetary and structural reforms of the type implemented during the Hawke-era.

Fraser won three elections for his party, and lost his fourth, a good deal more narrowly than is now thought. There is a continuing political lesson in the way he rushed to an early 1983 election through sheer political opportunism that blew up in his face. He had been tipped off about a plan to replace the then Labor leader, Bill Hayden, with the more popular Hawke, and rushed to Government House for a dissolution before this could come to pass. Alas for him, the Governor-General, Sir Ninian Stephen, wanted more paperwork, but Labor's coup was already a fait accompli. Hayden later claimed, perhaps rightly, that a drover's dog could have won the election anyway. If that was true it was because the economy, never in robust health since 1974, was beginning to deteriorate again.

As Hawke cemented himself in power, Fraser's old party, once a broad church, became more narrow, and much more dominated by economic rather than moral or social thinking. Politicians of the sort that Victoria had specialised in - fiscal "managers" (conservative on public finances, but, if uninclined to increase the size of government, not much disposed to significantly reduce it), with generally liberal and progressive views on social and moral matters - became a threatened species. For some liberals, such as Ian Macphee, the dividing point was industrial relations; with Fraser it tended to be social policy: foreign aid, refugee rights and Aboriginal affairs. It took Fraser about 25 years to publicly say that he no longer felt any affinity with the party he had once led; at that point, for most Liberals, it had been at least 10 since they had realised they had no affinity with him.

Malcolm Fraser was still, comparatively, a young man when he became prime minister at 45 in 1975, but he had already been a member of Parliament for 20 years, and had served as a minister, off and on, over nine years, twice in Cabinet, as education and later defence minister.

In 1970, he played a critical role in bringing down John Gorton, whose stocks in the Liberal Party had declined in major part because he was an aggressive nationalist and centrist, whose government was thought to have undermined the states and conservative values. Fraser then portrayed himself as a minister let down by a chaotic and personal style of prime ministerial management, then undermined, leaked on and lied about. He had, then, no chance of taking over from Gorton, but marked himself as the conservative's conservative, the remaining Liberal of the Menzies generation most admired and respected by Menzies himself, and the person who stood for process, regularity and sound management. There were coalition men who were his peers: Doug Anthony, Ian Sinclair and Peter Nixon - some of whom, unlike Fraser, had first become ministers under Menzies. It was with these, as practical politicians if not as ideological bedmates, that he had most practical partnership.

If Fraser was then regarded as conservative, it was already known that he was not reactionary, as many of the party's then conservatives were. The litmus test tended to be racial horrors such as South Africa and Rhodesia rather more than Aboriginal affairs or "moral issues" such as attitude to divorce, abortion or capital punishment. Fraser, like his contemporary Gough Whitlam, had no racial consciousness, and had developed friendships and decolonisation at Oxford, which he had attended after Geelong Grammar. By contrast, even Jim Killen was rabid in support of Rhodesia. Many Liberals (including Howard) notionally disapproved of apartheid in South Africa, but saw such matters through a Cold War prism, distrusted African (and Asian) nationalists as possible communists, and would not lift a finger for black rights, and inclined to be suspicious and distrustful of those who were.

But it was as a conservative, not as a liberal, that saw supporters, led by Tony Staley, push his cause as an alternative to the hapless and embarrassing Billy Snedden in opposition in 1974. And, given the chaotic style of the Whitlam government, as well as the economic blows raining down from the first oil shock, he fashioned also a pioneering image of a man, and a party, dedicated to restoring a wrecked economy, run by vandals, into a sound one, run by prudent and sensible adults. Before and after November 11, Liberals and conservatives saw the denial of supply, however irregular, as the essential saving of Australia from a party that had become mad, and, quite likely, bad, as well as dangerous to know. Had the "rescue" not been effected, they thought, the nation faced stark ruin. This justified their desperate measures.

The standoff - until resolved by Sir John Kerr - was a close run thing, with a good deal of nerve and tension, and incredible, and lasting displays of personal bitterness and spite. It also marked the beginning of a period, which has largely lasted to the present day, in which the differences between governments and opposition, and some of the camps of supporters, were as much personal as professional, with either side almost pathologically unable to accord the other any credit for sincerity, honesty and simple, but honorable, difference of approach or philosophy.

But if in part responsible for such a divide, Fraser had by then long reconciled with his most bitter Labor enemies, and had won their respect and admiration, if not quite forgiveness. The better ones recognise, too, that it was Fraser's continuation of some Whitlam era policies that entrenched them in the system. This includes most of the actual progress in Aboriginal affairs since 1970, FOI, judicial review and the Federal Court and administrative appeals system, and much of our human rights law.

I first met Fraser in 1971, but he made his biggest impression on me in 1978 when, for the first time, he went to visit Aboriginal communities in Central Australia and was shocked by what he saw. Before then he had been vaguely benevolent and well-meaning on Aboriginal policy (if inclined to suspect that it was badly managed, with a good deal of waste). When he actually saw living conditions, particularly at Papunya, and saw how crude, pathetic and ineffectual the practical policies in place, he was staggered and distressed. He could hardly talk of anything else for days and, thereafter, was impatient for action if, still, relentlessly cautious about offending state sensibilities, always deliberate for progress.

I  had a somewhat similar experience a few years earlier, perhaps from a not dissimilar rural background, and was working in Aboriginal health care. It was then, earlier than a few of my generation, that I realised that the man was genuine, and decent, and human, and with all of the capacity to be great. Nearly 40 years later, on his moral and his personal qualities, he stacks up pretty well against his contemporaries and his successors.