Canberra — Prime Minister Bob Hawke's Labor Party has been returned to power in Australia but with a reduced majority caused largely by a new voting system that backfired on the Labor government.
Mr. Hawke - who had been expected to win by far larger margins - is also likely to have his knuckles rapped by Labor colleagues for agreeing to hold Australia's first televised political debate. A strong performance by opposition Liberal Party leader Andrew Peacock is believed to have cost Labor votes as well.
The prime minister is nonetheless expected to be reelected Labor Party leader sometime in the next week and to be given a green light to continue policies that he has championed during his 20 months in power. To what degree he will be able to keep the left wing of his party in line, however, remains to be seen.
Hawke, like the majority of his party, advocates nuclear arms reductions and a nuclear free zone in the South Pacific. But he does not support the left wing's view that foreign military bases, particularly United States bases, be banned from Australia.
The voting system - which turned out to be the surprise fly in the election ointment - was intended to simplify Australia's voting for the Senate. In last Saturday's vote, citizens were allowed to mark the name of their top choice among electoral candidates, leaving the other boxes blank on the ballot. It was a shift from the previous system in which voters were asked to rank all the candidates in order of preference, meaning every box had to be marked.
The system went awry because many voters followed the same procedure in voting for candidates to the House of Representatives, although the old rank-all-candidates system was still in effect for House voting. Half a million votes - or about 7 percent of the total vote - were declared invalid, largely because of mismarked ballots for the House. That represents an increase of 5 percent over the number of ballots declared invalid at the last election. The ''simplified'' system did reduce confusion in ballot-marking for the Senate candidates, with 5 percent declared invalid, compared with 10 percent in 1983. Hawke and many other electoral analysts say most of the mismarked votes this election probably would have been cast for Labor. These invalid votes would probably have given the Labor Party an additional 2 or 3 percent of the vote, analysts say.
Until the votes were counted, public opinion polls and exit polls taken at polling stations were predicting an increase in the Labor vote of about 2 percent over its 1983 election win.
Instead, the Labor Party won only 47.9 percent of the vote, down 1.6 percent from its 1983 total.
The Liberals scored 34.2 percent, a small drop from their previous vote, while the rural-based National Party increased its vote by almost 2 percent to 10.8 percent.
Because of a redistribution of electoral boundaries designed to rectify a previous imbalance against the Labor Party, Hawke's government will have only a slightly reduced majority in the House of Representatives, about half its previous majority of 25 seats.
In the Senate, the fledgling Nuclear Disarmament Party gained about 8 percent of the votes in major states, but it is unlikely to win sufficient support from other parties to obtain the 12.5 percent it needs to elect a senator.
Control of the Senate will remain in the hands of the Australian Democrats, an independent party that occupies the ideological center between Labor and the opposition parties, and that, in the past, has allowed most of the Labor government's legislation to pass.
Besides being criticized for the televised debate, Hawke is also likely to be criticized by the Labor caucus for conducting an eight-week campaign instead of Australia's usual four weeks on the hustings. This went against the advice of party officials.
Hawke said after the election that it was ''galling'' that Labor's vote had suffered through technicalities in the voting system.
But he said Labor had a comfortable working majority in the House, and he was confident Labor would be reelected at the next election, in 1987-88.
Mr. Peacock's performance in the debate, and his steadfast insistence that he could win the election despite the polls, are likely to result in his retention of the Liberal leadership.
But there is likely to be a major review of the opposition's strategies, which resulted, for the first time in Australian political history, in every major Australian newspaper switching from supporting the conservative parties to supporting the Labor Party.