Foreign Minister Kevin Rudd announced his resignation during a news conference in Washington, where he was visiting on official business, saying speculation that he planned to seize power from Gillard had become a distraction.
"I can only serve as foreign minister if I have the confidence of Prime Minister Gillard and her senior ministers," Rudd said.
Gillard ousted Rudd as prime minister in June 2010 in an internal coup, and their center-left Labor Party scraped through elections later that year to lead a minority government. Polls now suggest Labor would suffer a devastating defeat, but Gillard maintains she has her colleagues' support.
Gillard was to hold a news conference Thursday to "make a further statement" on Rudd's resignation. Media reports and current and former Labor lawmakers said she will announce a leadership ballot of party lawmakers on Monday.
A Rudd supporter, Sen. Doug Cameron, said a Monday poll would be unfair because Rudd would not have time to canvass support.
"It's clear that some senior ministers are intent on putting a stake through Kevin Rudd's heart and I don't think that's justified," Cameron told Australian Broadcasting Corp. television.
Rudd left open the option of quitting politics, which would trigger a by-election and could cost Labor its single-seat majority in Parliament. That would give the conservative opposition coalition the chance to form a new government if it can win the support of independent legislators, or it could force early elections.
In apparent anticipation of a Rudd bid for the party's leadership, Gillard deputy Wayne Swan issued scathing criticism of the former prime minister.
"For too long, Kevin Rudd has been putting his own self-interest ahead of the interests of the broader Labor movement and the country as a whole, and that needs to stop," he said in a statement.
Labor senior strategist Bruce Hawker said he spoke to Rudd before his announcement and that Rudd is likely to challenge Gillard.
Before Rudd announced his resignation, Gillard had refused to comment on media reports that she intended to fire him as foreign minister for disloyalty. Rudd then criticized Gillard for failing to defend him from colleagues' criticisms that he was undermining the government through his own leadership ambitions.
Gillard said she was taken by surprise by the resignation.
"I am disappointed that the concerns Mr. Rudd has publicly expressed this evening were never personally raised with me, nor did he contact me to discuss his resignation prior to his decision," she said in a statement.
Rudd said he planned to fly back to Australia on Thursday to sort out his future. But in his resignation speech, he was highly critical of Labor's attacks on him.
"I can promise you this: There is no way — no way — that I will ever be party to a stealth attack on a sitting prime minister elected by the people," Rudd said. "We all know that what happened then was wrong and it must never happen again."
Rudd said his duties in Washington will be fulfilled by Australia's ambassador, Kim Beazley, a former Labor leader whom Rudd deposed in a party ballot in 2006 with the support of Gillard as his deputy.
Many Australians were angry when the government dumped Rudd, who was swept into office as prime minister by general elections in 2007. In Australia's system, the prime minister is chosen by a majority of lawmakers in the House of Representatives, not by voters.
Labor lawmakers moved against Rudd in 2010 because opinion polls suggested they were unlikely to win elections that year under his leadership.
After the 2010 elections, Labor under Gillard formed the first minority government in Australia since World War II.
Opposition leader Tony Abbott said Rudd's resignation confirmed that the government is unworthy to continue in office.
Associated Press writer Kristen Gelineau in Sydney contributed to this report.