Washington — Despite a major setback, the Reagan administration has reaffirmed its determination to press ahead with its search for an Arab-Israeli peace. Administration officials are clearly disappointed that King Hussein of Jordan appears to have ended his attempts to negotiate with PLO chairman Yasser Arafat an arrangement that would permit Hussein to enter peace talks with Israel. But the officials assert that President Reagan's peace plan is still alive.
The President himself plunged back into Middle East diplomacy on Monday with telephone calls to several Arab leaders. Behind all the high-level activity, however, Middle East specialists in the administration see no clear next step to take at this point in the face of what is viewed here as the PLO's inability to seize a historic opportunity.
''The President is determined to press on,'' says one official. ''The President and the secretary of state continue to rely on King Hussein's basic position, which is that he accepts Israel's right to exist. We continue to believe that King Hussein wants to enter the peace process, and will continue to seek support from other Arab leaders in order to do that.''
In an interview with ''CBS Morning News'' on Monday, Secretary of State George Shultz said that Hussein and PLO chief Arafat were close to a final agreement on Hussein's entrance into the peace talks when the PLO made unacceptable changes in its accord. The changes, said Mr. Shultz, included the requirement that the PLO be at the bargaining table and a demand for an independent Palestinian state.
The Israelis have repeatedly declared that they would reject PLO representation at peace talks. The idea of an independent state is anathema to the current Israeli government.
The Reagan plan introduced in September calls for a Palestinian entity on the West Bank of the Jordan River to be linked with the Kingdom of Jordan. It rejects Palestinian statehood.
But a number of specialists in the administration thought this might be enough to bring the PLO into the peace process behind Jordan. Jordan has declined to make a move without PLO acquiescence.
Jordan and the PLO had apparently reached agreement that the issue of PLO representation at peace talks be resolved by forming a unified negotiating delegation. PLO sympathizers, but not necessarily full-fledged PLO officials, would participate in the delegation. The PLO has now apparently backed away from the idea.
As seen from Washington, it is radicals in the PLO who have set back the Reagan plan. Those administration specialists who have argued that Mr. Arafat is a moderate who will eventually settle for a Palestinian entity linked with Jordan have been dealt a blow.
They still argue that Arafat is a moderate, but are now more inclined than ever to believe that the PLO chief will never take any risk that threatens to split his organization.
If this is the case, then some Israelis are correct in arguing that radicals in the organization hold what amounts to a veto.
Added to the power of PLO radicals is the behind-the-scenes influence of Syria, a nation that has always been able to play a spoiling role.
Some specialists outside the government have criticized the Reagan administration for what they regard as inadequate attention being made to the Syrian factor. Arafat's relations with Syria's President Hafez al-Assad have been severely strained.
But Syria backs an important faction within the PLO, and Syria seems to have tolerated, if not encouraged, the assassination by a Palestinian splinter group of Issam Sartawi, an adviser to Arafat who was considered in Washington to be a moderate.
''Arafat has got the Syrians breathing down his back,'' says William B. Quandt of the Brookings Institution, a former National Security Council staff director for the Middle East in the Carter administration. ''And Abu Nidal (leader of a radical splinter group) will assassinate people who go down the road to negotiations with Israel.''