Washington — On the eve of his second term, President Reagan finds himself on the horns of a foreign-policy dilemma: whether or not to accept the ruling of the World Court that it has jurisdiction to hear Nicaragua's suit against the United States for mining Nicaraguan harbors and supporting the anti-Sandinista rebels.
The issue has diplomatic, political, and moral implications.
If the United States refuses to take part in the court's consideration of the complaint, it will face international disapproval for defying a respected juridical body and the rule of law it professes to advocate.
The White House and State Department have expressed disappointment in the World Court ruling. American officials say a decision in the matter will be made as early as possible.
If it does participate as defendant in the suit, the US risks an eventual court finding that its paramilitary activities in Nicaragua violate international law and therefore should cease. Such a verdict, even if the court cannot enforce it, could be exploited for propaganda purposes.
It will be many months before the merits of the case are heard, but the ruling this week by the International Court of Justice (the World Court's official title) has touched off debate in the administration which the President alone can resolve. The court, rejecting the US argument that it had no jurisdiction in the case, ruled Monday, 15 to 1, to hear Nicaragua's complaint.
International legal experts differ on the soundness of the World Court's decision. Some believe, as US lawyers argued, that such a politically charged issue, involving armed conflict, should more properly be dealt with in the UN Security Council rather than by a limited juridical body. Others disagree. But now that the World Court has made its decision to hear the case, international lawyers say, the US would be wise to abide by that ruling. Otherwise, they say, the country's moral standing in the world as a consistent supporter of an international order governed by law would be damaged.
''If the court ruled against the US, it would have a limiting effect on our freedom of action,'' says Lee Marks, the State Department's senior deputy legal adviser during the Carter administration. ''But the larger political question is whether the total freedom of action in Nicaragua, which is a temporal foreign policy issue, is more important than trying to uphold and increase respect for an international legal order and the role of the World Court.''
The court's ruling was a ''powerful vote,'' notes Mr. Marks, taken primarily not by third-world or communist nations but largely by judges from such friendly and allied countries as Britain, France, West Germany, and Japan.
Although the World Court has heard only about 30 cases since it was established in 1945, it has proved to be a useful forum and the US has strongly supported it. Earlier this year the court adjudicated a sea-boundary dispute between Canada and the US. In 1980 the US took the Iranian hostage case to the court, whose decision against the Khomeini regime played a role in the ultimate release of the American prisoners.
''If we turn our back on the court, we could be accused of hypocrisy,'' says Herbert Hansell, former State Department legal adviser. ''We would be proceeding in a diametrically opposite direction of our advocacy of the rule of law in world affairs - because the minute we get a decision we don't like, we send a signal that we pick up our marbles and go home.''
Whatever the diplomatic risks, many legal experts say they believe that the Reagan administration should choose the option of arguing the case at The Hague (where the World Court sits). They say the US would be able to highlight legitimate arguments of collective self-defense and regional security in Central America. The very presentation of the US case would also bring pressure on the Nicaraguan government to desist from any threats to peace in the region.
''There is no way the US can get out of the present case,'' says W. T. Mallison, a legal scholar at George Washington University. ''It seems to be taking, to its embarrassment, the same position Iran took in the hostage case - that this is a complicated political issue and therefore the court is the improper forum to litigate the case. But the US is following a very undesirable precedent, and if we do not even present our arguments, that would diminish our integrity.''
Before the suit was brought last April, the Reagan administration suspended for two years US recognition of the court's jurisdiction with regard to Central America. But it did not give the required six months' notice to file its reservation, doing so only days before Nicaragua brought its complaint.
Some experts challenge the US government view that the court should not hear political questions. Mr. Hansell suggests that there is no doctrine or principle which says the World Court should refrain from issues with political overtones, and that the US cannot require that the court renounce jurisdiction in such questions.
But, while supporting the court's ruling and urging the US to present its case, some specialists say they think there are substantive aspects of the case which the court should not get into.
Seymour J. Rubin, executive director of the American Society of International Law, says such issues as whether the Nicaraguans are fomenting revolution in El Salvador or the US is arming the Nicaraguan rebels are ''inherently not susceptible of resolution at The Hague'' and can be dealt with only in a political process. The mining of harbors and compensation for such action, however, are justiciable, he says.
Whether or not the White House decides to participate in the case, this week's judicial action is expected to bolster congressional opposition to renewing US aid to the contra rebels in Nicaragua.