Washington — Straws in the wind do not a Reagan-Andropov summit make. But there are just enough hints of an improvement in relations between the United States and the Soviet Union to make diplomats stop and think. It may be mostly tactics and atmospherics. But compared with the US-Soviet charges and countercharges of just three or four months ago, the rhetoric from both sides today sounds a bit more civilized.
Superpower debate has hardly reached summit levels, but it has risen above the depths.
The latest straw came floating in this past week from Madrid: Soviet diplomats accepted a Spanish proposal that may lead to an East-West compromise at the 35-nation conference reviewing the 1975 Helsinki Accords on Security and Cooperation in Europe.
The Soviets eased their opposition to a number of Western demands for human rights provisions to be included in any final agreement. The West at the same time has dropped some of its proposals to expand human rights pledges.
If a final document comes out of the Madrid meeting, it would be the first significant US-Soviet agreement to be reached since the Soviets invaded Afghanistan in 1979.
''In a period when East-West relations are not exactly warm, getting the Soviet Union to sign a document like this is not insignificant,'' says a State Department official.
President Reagan has yet to approve draft documents containing the Madrid compromise. Max M. Kampelman, the chief US negotiator at the East-West conference, was scheduled to return here during the weekend to discuss the issue with senior administration officials.
If the President approves, a final signing ceremony at the Madrid conference could take place within weeks. If the US sends Secretary of State George P. Shultz to the ceremony, it would bring together Mr. Shultz and his Soviet counterpart, Foreign Minister Andrei A. Gromyko, for the first time since the two met at the United Nations more than six months ago.
Other developments that could point to a modest improvement in US-Soviet relations include:
* A tentative agreement now under discussion to open new US and Soviet consulates in Kiev and New York. The US broke off such discussions after the invasion of Afghanistan.
* Possible agreement on verification of the still-unratified US-Soviet threshhold test ban treaty. Officials at the US Arms Control and Disarmament Agency (ACDA) say they are working hard to formulate verification measures that both sides can support.
* A new Warsaw Pact offer at the Vienna talks on reducing conventional military forces in Europe. ACDA officials say the Warsaw Pact has made ''very comprehensive proposals'' that include agreement on certain verification measures for the observation of troop withdrawals.
* The Soviets' recent release of 15 Pentecostalists, five of whom lived for nearly five years in the US Embassy in Moscow. The emigration of the Pentecostalists could not have occurred without some behind-the-scenes talks between the Americans and the Soviets. The release may have been a signal that the two sides were ready to do business at the East-West conference at Madrid.
* A West German hint that Chancellor Helmut Kohl had heard ''new'' ideas from Soviet President Yuri Andropov on ways to break the deadlock in the US-Soviet negotiations on reducing intermediate-range nuclear forces (INF) in Europe. The West Germans didn't disclose what these ideas might be. But they revealed that Mr. Andropov proposed extending the INF negotiations a week beyond the scheduled July 15 recess and resuming them on Sept. 8, a week earlier than planned.
Skeptics are quick to point out that none of the above amounts to any substantive improvement in US-Soviet relations. There is no indication of any real progress in the INF talks or in the strategic arms reduction talks. Much of what is being said and done still may amount to little more than propaganda gestures meant to demonstrate the superior good will of one side or another.
Because of the public relations work being done by all sides, it is difficult , even for experts, to separate sound and fury from substance.
But as he weighs a decision on whether to run for reelection next year, Mr. Reagan has every reason to want to defuse concern that he hasn't done enough to pursue constructive negotiations with the Soviet Union. Reagan and Andropov want to convince West Europeans that they are interested in arms control.
West Germany's Chancellor Kohl has every reason to sound reasonable about what he accomplished this past week in Moscow. Kohl is pushing for a US-Soviet summit, officials here suggest, partly because he doesn't want to appear to be taken unaware should an agreement to hold a summit suddenly materialize.
When it comes to the release of the Pentecostalists, skeptics note that Jewish emigration is at a low point, that Soviet authorities appear to encourage antisemitism, and that they continue to crack down on human rights activists, including some who have tried to monitor implementation of the 1975 Helsinki accords.
Some leading experts argue that Moscow's almost certain reaction to the planned deployment of new American nuclear missiles in Europe, beginning in December, will simply be to deploy new missiles of their own in European Russia instead of buckling under and accepting Washington's terms for negotiation. In the view of such experts, the Soviets do not want to give the appearance of weakness and of caving in to American pressure.
Among the ''optimists'' on the subject, William G. Hyland, a senior associate at the Carnegie Endowment for International Peace, says the Soviets are slightly easing East-West tensions at the moment in order to ''soften up the Germans.'' But he also says that when one adds up all these straws in the wind, it appears that the Soviets are trying to create a ''prenegotiating situation'' that could lead to real progress in the INF talks.
Mr. Hyland says he believes it is significant that a senior Soviet military official told US congressmen last week the Soviet Union is prepared to consider a proposal developed by Paul Nitze, US ambassador to the INF talks, which would have reduced Soviet SS-20 missiles from about 240 to 75, to be matched by deployment of an equal number of US cruise missiles.