FOR the Germans, who had more riding on the success of Soviet leader Michael Gorbachev than any other West European country, his sudden removal is devastating news.Chancellor Helmut Kohl broke off his vacation in Austria yesterday and rushed back to Bonn, where he met with the parliament's party leaders and Foreign Minister Hans-Dietrich Genscher to discuss the crisis. According to a government official who asked not to be named, Bonn was completely surprised by Mr. Gorbachev's removal and unprepared for it. According to an official statement, the German government is "deeply disturbed and concerned" by the developments in the Soviet Union. Chancellor Kohl said he expects Moscow to live up to its international obligations and reminded the Soviets of their duties under the Paris Charter, which last November set the course for the democratization of Eastern Europe, including the Soviet Union. Of greatest concern to the Germans is the fact that, although a troop pullout has begun, there are still 416,000 Soviet troops, civil workers, and their families stationed on German soil. Those who have taken control in Moscow are assuring the world community that they will honor their international obligations. But this is no guarantee, warns Roland Freudenstein, a specialist on East Europe for the German Society for Foreign Affairs in Bonn. Such a statement is to be expected from Communist hard-liners, says Mr. Freudenstein. "They will have enough problems with their domestic situation. They don't want to complicate things" by adding a foreign policy element now, he says. Since reunification, Germany is geographically and politically closer to Eastern Europe. What happens there is of utmost importance to Bonn, which sees itself as a bridge between East and West Europe. Should instability spread in the region as a result of the about-face in the Soviet Union, it could have a major effect on Germany. The Germans fear, for example, that yesterday's events will trigger a stampede of nervous East European and Soviet refugees toward their country - though it is not clear whether Soviets will be able to leave under Moscow's newly declared six-month state of emergency. Germany is already flooded with economic refugees and ethnic Germans from East Europe, and politicians here are debating changing the asylum law in the constitution in order to bring the situation under control. In the last year, Chancellor Kohl has been Mr. Gorbachev's greatest advocate in the West, calling more loudly than London, Paris, or Washington for swift and generous help for the Soviets. Political analysts say that over the next few weeks Bonn will be in a difficult position as it tries to remain friendly enough with Moscow to ensure an on-schedule Soviet troop pullout, yet react with enough abhorrence to what appears to be a complete turnaround from a democratic course. The West may very well be faced with the question of economic and political sanctions against the Soviet Union. Bonn will steer its reaction fairly closely to coincide with the West's, predicts a European diplomat here, but "it will be disinclined to take a totally negative attitude" about the Soviets, he said. Indeed, the German official who asked not to be named, argued that Soviet hard-liners had not won yet, and that the democratic movement in the Soviet Union was not yet dead. "This is not the last word," he said about yesterday's ousting of Gorbachev, adding that more time was needed to study the developments. Kohl said yesterday he was right in using last year's opportunity for speedy reunification.