ANTI-INCUMBENT fervor is a game of inches.In off-year elections this week, voters sent clear signals that insiders were out and that little tolerance was left for politics as usual. Yet the vote that held the greatest potential to remake the political power structure - Washington State's referendum on term limits for members of the US Congress, among others - went down to unexpected defeat. The most striking wake-up call to Washington this week was the Pennsylvania Senate race, where a prominent Bush Cabinet member and popular former governor was thoroughly thrashed in what looked like a shoo-in only months ago. Both the Pennsylvania and the Washington State votes stand as symbolic defeats for the Bush administration. The president had made several campaign trips for Republican candidate Dick Thornburgh, who ran the Bush White House's Domestic Policy Council until recently. The race is viewed as a test case where a hard-hitting, what-about-the-little-guy Democratic campaign message caught fire and scored a stupendous upset. The term-limit movement has the endorsement of the White House, most visibly Vice President Dan Quayle. It is part of the White House effort to run against a Democratic Congress, portraying it as the establishment that lives above its own laws. Mr. Bush appeared to heed the warnings. Tuesday, he announced he was canceling his coming trip to Asia; the time had come to focus on domestic concerns. Voters were sending signals from all over this week: * Sen. Harris Wofford (D) of Pennsylvania, who ran 44 points behind a few months ago, beat former governor and US attorney general Dick Thornburgh in a landslide upset. * Washington State voters narrowly defeated term limits for state legislators and US representatives and senators from the state that would have been the nation's most stringent. (Local voters approved term limits in Cincinnati, Houston, and Worcester, Mass., and rejected them in White Plains, N.Y.) * Mississippi Gov. Ray Mabus (D) was ousted by a Republican construction contractor, Kirk Fordice, who will be the first Republican governor of Mississippi in over a century. * Kentucky, which bars consecutive terms as governor, elected Democratic Lt. Gov. Brereton Jones as governor over Republican Rep. Larry Hopkins. * The New Jersey Legislature changed hands. All 120 Senate and Assembly seats were open, and the Republican takeover gives that party not only a majority but enough votes to override gubernatorial vetoes as well. The vote was seen as a reaction against tax hikes under Democratic Gov. James Florio. * Houston Mayor Kathryn Whitmire was ousted by two opponents who face a runoff. Republicans may have succeeded too well in embracing the term-limits movement. The term-limit proposal approved by California voters last year passed, but by far less than anticipated a few weeks before the vote. One explanation is that Republican supporters gave it a partisan edge voters were uncomfortable with. Another explanation is that voters are far more willing to tell pollsters they want term limits than to vote for them. Currently, national polls run about 74 percent in favor of term limits. Term-limit referendums are rolling across the US - an efficient tool for ousting not only the voter's own representative but others as well. As many as 18 states may vote on them next year. In rejecting term limits, Washington voters refused to make the ultimate sacrifice of power for their chosen principle. Washington Rep. Thomas Foley is the most powerful single member of Congress. The message of term-limit advocates was that incumbent congressmen lose touch because they are practically permanent. The numbers back up the permanency argument. Of the incumbent House members who ran for re-election in 1990, 96 percent won. But even then, the anti-incumbent mood was beginning to show. Most recent elections have sent 98 or 99 percent of incumbent candidates back to office. In 1990, twice as many incumbents won with less than a landslide 60 percent of the vote as in the previous election. Fifty-three of them won with their lowest winning margins ever. Still, notes Texas A&M political scientist George Edwards, three-quarters of the incumbent congressmen running in 1990 were reelected with more than 60 percent landslides. "When people are elected to Congress, they tend to stay there until they want to leave," Dr. Edwards says.