Washington — Samuel P. Huntington, a history professor at Harvard University, has been a loyal Democrat. He worked for President Jimmy Carter as a national-security adviser from 1977 to 1981. This year he supported Walter Mondale, despite prospects of a Reagan landslide. But now the professor has doubts about 1988.
The Democratic Party, Dr. Huntington says, ''lacks a vision'' and has shown ''inflexibility.'' Without strong leadership in Washington, the party, he contends, has largely been captured by narrowly focused interest groups.
All this is severing some old ties to the party of Franklin D. Roosevelt and John F. Kennedy, the professor says. He found more evidence of that recently when he attended a party with about 50 intellectuals in Cambridge, Mass.
''I was about the only person around who had voted for Mondale,'' Huntington said with some surprise. ''These were all people who were good Democrats . . . 8 or 10 years ago.''
Are the professor's concerns about the party and its future exaggerated? Some experts think so.
Even though the Democrats lost the White House again in 1984, they gained two seats in the US Senate, and they still easily control the US House of Representatives. Furthermore, out in the countryside, they have about two-thirds of the 50 state governorships and control most state legislatures.
One political scientist suggests that there is no problem the Democrats have that could not be cured immediately if the unemployment rate moved above 10 percent.
The Democratic comeback in Washington would be swift.
Yet in private conversations, many leading Democrats lament what is happening to their party, and they concede that things may get much worse.
Robert Strauss, a former Democratic chairman, said after last month's election that interest groups will continue to reach for control over the party:
''The defeat will mean nothing to them,'' he told a reporter.
''The hunger of these groups will be even greater,'' he added. ''Women, blacks, teachers, Hispanics. They have more power, more money than ever before.
''Do you think these groups are going to turn the party loose? Do you think labor is going to turn the party loose? Jesse Jackson? The others? Forget it!''
Democratic problems, of course, are most pertinent to those living in the United States.
But there could also be repercussions around the globe - in the free world, the third world, and in communist-dominated nations - if the party continues to decline in influence in Washington.
Some political scientists, including Austin Ranney of the American Enterprise Institute (AEI), note that Democratic woes have become more and more serious since Vietnam.
That war shook America's confidence in itself, Dr. Ranney says. But it appears to have left one of its most lasting marks on the Democrats.
The party of Roosevelt and Kennedy, notes Ranney, was one that supported a strong foreign policy and heavy spending for defense. Until the early 1970s, it was outward-looking, the party of the Marshall Plan, the Truman Doctrine, and NATO.
Since Vietnam, Democrats have allowed Republicans to dominate the defense debate.
It is basically the Republican agenda - a rapid military buildup, toughness toward Soviet ambitions, and a hard line on arms talks - that sets the tone for American policy overseas today. The Democrats are deeply divided between old-line internationalists and newer groups that Ranney and some others suggest are trying to move the party toward isolationist positions.
This policy shift, which has left many longtime Democrats feeling uncomfortable, has been accompanied by a demographic change in the makeup of the party.
Professor Huntington, who prepared a paper on ''The Democratic Vision'' for the AEI's Public Policy Week recently in Washington, notes that at one time the core of the party's support lay with the New Deal coalition put together by Franklin Roosevelt.
It consisted of Southern whites, Northern city political machines, labor union members, and new immigrants, largely from Southern Europe.
That coalition has been challenged, and even overwhelmed, in recent years by what Dr. Huntington calls the ''new politics coalition.'' This new power within the party is made up largely of blacks, youth, feminists, liberal intellectuals, and Hispanics.
A third grouping, which supported Sen. Gary Hart this year, has also arisen. These voters, mostly members of the ''baby boom'' generation now in their 20s and 30s, are wealthier, better educated, and more independent than other Democrats.
Tugging and pulling between these three groups marked this year's voting in the Democratic primaries. It also steadily pushed Democratic Party positions leftward, away from a majority of voters.
In each major policy dispute, Huntington says, the ''new politics'' group held sway because of its strategic position on the left side of the party.
On foreign policy, for example, old-line internationalists wanted a more conservative stance.
But blacks, feminists, and others in the ''new politics'' group teamed up with young, affluent Democrats to push through liberal positions.
On economic policy, young, affluent voters wanted more conservative positions.
But this time, blacks, Hispanics, feminists, and others allied themselves with New Dealers, including the unions, to push through liberal policies.
On social policy, where New Dealers were generally more conservative, liberal policies again held sway. Again, it was an alliance between the new politics groups and the ''young affluents.''
These liberal positions drove away enough voters - particularly Southern whites and Roman Catholic conservatives in the North - to hand Ronald Reagan an easy victory.
There is considerable uncertainty over where the party goes from here. A number of its leaders feel boxed in by President Reagan and his policies.
The Reagan tax cuts of the past few years have sent the budget deficit soaring. This has created a climate of budget austerity.
Even Democratic activists are being forced to look at budget cutting and smaller government as a possible alternative to deficits.
President Reagan, of course, favors budget cuts, a smaller role for the central government, and a larger private sector. Until the Democrats can come up with a package that looks more attractive, their troubles could continue.