Washington — When Walter Mondale's staff members learned about the format for this Sunday's Democratic presidential debate, they were not pleased. ''We believe the voters want to hear a serious discussion of the issues. We don't think voters want to be entertained as if this were some kind of TV show, '' one top Mondale worker complains.
Such concern may be understandable. Mr. Mondale, who has steadily widened his lead over the seven other Democratic candidates, clearly has the most to lose in the debate to be broadcast nationwide (Sunday on PBS-TV, 3-6 p.m. EST). The program, sponsored by the House Democratic Caucus, features a ''free form'' style which could result in one of the liveliest political performances in years. Even the audience may take part.
''It's a gamble, a leap of faith,'' to put on a show of this kind, says US Rep. Charles E. Schumer, the Brooklyn Democrat who was the prime mover behind the debate. ''We're trying to explore new ground'' while avoiding the ''very, very tedious'' style of formal presidential debates, he says.
The first 90 minutes will be moderated by Ted Koppel, anchor man of ABC-TV's ''Nightline'' news program. Organizers are hoping that Mr. Koppel will keep the debate moving along at a brisk pace, while ''not letting any of the candidates get away with anything.''
The second 90 minutes will be turned over to TV personality Phil Donahue. His job will be to draw questions and comments out of a specially selected, politically balanced audience of about 300 at Dartmouth College in Hanover, N.H. , where the debate will be held.
Press secretary Maxine Isaacs says Mondale aides tried to have the format altered to a more traditional style.
''We had hoped to find a way to make the debate more substantive. We have no fear of debates, including this one. Walter Mondale appeared in 26 joint appearances with other candidates in 1983, more than anyone else. One element of our strategy is that Mondale benefits every time he is compared with his opponents.''
But the risk to Mondale is clear. Political analyst Austin Ranney points out that Mondale must go toe to toe for three hours with seven other contenders ''who don't have much to lose.'' If he waffles, or gets upset, or looks uninformed on a subject, it could be a sharp setback. Meanwhile, the others can pound the table, take strong positions, and make Mondale look weak.
Other candidates, on the other hand, are delighted.
''This is wonderful,'' says Oliver Henkel Jr., campaign manager for Gary Hart. ''You get tired of the same old business, the 45-second answer that everyone has heard before. Koppel is a much more penetrating interrogator. We welcome this kind of challenge.''
While the debate will be broadcast only on the Public Broadcasting System, it is still expected to draw an audience of about 5 million. Additional listeners will be able to tune in on National Public Radio, where it will be aired on stations across the country. Segments of the debate will also be seen on regular news broadcasts.
All of this could make the debate the biggest political event of January, just as the nation draws near the first presidential caucuses, in Iowa on Feb. 20, and the first primary, in New Hampshire on Feb. 28.
The debate's unique format was developed by a committee of five congressmen assisted by TV producer Norman Lear; Charles D. Ferris, former chairman of the Federal Communications Commission; Arnold Picker, former chairman of United Artists; Richard Kline, a Democratic fund-raiser; Marsiarose Schestack, a Philadelphia TV personality; and Vernon Jordan, former president of the National Urban League.
Past debates have had dramatic results.
It was the 1960 presidential debates that were credited with helping John F. Kennedy score an upset victory over Vice-President Richard M. Nixon. In early 1980, the Republican presidential debates (ignored by Ronald Reagan) catapulted little-known John Anderson into national attention, and later helped him wage his campaign as an independent. And it was the Cleveland debate between Mr. Reagan and President Jimmy Carter, observes Ranney, which convinced voters that Reagan was no ''bomb thrower,'' and that he could be trusted by those who were weary of Carter and were looking for a credible alternative.
This debate is just the first of several scheduled for the primary season - and any of them could provide a surprise breakthrough for a Mondale challenger.
Many eyes will be on Jesse Jackson, fresh from his Syrian triumph. But strategies are being carved out by the other candidates - John Glenn, whose second-place campaign has been faltering; Reubin Askew, who appeals to conservatives and moderates; Gary Hart, who hopes the debate focuses on issues; Ernest Hollings, with sharply honed debating skills; Alan Cranston, who has widened his appeal beyond the nuclear freeze issue; and George McGovern, whose positions appeal to liberal activists in the party.