The Dow Jones industrial average closed down 89.24 points to 12,878.13. The Dow and other indexes had been moving between small gains and losses for most of the day, then turned lower after Reid's comments in the early afternoon.
"We have to get away from the happy talk and start talking about specific things," Reid told reporters in televised comments.
Worries about the budget talks have been hanging over the stock market for weeks. Stocks slumped immediately after the Nov. 6 election over concerns that politicians would be unable to reach a deal to trim the deficit before a Jan. 1 deadline.
If that deadline isn't met, under current law a series of sharp tax increases and spending cuts will come into effect. Economists have warned that the measures could push the economy back into a recession. That deadline has come to be known as the "fiscal cliff."
"The markets are getting whipped around, rather sharply, on headlines," said Sal Arnuk, co-founder at Themis Trading. "For example, Harry Reid feeling we're not making enough progress on the fiscal cliff."
Last week stocks pared some of the losses that followed the election. President Barack Obama plans to make a public case this week for his strategy for dealing with the issue as he pressures Republicans to allow tax increases on the wealthy while extending tax cuts for families earning $250,000 or less.
The S&P declined as much as 5 percent in the weeks after voters returned a divided government to power, with President Barack Obama returning to the White House and Republicans retaining control of the House.
Earlier in the day, investors took little comfort from the latest deal to deliver financial aid to Greece and increases in U.S. consumer confidence and orders for machinery and equipment.
While stocks have gained this year as the Federal Reserve has maintained it bond-buying stimulus program, concern about global growth and the budget fight in Washington may limit further advances, said Uri Landesman, President of Platinum Partners, a New York-based hedge fund. The S&P is up 11 percent this year, the Dow 5 percent.
"The glass is half-empty right now," Landesman said.
The S&P rose as high as 1,465 in September, the highest in almost five years, after the Federal Reserve said it would extend its so-called quantitative easing program and buy more bonds. The program is intended to lower borrowing costs and stimulate hiring.
Two reports that suggested that the outlook for the U.S. economy may be improving failed to encourage investors to push stocks higher.
Consumer confidence rose this month to the highest level in almost five years, pushed up by steady improvement in hiring. The Conference Board's consumer confidence index rose to 73.7 in November from 73.1 in October. Both are the best readings since February 2008.
The government reported separately that U.S. companies increased their orders of machinery and equipment last month, a sign that business investment is rising. Orders rose 1.7 percent in October, the best showing since a 2.3 percent rise in May.
The yield on the 10-year Treasury note edged down to 1.65 percent.
Among other stocks making big moves:
—ConAgra advanced $1.34, or 4.7 percent, to $29.63 after it agreed to buy Ralcorp for $5 billion in a deal that will make it the nation's biggest maker of private-label foods. Ralcorp surged $18.57, or 26.4 percent, to $88.80.
—Corning Inc., a specialty glass maker, rose 78 cents, or 6.9 percent, to $12.13 after it said that North American television sales are stronger than expected in the fourth quarter, boosting demand for its products.
— Las Vegas Sands Corp. rose $2.33, or 5.3 percent, to $46.36 after the casino operator said it would pay a special dividend of $2.75 per share, distributing about $2.26 billion to shareholders before the end of the year.
—Monster Beverage Corp. soared $6.69, or 13.3 percent, to $51.97 as concerns eased about increased regulation for the energy drink maker following a letter to senators, made public Tuesday, from the U.S. Food and Drug Administration.