In 1981, Pocket Books paid $2.3 million for the paperback rights to John Irving's ''The Hotel New Hampshire.'' Ron Busch, the company's president, has since admitted that Pocket Books lost $1 million on the deal. It appears that the readers did not march into the stores to buy Irving's novel in paperback in the numbers that Pocket Books expected.
If reprint rights to all cloth books could be sold to paperback houses for six-plus figures, no hard-cover firm would ever feel the need to invade the paperback producers' turf by forming its own mass-market imprints.
But that's impossible because we, the readers, do not even buy enough ''average'' reads to justify such extravagance. Even when midlist books (''good'' books without huge ad budgets) are published, they are rarely promoted. We may not even hear about them. Unless there's a fluke, of course, and then publishers love us. But what is a hard-cover house to do when it can't make money selling reprint rights? If a paperback company isn't already in its family, it can start a new paperback line.
Early next year St. Martin's Press, which publishes James Herriot and other best-selling authors in hard cover, will establish a mass-market line. Its first book will be ''Death in Zanzibar,'' by M. M. Kaye, whose most successful novel, ''The Far Pavilions,'' sold millions of copies. Her later books haven't scaled such heights. St. Martin's will spend a minimum of $50,000 on each book.
Will we buy them? St. Martin's must think so. Risking the costs of mass-market production and promotion may be a wiser business decision than selling the rights to a paperback publisher for less money. By holding on to the books, St. Martin's will have to share paperback income with the author alone. Even if the sales of the mass-market edition go awry, some people will still say St. Martin's made a noble gamble.
We should remember that the nobility of publishing is anchored, however loosely, in the bottom line.
But we readers are hard to predict. Harper & Row thought that we would care only a little about ''In Search of Excellence: Lessons from America's Best-Run Companies,'' by Thomas Peters and Robert Waterman Jr., so they printed only 10, 000 copies at first. Surprising everyone, we bought enough hard-cover copies of ''In Search of Excellence'' to send the number in print to 1,050,000 copies in less than a year (an achievement rare indeed). Now similar books are scheduled from a variety of publishers. Should we fail to produce frenzied sales of those book-alikes, our enemy status is likely to be resumed. But if another midlist book hits the big time, we'll be the darlings of publishers again.