Love story to a family flows with turbulence and calm; A River Runs Through It, by Norman Maclean, photographs by Joel Snyder. Chicago: University of Chicago Press, 124 pp. $25.

When Norman Maclean, professor emeritus of English at the University of Chicago, sent his first fiction manuscript to New York publishers, it was met with quick, curt rejections. The book was ''too Western,'' replied one publicist. ''Too many trees,'' criticized another.

But, publishers, take heed. Don't judge a book by it's trees. For when ''A River Runs Through It'' finally landed in the hands of readers in the spring of 1976, it rapidly acquired a devout following. Selling 20,000 cloth copies and 50 ,000 paperbacks, the book succeeded in a way that seemed remarkable for a university press that never before had published a work of original fiction.

Today the book, now in a fifth printing, is still popular. The new Christmas-gift edition includes 15 exquisite color photographs of Montana's Big Blackfoot River, which has captivated the book's readers over the years.

Taken from a slice of the author's life, ''A River Runs Through It'' is a story based on Maclean's family, their relationships to each other and to the river. It is a story that has managed to tug the heartstrings of literary buffs, outdoorsmen, fly-fishermen, and urbanites alike. For within these picturesque sketches of Montana living, Maclean relates a deeper message about the unspoken love that flowed through his family. The moments when this happened were clearest for him during the precious hours they shared fly-fishing.

''My family was all British by heritage. We were brought up in that kind of reserve that the British have,'' Mr. Maclean explained in a recent telephone interview.

''We had this very, very deep, rather inexpressive love for our family. . . . The river and fishing and our love for the woods were the one thing that we could all talk about. . . . The river was a kind of substitute; it was a way or a means of saying what we thought of each other by showing how much we knew about the earth and loved the earth. . . .''

So, when retirement years set in, Maclean settled down to translate his feelings into words. The result is a sensitive outpouring of both turbulent and calm recollections, which flow as gracefully as the river that runs through it. And, inadvertently, Maclean creates a stirring philosophical work that may prompt readers to deeper contemplation.

The reader gets to know members of the Maclean family. The author's father is an eccentric Presbyterian minister and fly-fisherman, who, in early years, tells Norman and his brother that ''all first-class fishermen on the Sea of Galilee were fly-fishermen and that John, the favorite, was a dry-fly-fisherman.''

Mrs. Maclean is a doting Scottish woman who packs lunches for their expeditions and thoroughly understands their preoccupation with trout. And Norman's brother Paul is the superior fisherman of the family. Much of the book is written about him, the black sheep in the family. His antics and independent nature eventually lead to tragedy. And from this comes the book's depth.

''Fundamentally, I think it's a love story to my family,'' Maclean commented about the writing of the book. ''I sometimes think of it more as a poem than as a love story; it's kind of a love poem.''

''A River Runs Through It'' was nominated for a Pulitzer Prize when first published, but the committee decided not to award a fiction prize that year.

Speaking of the photographs in the new edition, Brenda Nelms, a University of Chicago Press spokesperson, says, ''We thought people might enjoy seeing the locations of the story.''

Thus, Maclean and his photographer son-in-law set out for Montana to tackle the Big Blackfoot - not with their trusty fly rods and Bunyan Bug flies, but instead, with camera in hand. They floated downriver by raft, stopping to photograph the water's moods, the wild Montana terrain, and those illimitable trees.