NOT all the Rembrandts in London are in the National Gallery on Trafalgar Square. Nor are all the Van Goghs in the Tate Gallery on the bank of the Thames. In fact, the British capital is remarkably well endowed with smaller museums and art galleries where an extraordinary range of superb art can be seen.
According to one guidebook, there are 150 museums. At the top of the scale are the National, the Tate, the vast Victoria and Albert Museum, and the stupendous British Museum.
But some of London's finest treasures are enjoyably displayed in places much less on the beaten track. And many are in buildings intriguing in themselves - often one-time private houses.
There's the exceptional Wallace Collection for instance. A collection of world status, it is notable for paintings, but also for furniture and sculpture. Sevres porcelain and clocks, majolica and armor, medals and miniatures. Originally, it was the collection of one family, and particularly one member of that family, the fourth marquess of Hertford, but it is only part of his collection. Brought over from Paris in the 1870s by Richard Wallace, the marquess's natural son, the collection was later bequeathed to the British nation by his wife. Her stipulation was that nothing could be borrowed from it nor added to it.
Museums often have a predominant atmosphere of taste or period, and the Wallace Collection's is surely French 18th century. Here you can see a brilliant gathering of Watteaus and Bouchers, Fragonards, Paters, Lancrets, and Greuzes. No Chardins, though, which does hint at the Hertford preferences. The emphasis tends toward pleasure rather than realism, the soberness of a schoolteacher or scullery maid or saint being not so favored as a fashionably pretty woman.
Nevertheless, the Wallace Collection's range is not confined to the fetes galantes of Watteau and company; there are notable Murillos, a number of Rembrandts (his son Titus portrayed in his teens among them), fine de Hooghs, Hals's well-known ''Laughing Cavalier'' (though strictly neither laughing nor a cavalier, as the catalog observes), ''Perseus and Andromeda,'' by Titian, and ''Lady With a Fan'' by Velasquez. There's the mix of grace and ponderosity of Poussin's ''Dance to the Music of Time'' and a group of excellent works by Reynolds. One happy opportunity is to be able to compare three different styles. There are three portraits of one woman on view, Mrs. Robinson, by Reynolds, Gainsborough, and Romney - three of England's most imaginative portraitists. Flemish 17th-century Rubens is also here in exuberant force, my own favorite being his harvest-glowing, pervasively vital ''Rainbow Landscape.''
A small London museum with a superlative collection of Rubens, however, is the Courtauld Institute Galleries. This collection is London University's answer-in-the-making to Oxford's prestigious Ashmolean Museum and Cambridge's Fitzwilliam - though it is considerably smaller and far younger (its opening was 1958). It is, nevertheless, an outstanding collection, or a joining of several collections. The latest is the Princes Gate Collection given by Count Seilern and first shown at the galleries three years ago. It includes 23 drawings by Rubens and 32 paintings - most of them oil sketches by him - and a whole lot more.
Samuel Courtauld's own collection-within-the-collection is remarkable for its Impressionist and Post-Impressionist works: a glorious bunch of Cezannes, Renoirs, Seurats, Monets, and two wonderful Van Goghs. There are numerous other fine paintings and drawings, but the collection is dominated, I always feel, by the last large-scale Manet, ''Un bar aux Folies-Bergere,'' that paint-rich celebration of the brilliance of artificial light, of shine and sheen, of mirror and marble, of glass, of the skin of oranges, with the immobile girl standing behind the counter like a figure from an altarpiece, her expression slightly abstracted, her complexion peach-clear.
The only problem is that until Sept. 1, this masterpiece and all the other Impressionist pictures are on tour in Japan and Australia. The Courtauld gallery staff meanwhile has to pacify disappointed tourists expecting to see paintings for which the museum is justly famous.
This London University collection has now far outgrown the pleasant galleries that still house it, so only a selection can be seen at any one visit. A plan to display it much more adequately at Somerset House still seems a number of years away.
One small and unusual picture, by Pieter (''peasant'') Brueghel is also sadly not to be seen at the Courtauld Galleries. But this is because it was stolen in 1982. It is a grisaille painting of ''Christ and the Woman Taken in Adultery.'' Curators are optimistic that it will eventually reappear. But for now this heartfelt little picture, which the artist deliberately never sold, (apparently because he wanted its message of religious tolerance for his own contemplation), is an absent witness to the security problems faced by London's smaller museums.
No museum has been a victim in this respect more strangely than the Dulwich Picture Gallery. Opened in 1814, 10 years before the National Gallery, it was the earliest public art gallery to open in London. This gallery houses no fewer than seven Poussins, four Murillos (two of urchins), six Van Dycks (two of them fine contrasting male portraits), plenty more works by Rubens (one the small but delicious ''Hagar in the Wilderness'') and three Rembrandts. Or at least there should be three Rembrandts.
The catch is that his portrait of ''Jacob de Gheyn'' is missing again. Again because this is the fourth time it has been stolen. The portrait, missing since May 1983, has in the past always come home again. But what it is about this painting that so appeals to thieves no one knows. The lack of security in the museum does, however, present serious difficulties to its governors - and to current visitors who miss certain of the collection's gems because they are being rendered burglar-proof. Visiting this attractive, well-lit gallery, in a building of enchanting neoclassical scale and detail designed by Sir John Soane, involves a fairly long journey from London's center. But choose a sunny afternoon, and nothing could be pleasanter.
Grossly underfunded, the Dulwich Picture Gallery is embarking on an effort to publicize its treasures and appeal for some (STR)500,000 (about $720,000). A selection of its paintings are being shown at Agnews in the center of London (March and April) in aid of this drive for funds. For five or six months in 1985 , 35 of its finest masterpieces will visit the National Gallery in Washington, D.C., and perhaps another museum in the United States. Without such drastic measures, rising costs seem sure to close the gallery to the public.
If the Dulwich Gallery has its own atmosphere, it is certainly 17th century, but this collection is not confined to the 17th century. There are memorable 18 th-century paintings - a Watteau of delicate vivacity, a good Hogarth portrait, Reynolds's bow to theatrical presence, ''Mrs. Siddons as the Tragic Muse.'' Gainsborough early and late is impressively represented, the most wonderful perhaps being the consummate ''Linley Sisters.''
Seeing the city's smaller museums is a good way to extend one's awareness of the capital and escape from the overvisited center. To go from Dulwich to Hampstead Heath is to travel from southeast London to northwest. And near Hampstead Heath is a fine Adam House, in lovely parkland, with a notable collection of paintings in it. This is Kenwood. The paintings were bequeathed by the first earl of Iveagh in 1927. But he collected them as though he were a typical 18th-century connoisseur and gentleman.
Here we find Reynolds and Gainsborough, Romney and Raeburn in fine fettle, particularly in a number of charming pictures of children.
It is, however, a couple of Dutch 17th-century masterpieces which are alone sufficient to draw one to Kenwood: Vermeer's ''The Guitar Player'' and a self-portrait of Rembrandt.
Anthony Blunt wrote admiringly of the former's ''subtlety of light, delicacy of color, and boldness of design.''
And the stunningly powerful self-portrait of Rembrandt at the age of (about) 57 is one of the most moving paintings anywhere. His gaze, questioning yet knowing, kind but dissatisfied, is a kind of universal study of all artists, their self-appraisal, their humanity.
These four small museums in London are of such high standing that to not visit at least one of them on a trip to this city would be a great pity. But a multiplicity of other fine museums offers the traveler an even wider choice.
Wellington Museum, at Apsley House on Hyde Park Corner, was the home of the first duke of Wellington (victor at Waterloo). Velasquez's ''Waterseller of Seville'' and ''Two Young Men Eating at a Humble Table'' are the outstanding works here.
The William Morris Gallery takes some finding, but repays the effort. It is housed in a striking Georgian House where this many-sided man lived as a boy. Inside is a pleasantly displayed array of objects, all connected in some way with Morris either as a designer, pre-Raphaelite painter, writer, poet, employer , socialist campaigner, influence, or inspiration.
Anyone intrigued by even more colorful excesses of High Victorian taste, should (map in hand: it's not easy to locate!) hunt for Leighton House. Take a dazzled look at its extraordinary ''Arab Hall.''
Lord Leighton, a noted painter, lived in this house for 30 years. Decorated with an eye-tickling multitude of Persian and Saracenic tiles, the Arab Hall is pure and exotic fantasy, its only hint of coolness and simplicity being a central fountain.
Sir John Soane is by no means only remembered in London's museum world for his elegant Dulwich gallery building. His own house is now known as Sir John Soane's Museum. He acquired it in 1811 and redesigned it as a strange and highly idiosyncratic home for his collection of treasures and for himself. His treasures remain - antiquities galore, objets d'art, and paintings.
Apart from art museums, London can boast remarkable ''subject'' museums. At the Bethnal Green Museum of Childhood (currently undergoing reorganization) games, toys, dolls, teddy bears and doll houses - not to mention children's clothes - abound. The rather chaotic refurbishment tempers but doesn't totally ruin the delights of the place.
If there is one ''subject'' museum that might justifiably be thought a priority for any visitor to London, it is likely to be the Museum of London. This is a most impressive - and time-absorbing - exhibition, theatrically set out, presenting the long development of the city from prehistory to the 20th century.
(Brief information of all the notable museums can be found in the ''London Art and Artists Guide'' (Art Guide Publications, $: 3.95 or $5.95) or in the much more detailed Blue Guide ''Museums and Galleries of London'' ($: 7.95 or $ 11.95).)