NELSON MANDELA now has a firm popular mandate to move toward substantive negotiations with Pretoria on a new political system for South Africa. His election to the presidency of the African National Congress last week, with longtime colleague Walter Sisulu as deputy president, puts the ANC's two most prominent moderates at the helm.The voyage ahead, however, is likely to be rough. While it tightened the organizational structure and helped set a course, last week's historic ANC conference - the first such gathering in South Africa in 30 years - also illustrated the internal frictions Mr. Mandela has to deal with. Those elected to the group's national executive committee include prominent hard-liners like Chris Hani, head of the ANC's military wing. Many militants within the ANC have little taste for negotiating with the National Party and President Frederik de Klerk. Some have never agreed with last year's relinquishment by the ANC leadership of its "armed struggle." But the frictions between moderates and radicals, while present, have for the time being been subordinated. The conference's biggest accomplishment may have been an infusion of new organizational talent in key positions. Cyril Ramaphosa, as the ANC's new secretary-general, brings sharp administrative and negotiating skills to his job. As head of South Africa's mine workers, he also represents the younger, home-grown leadership that developed while Mandela and others were in prison or exile. The ANC's agenda emphasizes mass protests, government action to stop the violence between ANC adherents and Zulu partisans, and continued international sanctions against South Africa. In all these areas a fine line exists between practical politics and hollow, even counterproductive, rhetoric. Mass strikes and protests may have their uses, but if they get out of hand both economic well-being and the climate for negotiation could suffer. The government must prove it intends to quell, rather than foster, black-on-black violence. But the ANC, too, has a role to play in bringing peace to the townships. The congress wisely chose a flexible policy on sanctions, since Mr. De Klerk's reforms have already started a gradual phaseout. The ANC has shown itself capable of democratically electing a vigorous team to negotiate with the government. Difficult compromises lie ahead for both sides.