Boston — Overcrowded prisons are fast becoming one of the nation's biggest and most vexing problems. New or expanded corrections facilities are under construction or being discussed in over half the states.
But there is little agreement among police, lawmakers, and corrections officials that building more prisons to lock up wrongdoers is the most cost-effective or best approach to slowing the crime rate. Alternatives to this approach are being debated at both the state and federal levels.
Several proposals in Congress aim to give more emphasis on incarcerating ''career criminals'' - those convicted three or more times for the same or similar types of serious crime - while giving lesser sentences for nonrepeaters.
Within the Reagan administration, the Justice Department is looking into alternatives to imprisonment for dealing with those guilty of nonviolent criminal acts.
State legislatures from Massachusetts to California similarly are trying to determine how best to use limited prison space. They're considering measures that would provide longer jail terms and mandatory sentences for those deemed the greatest threat to society. About two-thirds of those currently in prison have served time for previous offenses, according to federal Bureau of Justice Statistics. Considerable attention in recent months has been focused on a Rand Corporation study released last fall which recommends a system of ''selective incapacitation'' - sentencing based on a person's previous criminal record. Such a system could reduce the crime rate without overcrowding prisons, it suggests.
The six-year study, involving some 2,100 prison inmates in California, Michigan, and Texas, identified seven background ''variables'' that could be used to separate robbery and burglary convicts into low-, medium-, and high-risk categories. Those thought most likely to commit such crimes again would be given the longest terms of confinement.
Although no state thus far appears to have implemented the Rand recommendations, Peter W. Greenwood, who headed the research project, noted that measures embracing some of the conclusions are pending in at least a few state legislatures, including California's.
Critics of the Rand recommendations contend that fixing sentences on the basis of possible future criminal activity would be unfair to those who might be reformed, scuttling the concept of rehabilitation.
''We knew this is a hot issue,'' Dr. Greenwood says of the proposal to classify repeat offenders. The study suggests that practice could reduce the number of jailed robbers by 5 percent - on the premise that they wouldn't be likely to repeat the crime - and reduce robberies by up to 15 percent. He describes the proposal as ''an attempt to be more logical about how you put people in prison.''
Anthony Travesano, executive director of the American Correctional Association, suggests that only repeat offenders and those convicted of violent crimes need occupy valuable and increasingly expensive prison space. Others could be dealt with through community-service programs and other alternatives to imprisonment, he says.
US Attorney General William French Smith, in a speech last month, asserted: ''We must recognize that we cannot continue to rely exclusively on incarceration and dismiss other forms of punishment.'' He added that while in many cases the $ 10,000 a year it costs to keep an inmate is worth it, ''in other cases it is too high a price.''
Nevertheless, many states are passing more mandatory sentencing laws - a move that's seen as a way to get tougher on criminals, but that could pack prisons even more. While it is uncertain how many mandatory sentencing laws may be on the books, at least 37 states, most of them over the past few years, have prescribed such punishment, usually involving longer terms and without possibility of parole.
In Massachusetts, one-year incarceration is required for anyone convicted of possession of a handgun without a permit. Mandatory prison sentences also are provided in the Bay State for those convicted of selling illegal drugs or committing assaults on the elderly. And nearly a dozen proposals to prescribe similar minimum punishment for other crimes, such as breaking and entering, currently are pending before Massachusetts lawmakers.
In Congress, US Sen. Arlen Specter (R) of Pennsylvania is sponsoring a measure to make third-time robbers and burglars subject to federal, rather than state, prosecution and subject to a 15-year mandatory sentence in federal prison.
Similar legislation cleared Congress last year, but was vetoed by President Reagan in January because of opposition to another section of the omnibus anticrime measure.
Among other tough measures to combat the career criminal is a bill filed by Rep. Donald J. Albosta (D) of Michigan. The bill calls for 20-to 30-year imprisonment without parole for a third conviction for selling drugs.