An autopsy was scheduled for Thursday, and no cause of death had been released.
In a statement issued by Robert Kennedy Jr.'s chief of staff, the family described Mary Kennedy as "a genius at friendship."
"Mary inspired our family with her kindness, her love, her gentle soul and generous spirit," the family said.
The former Mary Richardson, a longtime connection of the Kennedy clan, married Robert Kennedy Jr., a prominent environmental lawyer and the son of Sen. Robert F. Kennedy and nephew of President John F. Kennedy, in 1994. The couple had four children, the youngest born in July 2001. Robert Kennedy Jr. also has two children from a previous marriage.
Mary Kennedy was an architect and designer and had overseen the renovation of the couple's home into an environmentally advanced showpiece. Her family cited her devotion to her children in remembering her.
"We deeply regret the death of our beloved sister Mary, whose radiant and creative spirit will be sorely missed by those who loved her," the family said in a statement issued by Lawrence. "Our heart goes out to her children who she loved without reservation."
Neighbor Leslie Lampert, who owns the Cafe of Love restaurant a short drive from the Kennedy home, said Mary Kennedy was "at all times just a lovely individual."
"She was community oriented," Lampert said. "She was always kind in our presence."
Another neighbor, Kim Fraioli, a trauma therapist who lives a few houses down from the Kennedys, said the family was private.
"We left them alone," Fraioli. "We didn't have any interaction. I think it's a tragedy. It's very sad for their family and the surviving children. My heart goes out to the family."
At the home on Wednesday, a red brick mansion with a columned porch entrance set in a heavily wooded acreage, police kept media away.
Mary Richardson had known the Kennedys for years, through her friendship with Robert Kennedy Jr.'s sister, Kerry Kennedy, whom she met at boarding school. She had been Kerry Kennedy's maid of honor at her wedding in 1990.
She had had trouble with drugs and alcohol and had two high-profile arrests around the time her husband filed for divorce in 2010.
Kennedy was first arrested May 15 of that year on a charge of driving while intoxicated after a police officer reported seeing her drive her car over a curb near the family's Bedford home. Her only passenger was a dog, and police said she had a blood-alcohol level of 0.11 percent; the legal limit is 0.08 percent. Her license was suspended.
At the time of her sentencing, famous family and friends spoke out in support of her.
Her mother-in-law, Ethel Kennedy, wrote in a letter that she "is a caring, nourishing mother who has nursed her four children through lifelong bouts of debilitating allergies," according to an account in Westchester's The Journal News at the time.
Kerry Kennedy, in her letter, said, "When I look at my three daughters, my wish for them is that they are as blessed as I have been to have a companion, a confidante, a friend, like Mary Richardson."
Mary Kennedy was charged later that year with driving under the influence of drugs, but that charge was dismissed in July 2011 when a judge said the evidence showed she didn't know the medications she had taken would impair her ability to drive.
The unexpected death of another person connected to the storied Kennedy clan brought to mind the other sorrows the famous family has suffered.
Shopping in Bedford, Diane Glokler said, "I've always just thought that family is very tragic. They keep having tragic things happening to them. It's heart-wrenching."