Officials said firefighters saw and heard people in the building that they were unable to save.
Many of those unaccounted for were confined to wheelchairs and walkers and only five residents in the center were fully autonomous, said Ginette Caron, acting mayor of the small town of L'Isle-Verte, about 140 miles (225 kilometers) northeast of Quebec City.
She said some had Alzheimer's disease.
The massive fire in the three-story building broke out around 12:30 a.m. and raged through the night. Firefighters arrived within eight minutes of getting the alarm and several fire departments in the region were called in to help. But firefighters were unable to carry out a complete evacuation because of the intensity of the fire. About 20 residents were transported to safety.
"It was a total fire," said L'Iles-Verte fire chief Yvan Charron.
The fire broke out in -4 Fahrenheit (-20 Celsius) temperatures, causing equipment to freeze, Charron said.
As morning dawned and the fire was brought under control, the burned section of the facility resembled a macabre ice palace, with sheets of ice and thick icicles covering the structure.
The search for the missing was still hampered Thursday evening by the cold and thick ice and the fact that the building has collapsed, said Quebec Provincial police Lt. Guy Lapointe. Officials said they would work through the night.
"We can keep some hope for those unaccounted for, but there's very little doubt that the loss of life is considerable," Prime Minister Stephen Harper said.
Outside the building, Jacques Berube, 70, was getting ready to hear the worst about his 99-year-old mother, who is blind but still mobile.
"I went near the building; the corner where her room was is burned," he said. "I'll just have to wait and see. I don't like it. But I don't have any choice. It's just reality."
Pierre Filion, who had a cousin and an aunt living in the residence, said the tragedy had shaken the tightly knit community of 1,500 people.
"Everybody knows everybody," he said. "It's a small community. It's going to take a long time to start living normally."
Filion said his two missing relatives who lived there were both in their 70s.
"I went there 15 minutes ago and the only thing we saw was smoke and smoke," he said.
Parts of the Residence du Havre, which opened in 1997, had sprinklers, while others didn't.
Charron chief said sprinklers did go off, triggering the fire alarm and allowing firefighters to gain access to about one-third of the building.
Quebec Provincial police Sgt. Ann Mathieu urged people who have any information on people considered missing to call police.
Mario Michaud, who lives across the street from the building, said he witnessed the unfolding drama shortly after midnight.
"The fire had started on the second floor. I woke up my girlfriend and called 911. I saw the firefighters and they got to work. A woman on the second floor was shouting and she went out on to the balcony. Her son went to get a ladder but he couldn't get to her. She burned to death," Michaud told local newspaper Info Dimanche.
At least three people were injured in the blaze. The extent of their injuries was unclear.
The building was home to more than 50 people and also housed a social agency, a pharmacy and a hair salon.
A Quebec Health Department document updated last July said the building, with one elevator, was constructed entirely of wood.
Most residents were older than 75 and 37 of them were older than 85. The building included both single rooms and apartment-style dwellings. The document also indicates the building was only partially fitted with a sprinkler system but did have a fire alarm. There were smoke detectors in every room and in the building itself.
The fire comes just six months after 47 people were killed in Lac-Megantic, Quebec, when a train with 72 oil tankers derailed and exploded in the small community.
In 1969, a nursing home fire in the community of Notre-Dame-du-Lac, Quebec, claimed 54 lives.