One was a five-day-old baby, Hope Erin Evans from Aberdare, who died at Singleton Hospital after contracting ESBL E. coli.
Another very premature baby also died.
There were three more non-fatal cases. Abertawe Bro Morgannwg (ABM) University Health Board said they appeared to be isolated incidents.
Investigations will include how cross-infection occurred in the maternity and neonatal unit.
One case is that of a mother who has tested positive for ESBL E.coli but who has not shown any symptoms nor required treatment.
The health board said it suspected she had contracted the bug in the hospital, but that they were waiting for test results to confirm this.
It said that while both babies died in Singleton, only one had contracted the infection inside the hospital.
Meanwhile, an inquest has been adjourned into the death of Hope Erin Evans, who died on 4 November.'Difficult to eradicate'
ESBL (extended-spectrum beta-lactamase) producing E. coli are strains resistant to antibiotics, making them difficult to treat.
In many cases, only two types of antibiotic taken by mouth and a "very limited" group of intravenous antibiotics are effective, according to information from the Health Protection Agency (HPA).
E. coli can lead to serious infections such as meningitis and blood poisoning, which can be life- threatening.
The hospital has said it is only delivering full term babies, that is of 36 weeks or more gestation, at the moment.
As a precaution, the unit has undergone a deep clean.
Microbiologist Prof Hugh Pennington told BBC Radio Wales ESBL E. coli caused problems for young babies because their immune systems were not well-developed.Continue reading the main story ESBL E. coli is not the same as the E.coli O157 which causes food poisoningESBL stands for Extended Spectrum Beta LactamaseESBL E. coli is most often found in the gastrointestinal tract but may cause urinary tract infectionsESBL E. coli is resistant to commonly-used antibiotics such as penicillin, but can be treatedIn most people ESBL E. coli does not cause harm but in vulnerable individuals it can cause serious infectionsSource: ABM health boardProf Pennington believes the hospital is taking the right steps to try to deal with the outbreak.
"These bugs . . . are really quite good at getting about and once they get into something like a neonatal unit, history tells us with other related bugs they can be really quite difficult to eradicate," he said.
"I'm sure that the people at Singleton are doing their utmost to make sure that there aren't any problems.
"They have restricted admission to the very young babies because they are the ones really at risk from this particular nasty bug. It's a very reasonable approach to take."
"These appear to be isolated incidents which have been contained, and there is no evidence of the infection spreading further," Dr Ferguson added.
"Checks have been taken of patients, equipment and areas in the maternity/neonatal unit and no evidence of ESBL E. coli has been found.
"The unit has an excellent record for hand-hygiene and general infection control adherence. Reported infection levels in the unit have been below the national average in recent years."