Edin Dzeko was on target for Bosnia on Saturday, as they sprang a surprise, while Albania also won.
Bosnia gave their hopes of reaching the 2012 European Championships a welcome shot in the arm as they recovered from falling a goal down to see off Romania 2-1.
France continue to lead the way at the top of Group D and the Bosnians knew they had to pick up three points over the weekend if they were to keep their qualification dreams alive.
It did not look good for them on 29 minutes when Ciprian Marica put the visitors in front.
Against the run of play, he found space on the left-hand side of the area to fire home a Razvan Rat delivery.
The home fans started to get restless as the game wore on, but eventually saw their side equalise on 63 minutes as Vedad Ibisevic headed home from Zvjezdan Misimovic's left-wing cross.
And Manchester City striker Dzeko completed the comeback in the closing stages with another header from 12 yards out.
Belarus' qualification hopes were dealt a serious blow as they suffered a narrow 1-0 defeat in Tirana against Albania.
The two nations are now locked together on eight points, four adrift of table-toppers France.
The win was just reward for the hosts, who were the better side throughout and dominated from the opening whistle.
All of their pressure finally paid off on 62 minutes as Hamdi Salihi coolly slotted home from close range to settle the tie.
He should have had a second just a few minutes later but this time his effort, from a similar distance to his first, came back off a post and was cleared to safety.