Northampton boss Aidy Boothroyd felt the loss of Danny East played a big role in his side's defeat at Bristol Rovers.
Second-half goals from Tom Eaves, Garry Kenneth and Ollie Norburn gave Rovers a first home win of the season.
Adebayo Akinfenwa replied for Town, who had centre-back Charles sent off for a professional foul on Derek Riordan after 24 minutes.
Full-back East had to be substituted in the 51st minute, and then just two minutes later Eaves put Rovers ahead.
Boothroyd said: "With 10 men we were still causing them problems and I thought the pressure might grow on them the longer we kept a clean sheet. But after the injury I was forced to change things and we never looked as comfortable. Danny's shoulder popped and he will be out for a while.
"We have to look at ourselves over the red card because we were left one-on-one at the back and were exposed. It might have been a yellow card on another day, but the referee has made the decision and there are no complaints."