Gillingham caretaker boss Peter Taylor was left to rue some poor defending after Port Vale came from behind to beat the Gills.
An early goal from Danny Kedwell, who headed in a cross from Amine Linganzi, had given Taylor's team the lead at the break.
But Chris Robertson headed a 47th-minute equaliser before substitute Tom Pope nodded in a winner after 63 minutes.
Robertson made no mistake from close range after Carl Dickinson helped on a corner from Chris Lines and Dickinson also provided the assist for Pope's sixth goal of the campaign.
Taylor said: "We were excellent in the first-half and that really pleased me. We played some good football and could have scored more than one goal.
"But the massive negative was the start of the second-half when we were put under pressure and conceded two very poor goals.
"For the first goal we didn't win either the first or second headers and our defensive shape should have been better for the second goal.
"We can keep working on things to try to improve, but a manager can't do anything if players lose concentration and switch off during the game. That creates a dangerous situation.
"I was also pleased with how we finished the game and we came close to equalising. We should have done better with a couple of free headers in the Vale penalty area.
"I've been in charge for three matches now and I've recognised strong areas and weak areas of the team.
"I'll have a word with the chairman and tell him where I think we need to improve and we will see what happens."