Ashley Barnes scored with a penalty as Albion made it two wins out of two at their new home by seeing off their old landlords 1-0.
The Seagulls played at Gillingham for two years until 1999 and had to wait another 12 years before their new Amex Stadium finally opened for business.
They grabbed a dramatic win over Doncaster in their first game at the weekend and kept up the tension by making hard work of beating the League Two Gills.
Poyet was a League Cup runner-up with Tottenham in 2002 and fancies a run in the competition this season, but Brighton needed a second-half penalty from striker Barnes to squeeze into round two.
"Cup games are hard," Poyet told Sky Sports News.
"You have to give credit to Gillingham for their display, they tried their best and it was one of those games where maybe one decision is going to go with you or against you.
"I'm pleased that it went our way and let's see who's coming now."
Andy Frampton was penalised for bringing down Buckley for the decisive spot-kick but Gillingham felt hard done by.
"Obviously we are disappointed, I'd like to see the penalty again," manager Andy Hessenthaler told Sky Sports News.
"Talking to our captain Andy Frampton, he said he hasn't touched him, his momentum has taken him down and the ref has made the call, but I think there was a foul leading up to it.
"So we are disappointed to lose it on a penalty."