The Conference club often lived dangerously as Notts County created a string of fine chances but were repeatedly denied by a combination of poor finishing, heroic defending and great goalkeeping.
It was the hosts who took the lead two minutes before the interval when Steve Perkins' low shot squirmed under the hand of Steve Mildenhall.
The Magpies drew level through Nick Fenton's header on 69 minutes and two minutes later, Gravesend were reduced to ten men when Paul Gibbs was sent off for a rash challenge on Simon Baldry.
With the tie seemingly heading for a replay, Magpies striker Clive Platt fired a low shot beyond substitute keeper Alex O'Reilly while Mildenhall redeemed himself with a late save from Lee Shearer.
Conference outfit Accrington Stanley almost pulled off a shock when they led against Second Division Bournemouth for a lengthy period before forcing a second round replay.
Paul Mullin put the visitors in front on nine minutes but The Cherries salvaged a replay on 56 minutes courtesy of Marcus Browning.
Accrington's Conference rivals Aldershot held out for 83 minutes at Colchester United before Rowan Vine broke the visitors' resistance.
Northampton Town ended Weston-Super-Mare's run in the competition with a comprehensive 4-1 triumph at Sixfields Stadium.
The turning point of the contest came on 36 minutes when the non-league side's Mark McKeever was sent off for deliberate handball. Martin Smith dispatched the resulting penalty.
Josh Low and Marc Richards then extended The Cobblers' advantage before Billy Clark scored a consolation, only for Richards to net again in the final minute.
Non-league Hornchurch's FA Cup giantkilling run ended with a 1-0 home defeat by Tranmere Rovers.
The decisive goal arrived on 26 minutes when Gary Jones (pictured) converted Micky Mellon's cross from inside the penalty area.
In a thrilling seven-goal affair at Boundary Park, Blackpool romped to a 5-2 victory over Second Division foes Oldham Athletic.
A hat-trick from Scott Taylor proved the highlight and he had The Seasiders 2-0 up after 20 minutes before Keith Southern added to the margin before the break.
Taylor completed his treble on 57 minutes before John Eyre pulled one back for The Latics. Leam Richardson restored the four-goal cushion before Jermaine Johnson replied in the closing minute.
Cheltenham brushed past Leyton Orient with little difficulty following the 18th minute sending off of goalkeeper Glenn Morris.
The O's shot-stopper was dismissed for a professional foul on Damian Spencer, with Grant McCann tucking home the penalty.
McCann scored again on 34 minutes with Bob Taylor making it three on the hour mark. Matthew Lockwood scored the visitors' consolation on 69 minutes.
Macclesfield forged ahead against Cambridge courtesy of Matthew Tipton's 22nd minute penalty after he was fouled by visiting player/manager John Taylor.
But Cambridge have another bite at the cherry after John Turner scored within 60 seconds of the hosts taking the lead.
Bristol City and Barnsley must replay at Oakwell for the right to be in the third round after a 0-0 draw at Ashton Gate.
Click on the links below for an individual match summary:
Bournemouth 1-1 Accrington Stanley
Bristol City 0-0 Barnsley
Cheltenham 3-1 Leyton Orient
Colchester 1-0 Aldershot
Gravesend 1-2 Notts County
Hornchruch 0-1 Tranmere
Macclesfield 1-1 Cambridge
Northampton 4-1 Weston-S-Mare
Oldham 2-5 Blackpool
