West Indies claimed their first ever Test win in South Africa with a 128-run success on the fourth day at St George's Park, Port Elizabeth.
Fidel Edwards and Jerome Taylor each took three wickets as the Proteas were bowled out for 260 in pursuit of an unlikely victory target of 389.
It was also West Indies' first Test win of any kind since beating Pakistan at Bridgetown in May 2005 - 21 Tests ago - and their first away from home since winning in Zimbabwe more than four years ago.
South Africa slumped to 45-4 inside 19 overs of their second innings and, despite battling half-centuries from Jacques Kallis and AB de Villiers, there was no way back.
Daren Powell set the tone in the third over by trapping Herschelle Gibbs lbw offering no stroke for a 13-ball duck to go with his four-ball effort in the first innings.
That was followed up by Edwards, who dismissed Hashim Amla (8) and Graeme Smith (11) in successive overs to leave the Proteas struggling at 20-3.
Total capitulation looked on the cards at 45-4 after Ashwell Prince (10) played away from his body and edged a Jerome Taylor outswinger low to Gayle at first slip.
But Kallis finally found a willing ally in de Villiers and the fifth-wicket pair saw South Africa to tea without further loss.
But their 112-run stand was ended in the fourth over of the final session when umpire Russell Tiffin dispatched Kallis for 85, caught behind attempting to pull a short delivery from Edwards.
It was an unfortunate decision as the ball touched Kallis' shoulder rather than the edge of the bat on the way through to wicketkeeper Denesh Ramdin.
After Mark Boucher (13) and Paul Harris (0) had departed cheaply, de Villiers lost patience and chipped to mid-on for 60 to become Taylor's third victim.
Some lusty blows from ninth-wicket pair Andre Nel and Dale Steyn delayed the inevitable during an entertaining partnership of 67 in just 60 balls.
But medium pacer Darren Sammy (1-29) ended the fun by having Nel (34) caught behind, and the victory was sealed when last man Makhaya Ntini (1) was caught attempting to hit off-spinner Marlon Samuels (1-14) out of the ground.
Harris spins a web
In the morning, left-arm spinner Harris took the final two wickets to end West Indies' second innings at 175.
Harris bowled Powell (6) via the inside edge in the fourth over of the day after West Indies resumed on 146-8 in their second innings.
And he added the scalp of Taylor (22), caught at long-on in his next over, to finish with figures of 4-35.
The second match of the three-Test series starts at Newlands, Cape Town on January 2, live on Sky Sports.