It had been a frustrating evening with a succession of missed chances - until goals from Daylon Claasen and Judas Moseamedi in the final five minutes earned the Team of Choice a point and extended their unbeaten run to seven games in all competitions.
From the start Maritzburg were on the front foot when Pogiso Sanoka's fifth minute cross located Gabriel Nyoni, but he was unable to keep his header on target.
Micah Lea'alafa then set up a chance for Thabiso Kutumela four minutes later, but the striker's shot bounced back off the upright as United kept the heat on Celtic.
More opportunities followed around the half hour mark, but 0-0 was how it stayed as Nyoni's header from a Dan Morgan throw was saved before Jeremy Brockie went close with a flick from a Keegan Buchanan corner.
United again threatened early in the second half as Siyanda Xulu sent a header wide from another Sanoka delivery.
Having failed to turn chances into goals, the hosts were to suffer a big sucker-punch in the 53rd when Celtic took the lead through Sera Motebang.
Twelve minutes later, Xulu was forced to leave the field after having suffered a blow to the head in a challenge with the opposition keeper while also having hurt his ankle. With three subs already made, United were left to battle it out with 10 men.
The chances meanwhile continued to fall to the Team of Choice, yet it wouldn't fall for them in front of goal as Moseamedi failed to convert Bandile Shandu's cross, before Shandu blazed over the bar after Claasen and Moseamedi had combined.
Instead, the large home crowd was to be stunned as Maritzburg found themselves 2-0 down when Siphelele Luthuli tapped in at the end of a swift counter attack.
Still though the hosts continued to go at the opposition in search of a goal and after Claasen skipped into the box and was fouled, a penalty was awarded in the 86th minute.
He stepped up to take the spot-kick himself and converted low into the bottom right corner to give the home team hope.
And hope soon turned to joy on the field, and up in the stands, as Moseamedi jinxed into the box in the 90th minute and fired into the top corner of the net to complete a spirited late fight-back.