Danny Wilson’s men – who have now played all four Irish provinces on the road in consecutive games - had led going into the closing stages of the first half in Belfast before a try from Paul Marshall sparked a try blitz from the hosts.
With Blues down to 14 men following Josh Turnbull’s yellow, tries from Nick Williams and Stuart McCloskey earned the home side the bonus point before the visitors showed heart and desire to dig deep and fight-back.
Captain Josh Navidi showed all his power to bulldoze over in the corner and Rhys Patchell kicked the visitors back within bonus point range with seven heart-stopping minutes remaining.
A late rally ensued and real anxiety swept the stadium as Ulster were forced to defend for spells and hold out in the closing minutes to deny Blues a share of the spoils.
For boss Wilson there were plenty of reasons to be positive. Performances have been produced on the road and now results surely will come.
Cardiff Blues have run all four Irish provinces close – losing by seven points twice and eight points twice – and emerged for an uncompromising run of away trips to the Emerald Isle with real credit.
Now the focus turns to Italy and Zebre next weekend as Blues look to return to winning ways before getting back on home soil after five away games with the visit of champions Glasgow Warriors.
The visitors started the game with real passion and showed plenty of intent in the opening minutes of the fixture as Patchell turned down a kick at the posts and opted for the corner, after Craig Mitchell’s powerful burst.
However, when the visiting forwards earned another penalty just a few minutes later, this time in the shadow of the posts, it was Patchell who drilled over the resulting kick for the lead.
Ulster looked to put Blues under real pressure at the breakdown with their first attack of the game and when they won a lineout, after an unfortunate deflection of a visiting boot, the drive followed. Blues, though, held firm in a cauldron of noise and a knock saw the chance gone.
Ireland ace Andrew Trimble looked an early threat with ball in hand as he menacingly hunted space while Dan Fish looked assured for the visitors under the high ball while his range of kicks forced Ulster back.
Cardiff Blues looked in complete control until two powerful runs from home Number Eight Williams gave Ulster sudden go forward. With tacklers tied up in midfield, the home side spun the ball wide at pace and Trimble exploited the room on the touchline to race in.
The deadly accurate boot of Patchell put Blues back ahead three minutes later while spinning scrum half Tavis Knoyle took a sliced clearance brilliantly by the touchline and rolled out of traffic to set Blues off again with new boy Cam Dolan and Josh Turnbull thundering in contact.
A huge shift from the Cardiff forwards saw them relentlessly take the ball on around the fringes before another penalty, this time for hands in the ruck, saw Patchell again dissect the sticks for a 9-5 lead.
Just seconds later, though, a bizarre game changing penalty went against Fish, when he appeared to simply contest a high ball with Trimble, and it gave Ulster an attacking platform they never looked like wasting.
The home side turned the screw as Blues defended valiantly on their own line before Turnbull was yellow carded. The resulting scrum saw Marshall pick up at the base and dive over for a slender half-time lead.
Cardiff Blues started the second half a man down and knowing they would face an onslaught from the home side. They were not disappointed as a scrum penalty gained them field position and then a lineout was shunted over with Williams touching down for a 19-9 lead.
The momentum was now all Ulster and a third try came on 50 minutes as slick handling opened up space for big centre McCloskey to score and extend the lead further, with the bonus point now in the bag.
Blues needed a foothold in the game and it came courtesy of wing Richard Smith, who produced a powerful hand-off to race out from under his own posts, before back row Dolan tore into open pace.
A penalty for a late tackle on Fish presented Blues with a lineout attack before two quick penalties against home prop Wiehahn Herbst, the second for stopping Knoyle as he darted to the line with a quick tap and go, saw him yellow carded.
Cardiff Blues now had the extra man and made it count straight away with a sublime finish from flanker Navidi who bounced off two tacklers and stretched out an arm to crash in the right corner to cut the lead to 24-14.
The home side suddenly looked anxious and a high up and under from Fish forced more panic.
Adam Thomas then reacted first to scoop up a loose ball to stop a rare Ulster foray into the visiting 22 and as the game entered the final ten minutes with everything to play for.
Blues secured the penalty they wanted, after Ulster infringed as they contested the breakdown and Patchell nudged the visitors back within seven.
A yellow card for a dangerous tackle by wing Summerhill ensured Blues finished the game a man down.
But there was still a grandstand finish as the visitors threw everything at Ulster in the closing stages before a knock-on and the final whistle was greeted with both delight and relief from home fans.
Timeline:
04m Patchell p 0-3
19m Trimble t 5-3
22m Patchell p 5-6
36m Patchell p 5-9
40m Marshall t Humphreys c 12-9
46m Williams t Humphreys c 19-9
50m McCloskey t 24-9
58m Navidi t 24-14
73m Patchell p 24-17
Cardiff Blues: 15 Dan Fish, 14 Aled Summerhill, 13 Tom Isaacs, 12 Adam Thomas, 11 Richard Smith, 10 Rhys Patchell, 9 Tavis Knoyle (Jones 61); 1 Sam Hobbs (T Davies 73), 2 Matthew Rees (E Lewis 65), 3 Craig Mitchell (Filise 53), 4 Lou Reed, 5 James Down, 6 Cam Dolan (Cook 56), 7 Josh Navidi, 8 Josh Turnbull
Replacements: 16 Ethan Lewis, 17 Thomas Davies, 18 Taufa’ao Filise, 19 Chris Dicomidis, 20 Macauley Cook, 21 Lewis Jones, 22 Jarrod Evans, 23 Garyn Smith