Thandolwenkosi Ngwenya's 59th minute penalty was the difference at the Sugar Ray Xulu Stadium, as he buried his spotkick past Stellies stopper Sage Stephens.
Hunt's side were forced to play their home match in Durban due to unavailability of a suitable venue in the Cape Town.
Reflecting on the match, Hunt insisted that the result was not a fair reflection of the match as his team lost to a breakaway goal after controlling much of the tie.
"Yeah, I'm disappointed. I don't think we deserved to lose the game, not by one or the other, no chance," Hunt said on SuperSport.
"But they got the break, they got the penalty, so that's it. Not much I can say about that, but I don't think we deserved to lose the game."
"Yeah, it's not quite easy to accept a defeat like this one when obviously you have all the circumstances [not having home venue] like that going on," Hunt added.
"It leaves an even bitter taste in your mouth because, as I said, if you talk about footballers, they didn't deserve to lose that game.
"And it's very difficult because you take a defeat, but we've got to take it and move on and try and go from here."
Stellenbosch next face Polokwane City at the Peter Mokaba Stadium on Tuesday, 11 August.
