A last-gasp Winston Reid header led to huge celebrations among the All Whites camp, with Herbert delighted with the display in Rustenberg.
He said: "I am very, very proud. Can it get any better? I think the nation will be in reasonably good spirits.
"It would have to be (our finest hour). We've never won a point in a World Cup before. We've thrown some really good punches and I thought it was an extremely well deserved result.
"We'll keep on dreaming because we are here. We have a chance like anyone else, but we still have two heavyweights to go and it will be tough.
"We are competitive and the brand of football we now play is more conducive to high-level results and performances."