Someone asked on a mailing list for possible ways to fix a bad voice the produced using festvox. I realized that my answer took my quite some time to figure out without any help, So I thought I'd post my response here.
Basically, if your voice is bad, chances are your labeling of some of the prompts is bad. (even if it isn't, it doesn't hurt to make sure they are good). You want to fix the bad labellings
To do so, copy the contents of your wav folder and the contents of your lab folder into the same directory (or setup links to make it seem that way). Once you've done that, open up the wav files with waveurfer, and choose the "transcription" view for all of them. Now you can go through one by one and check if the labellings are right. Options are: re-record the ones with bad labellings (remember to run bin/make_lab again before checking the labels again, I made this mistake once, and kept re-recording and thinking that the autolabeller sucked. Also, to save time, you can run bin/make_labs prompt-wav/test001.wav to just relabel test001.wav, instead of doing it to all the recordings, which can be time-consuming.), or hand-correcting the labels. You can literally just drag the labels from within wavesurfer (remember to copy your changes back to the lab/ directory).
Once you've got all the labels as perfect as you care to have them, just repeat all the steps after "bin/make_labs prompt-wav/*.wav" from whatever tutorial you are following and you should get the voice built with proper labeling (Come on, I know that if you knew how to do anything with festvox without a tutorial in front of you, there's no way you would need to be reading this post).