Update: I think the problem is with my headset. The built-in speaker gives much better quality, but its not very loud and has intermittent fuzz sounds. I suppose this shows that the hiss is not due to the sound card (I was considering get an external sound card). I guess the problem is the microphone.

>I think the problem is with my headset.

a USB microphone (or headset) might help you with this...

Au contraire. The USB headset appears to be the culprit.
http://avp.stackexchange.com/questions/7142/least-expensive-microphone-for-hiss-free-pc-recordng

>I'm having difficulty recording hiss-free audio from my PC using a headset combo.

Thanks for the link!

My understanding is that it is best to have all the audio processing done externally to avoid any interference from the motherboard/power supply.

USB mic usually works best from a cost-benefit perspective.

from the top answer on stackoverflow, in the discussion section, it says:

if it is a USB headset, then my answer is less relevant in this case - you aren't recording in through the sound card; instead, your headset has an amplifier and an Analogue to Digital converter. So any analogue hiss will be in the circuit prior to the A/D and any digital interference will be after that. In your case, you should be able to find a better microphone.

maybe get a better USB mic/headset, but I wouldn't spend $200 to play around with speech rec.

from your original post:

>I'm afraid the hiss will adversely affect my the quality of the acoustic model that I train with it.

depends what you want to do with your acoustic model - if all you are doing is playing with speech rec on your particular PC, then training an AM with audio recorded from the same mic/PC combo should still give you good results. 

If you want to use the AM on other PCs with a different mic, then getting a clean signal is the best approach.  See this post for more info: More on Collecting Speech Audio for Free GPL Speech Corpus

I'm glad you found the thread to be of some help. I started it.

Earlier today I asked my boss who knows a great deal about USBs and device drivers (they are part of our application stack) why audio recorded from a mic connected to a USB port would have a hiss, re my experience. His answer/educated guess was that all USB devices share a common bus and no device could have exclusive access to the bus and the hiss might therefore be inteference or interruption of the signal. This would not be the case with an audio card as it is possible to obtain exclusive access to it -- which sort of makes sense.

Problem solved -- finally. The answer, get a preamplifier (I got the Scarlet 2i2 , http://uk.focusrite.com/usb-audio-interfaces/scarlett-2i2 ). There are cheaper preamplifiers (that cost in the region of $30) out there that may work as well. As I understand it, a preamplifier allows the input gain (roughly translated the recording volume) to be controlled such that the hiss virtually disappears.

Please share if you get good quality audio with a cheaper preamplifier.

In case anybody should be asking the same question now, since this thread was originally posted Focusrite have introduced a smaller, cheaper model, the Scarlett Solo, that has a single microphone input.  If all you ever use is a single microphone it'll give the same quality as the 2i2 for rather less money.