- Go back to the Recording tab in the Sound panel,
- double-click your input device, and
- switch to the Listen tab.
If you have Listen to this device checked, you'll hear everything from that microphone in the select output device. This can be useful in certain circumstances, but in most cases will just drive you crazy.
Uncheck the box and hit OK to stop that.