A quick search tells me the voltage on the 5v pin isn't always the same. I came across a thread, similar to this, where a voltage regulator was suggested, to keep it at 5v...but should be possible.
5v seems too low, but you could always try it using a spare USB charger to get 5v out and see if that powers your converter reliably.