Just like DragonmasterDan, nothing pisses me off more in a podcast than screwed up facts. Having said that...
-VGMpire is growing on me - they get some stuff wrong and banter on about bullshit every once in a while but it's a cohesive show with great music choices and some good background info on the games/developers involved. The hosts have good chemistry together and are pretty funny. Has taken up the VGM torch since the Legacy Music Hour went kaput.
-The above mentioned Retronauts link is their new Kickstarter-backed series of podcasts...think they're planning on fundraising for this every year to keep it going, which is sure to cause some controversy. This latest "version" of the podcast started out the gates kicking ass; they had some really good topics lined up (the game music episode was top notch). In recent weeks it's really shit the bed...part of the Kickstarter deal was that certain levels of backers got the pick topics of discussion, and lot of the topics are horrible and/or a snoozefest. But it's still my favorite retrogaming podcast regardless.
-I've been a long-time listening of the Insert Credit podcast. It's kind of an acquired taste - like ESPN's "Around the Horn" but for games. The show is about 50/50 retro and new games, so it's not a true "retro" podcast. The hosts can be polarizing also. Some episodes are stellar though and they always broadcast the recording of the show live on Youtube.
-HardcoreGaming101's Game Club 199x podcast is another one of my must-listens when it comes out about once a month. If you can put your hatred of HCG101 behind you, this is a good "book club" style deep discussion about one game a month, spanning PC and console. They pick some really obscure ones to discuss sometimes that are really interesting.
-Not truly game-related, but the VGMpire guys do a podcast called Lasertime that's generally about 80's/90's pop culture nostalgia and it's awesome. Listen to their "Shame Songs" episode for a good laugh.