I agree that the single controller port is a legitimate flaw, as is the lack of AV out without the TurboBooster. I mean, even the PC-FX, which is way more focused on solo-gaming, has two ports. Stuff like that is why I see the N64 as a superior multiplayer gaming machine than, say, the PS1. Having those ports built in is really convenient.
I'm totally mystified by people who would take the form of the games as meaning that the games are worse than bulky carts, though. Outside of like Neo Geo, games have, for the most part, only gotten smaller and smaller as they got more technologically advanced. Especially now that they're on cards again, with the DS, 3DS, and Vita, and only getting tinier and tinier, you'd think people would see HuCards as a good thing. And who the hell wants stuff that takes up a ton of room? I love how little space my loose HuCards take up. When I then consider how much space my loose NES carts consume, it's just depressing. At the very least, I wish they were the size of Famicom carts instead of being half empty space for no good reason at all.