It's like any other language, though.
It's best learned through immersion.
Unfortunately, playing games and watching TV isn't exactly immersive. It's only one part of the puzzle.
Just remember, even in Japan, there are idiots that know Japanese by virtue of being native. If those dingalings can do it, you can eventually figure it out too.
Same for any language, really.