I think everyone's spot on with recommending the PC Engine Duo-R or RX. I myself just really got into the scene about six or seven years ago when I discovered Ys I&II, and then I spent a few years just trying out a lot of the library before knuckling under and getting a Duo-R. I got extremely lucky and snagged one on E-Bay for a little over $100; although I had to buy the power supply, AV cables, and controller separately. I spent about $160 total to put it all together and I've had zero issues despite putting a lot of hours in on it.
Esteban's guide for Arcade Card games is a great resource. I have Gulliver Boy and 3x3 Eyes, which can actually be played without the card, but with the card you get much faster load times and even more animation in cutscenes. I wouldn't recommend it for everyone, but I'm very happy with my Arcade Card Duo.
If you like Splatterhouse, the Japanese HuCard has the benefit of being both cheaper and uncensored. The US version actually makes some really odd cuts to relatively tame parts of the game.