Not always. It certainly depends on the type of game. But I'm getting tired of hearing folks talk about entire genres of games (like MMOs) saying the graphics are not that important.
Half the fun of a Harry Potter movie is to be dazzled by the wondrous imagery. Likewise, in many RPGs and MMOs, much of the attraction is the imaginative visuals.
There's value in a text-based game that lets the player's imagination create the imagery. There's value in a Habbo-style presentation that gives players only a visual foundation from which to imagine. But there's also value in the big-budget productions that do most of the imagining for the players.
Games don't have to strangle your computer to present impressive visuals. One of EQ2's most attractive features to me is the cinematic graphics, but WoW's graphics also impress me. No, not to near the same extent (I probably took 10 times as many screenshots of EQ2 experiences than WoW), but WoW's are good enough that I'm experiencing the gameworld more than creating it in my head. It's a very different form of entertainment than something like Habbo or Puzzle Pirates.
That's basically what it boils down to: I want to experience RPG worlds, not create them (as a player). Good graphics enable that. I've tried many free MMOs, and one of the reasons not a single one has interested me is that the graphical environment doesn't suck me in. Honestly, if the graphics are there, I can go a long way in creating my own gameplay.
I don't prefer good graphics only because I'm part of the computer generation which follows each new wave of visual technology with enthusiasm. In the movie and TV industry, many works undeniably succeed largely in thanks to their visual quality and imagination. If your game is a video game, you can expect many people to care about the video element.