I was a flight simmer before I was a race simmer and, initially, I liked the great graphics more than the realism. Like all simmers though I eventually started turning up the realism settings and learned more about the aircraft and bought add-ons to increase the realism.
I still fly FSX because the 'newer' sims out there don't give me everything which that setup does.
With sim racing, I started out in Forza 3 with an MS FF Wheel and very forgiving gameplay, but great customisation options. (
Seriously, why can't a modern PC sim make it that easy to PAINT YOUR CAR? 
)
I then decided to try a PC sim (Race 07) about 5 or 6 years ago, which led to me buying a G25 and setting up a racing rig and I never looked back. My MS FF Wheel has sat in a corner ever since that day. I raced GT5/GT6 on PS3 but even there the handling doesn't feel 'right' on the cars, no matter how good a sim the creator tries to claim it is, but it does have quite pretty graphics.
Currently I race:
Race07 (still, because it has the MINI!),
Assetto Corsa (very pretty but needs optimisation, decent handling, FFB not too fiddly but lets go suddenly in corners,
no MINI),
R3E (great physics, nice handling, nice looks, FFB easy to tune in once you ignore other users and go for what feel you like,
no MINI, no dedicated servers) and
rF2 (Nice physics, nice handling, looks good with some fiddling, FFB challenging to set up for the response I want,
no MINI)
I'm happy that I have all 3 'new gen' sims and I'll keep racing all of them as long as they have something I want.
Did you notice the complete lack of MINIs there? Did you?
Looks aren't everything, but without good graphics it is much harder to spot your braking point. I had a config issue in Race 07 last night for a 40 minute Mini race. Had to wipe and restart config but forgot to change the graphical settings to Max. This resulted in some trackside objects/trees/etc. not being drawn and two of my braking points vanished! It made those corners 'interesting' to say the least.
On some real life race tracks, my braking point in the past was something as simple as a crack in the concrete on the outer kerb, or a blob of paint on the track. Without that level of detail it becomes less 'sim' and more 'guess', in my honest opinion. Graphics matter, not just because they look pretty.