After three years of writing and reading about science fiction and fantasy while being surrounded mostly by mainstream-readers, there are only two conclusions I'm quite certain of: a) those who say that mainstream is an enlightened genre and that SFF is irresponsible wish-fulfilment don't know what they're talking about, and b) those who say that SFF is enlightened and mainstream is limited and backwards don't know what they're talking about.
On this rather fascinating journey, one comes across a number of pronouncements, all very shiny and impressive, and all very dubious. The first thing one hears about imagination versus realism is: "If you really mean to describe the real through imaginary things, why don't you just cut short and describe the real as it is? If you don't mean to describe the real, then why should we bother reading you?". These are very foolish questions behind the apparent common sense. The dichotomy they pose (describing the real versus making stuff up) is non-existent in fiction. It's "fiction" precisely because it's made-up. The common answer to this is,"Yes, but realistic fiction describes things that could have existed, and that are representative of the real, even if they didn't literally exist. Speculative fiction does none of that."
If anything, such a comments is very telling about the dangers of realism. A good writer who writes realistic fiction will see their words taken as truth. Readers will take ot for granted that their capacities of observation are sound enough, and that the characters, situations and places they invented must be representative of reality. In fact, realism (I'm not talking about realistic fiction; realism happens in the wildest fantasies as well, just as some mainstream writers will include some fantasy elements in their writings at times) relies on an act of faith from the reader: even if what is described never existed, it is supposed to be a very accurate approximation of the real. As soon as a writer is brilliant enough to craft sentences that will resonate with their readers, they will be branded as "a keen observer of the real", and their words taken as documentaries. The paradox is, for that to happen, they need to tell the readers things they already knew, or believed in, only worded in a strinking way. A writer is good as portraying the real if they show you something you already observed yourself. In short, if what they write resonate often enough with your own personal biases, prejudices, and occasional accurate observations, you will trust them. In this, realism can be even more misleading than fantasy–at least fantasy doesn't pretend to teach you anything.
What fantasy does (or any genre that relies heavily on the imaginary) is to write about situations that could happen, if such-or-such (usually very unlikely) conditions were realised. Fantasy doesn't send the readers any messages about how they should read the real: they tell us about possibilities, no holds barred. Anything is possible, there are no limits to what a competent can invent. The idea is beautiful. The reality of the practice… is much more problematic, as has often been pointed out.
Imagination is not without limits. Everybody, including writers, sees the world as a numbers of parameters, some of which can be changed, others of which are taken for granted so much that writers seldom even think they could change them. The results can be quite baffling, once you start thinking about them; the problem, of course, is that this kind of awareness takes a lot of work, a work that's still in progress for most of us. You've certainly noticed, for example, how much easier it seems for writers to imagine elves, gnomes and aliens than to create non-white characters. This is a very common pattern in works that primarily rely on imagination: the incredible, the monstrous, the shocking, seems to be much easier to imagine that plain daily-life realities we're surrounded by but seldom admit into our own private representations.
Examples are rife. My boyfriend and I are playing Final Fantasy Tactics at the moment, and we were quite amazed to discover that the "charm" spell can work on persons of the opposite sex, on monsters, but not on persons of the same gender as the spell-caster. In other words, it was apparently easier for the developpers to imagine a girl charming a mind-flaying weresquid than a girl charming a girl. In many books describing advanced civilisations, you will come across description of societies that enjoy complete sexual freedom, to the point that incest is permitted. Aside from the fact that I find this portrayal of incest as the "last frontier" in sexual freedom, that we have to conquer to be finally liberated, really very icky (incest is not cool–it's very real, and it destroys lives), I can't help wondering how come it seems to be easier to talk about incest-allowing societies than to portray places when women can have all the sex they want and not be considered as pleasant sluts, including by the writer (ie. be portrayed as complex, evolving and not exotic characters). Or to create societies that have no concept of virginity, of marriage, of paternity, or whatever, and are still dignified. There's a very wide continuum between the level of uneasiness around sex in our society and permitting incest, but that continuum is hardly ever explored.
I could carry on and on. What about utopian societies where people eat little pills, live in zero-gravity chambers, communicate through virtual channels all the time and are very happy despite never having seen a tree in bloom? Are they really that much harder to imagine than a society that finally managed to wrap its collective mind around the difficult notion that natural resources have to be carefully managed if every human is to have their share, and that just because you can afford to buy something doesn't mean you're entitled to it? Is it really that much easier to picture a utopia humans somehow happily surviving the complete destruction of nature than to portray them finally growing up and learning to manage what they have in a responsible way?
One could always argue that the extreme is always more interesting, but I don't think it's a good excuse. People get desensitised to the extreme. Nature-less futures have been clogging science fiction for nearly a hundred years. They're not mind-boggling anymore. And who can honestly say they felt shocked and shook in their most deeply-held values when they read about Jaime and Cersei Lannister, or the Targaryens, in A Song of Ice and Fire? If imagination had no limits, it wouldn't walk on the same exhausted paths over and over all the time. The fact is that imagination relies on representations that range from clichés to generally-held, unquestionned beliefs. True imagination doesn't consist in making up stuff that doesn't exist. It consists in making up stuff that has never been made up before. That's completely different, and that can very well involve a large measure of realism. But as long as fantasy writers believe that they've done their job if they pictured something that didn't exist in the real world, they risk coming back to the same conventional and faux-shocking, faux-extreme representations.
As far as I can tell, both camps, mainstream-lovers and SFF fans, can take the reproaches they try to pin on the other side for themselves: realism relies on make-believe, when imagination comes back to replicating common representations over and over. Perhaps it's just time for everybody on both sides to realise this, learn some humility, and then work on their inherent weaknesses to develop their full potential.