I originally wrote this a couple of years ago when I was called upon to defend an assertion -- that science fiction is best regarded as a special case of fantasy.
I would like to start with a little essay about how I came to this conclusion -- that science fiction is actually a subset of fantasy -- because it defines terms and addresses many counterpoints that have been raised.
First, when I say "fantasy" I refer to stories that are about things that aren't. This includes both the purely imaginative (probably won't happen) and the speculative (might happen). Fantasy in this sense is not on a continuum against science fiction, it is on a continuum against realism.
(Although "continuum" implies that it's something resembling a straight line, which I would not be willing to defend.)
One possible rule of thumb: it is a fantasy if filming it would require obvious special effects.
There is a smaller definition of fantasy, call it fantasy definition 2, which basically means "vaguely resembling Tolkien." I believe this is a marketing distinction rather than a literary one, since it doesn't even include most fantasy. It does have a vigorous fan base, however. That fact leads the term "fantasy" to be used as shorthand for "this particular type of fantasy involving elves and dragons and such." So -- those types of stories clearly are fantasy, but they are not the definition of fantasy. They are not the container, they are one of the things contained.
Most of what we think of as "genre" is really just marketing. They put the books in whatever section they expect people to look for them in. So, The Handmaid's Tale is in unclassified "literature" because Margaret Atwood isn't marketed as a science fiction writer. It doesn't have any bearing on whether the book is science fiction. And it's not supposed to. It's supposed to help Atwood fans find her stuff.
Sufficiently popular fantastic writers will often get moved to the "literature" section -- it doesn't make any difference to the content of the books.
One common way people attempt to distinguish science fiction from (other) fantasy is to claim that science fiction is about what is "possible." This definition is problematic because "possible" is not intrinsic to the work. Real world possibilities are always shifting -- some things that seemed impossible once are now possible, other things which once seemed possible are now thought to be impossible. Re: any 50s science fiction about faster than light travel, or life on the Moon or Mars.
This definition would lead us to the fairly awkward position of deciding that most "golden age" SF is no longer science fiction, because it is now known to be fantasy.
But, people don't tend to do that -- we still see Podkayne of Mars as science fiction. So obviously, we're using a qualifier other than "what is now possible."
You might at this point be tempted to use "what the author thought was possible at the time it was written" as a qualifier, but this is still problematic, because the author's personal beliefs are also not intrinsic to the work. We can't conclude that every SF writer who deals in time travel to the past actually believes such a thing is possible.
So, what makes something science fiction, if it isn't the fact that its scientific premise is possible, or that the author believed it was possible when the story was written?
Well, let's look at Frankenstein, held by many to be the first science fiction novel. Mary Shelley writes of a man assembling a new man out of parts of dead men, then bringing him to life through unspecified scientific means.
(Note: movies have implied that this means was electricity. The novel doesn't say, ostensibly because Dr. Frankenstein doesn't want anyone to replicate his methods. But other writings by Mary Shelley at the time show that she was familiar with experiments in galvanism and that her knowledge that a frog's limb could be brought to "life" with electricity probably did influence her thinking when writing the novel.)
There was no concept of scientifiction (early name for SF) in her day, so in her mind she was writing a horrific fantasy, and her twist on it was to involve modern science. Also, we now know that what she describes in Frankenstein is impossible -- an intact and very recently deceased body might be brought back to life through electricity, but a sewn-together collection of corpses never could.
And yet, the book is retroactively classified as science fiction, and nobody seriously objects. Why is that?
Well, its premise is scientific -- it can be expressed as "suppose, using science, we could do X." Also, it is concerned with the effect of technology and scientific discoveries on the human psyche and human society, addressing concepts such as unintended consequences and creator responsibility.
So, it is science fiction because it wrestles with scientific issues and concepts. "Being science fiction" is essentially an attribute of what is otherwise a fantasy novel.
But, if you still think SF and fantasy are two distinct and separate genres, consider: Is alternate history science fiction, because it is generally published as science fiction? Or is it only science fiction if it involves quantum multiple universes theory, or if the change catalyst is something specifically scientific? (I mean, The Difference Engine, which is alternate history specifically based on technology, as opposed to The Man in the High Castle which is alternate history based on a different military outcome to World War II.)
Or is all alternate history science fiction because Philip K. Dick, an SF writer, wrote the first one? Is Zelazny's Amber series SF with fantasy trappings, or fantasy with vague pretensions of being science fiction? Is Edgar Rice Burrough's Mars series science fiction, even though it has no scientific elements other than being set on Mars? How about the works of China Mieville, who writes extremely strange industrial alternate world fantasy which is also science fiction and horror? Or about Lovecraft -- his Dreamlands series is pure fantasy, but most of his Cthulhu Mythos stories are more or less science fiction. And what would you make of Terry Pratchett's novel Strata -- SF in which space explorers land on a disc-shaped planet and discover an absurd world built by beings so advanced that the result is indistinguishable from magic?
(He wrote it in 1981, so it predates any of the Discworld books -- however, the flat world is clearly a Discworld prototype.)
While there are certainly loose and generalized distinctions between most SF and most non-SF fantasy, I don't find that they hold up well under close analysis. There are too many exceptions. After a while it seems like everything is an exception and the distinction just isn't worth making any more.