After days of conversations with everyone from internet people to people on the streets, this is where I stand on the subject of LOST.
The internet seems to have split into two camps about LOST: people who wanted to mysteries explained, and people who really liked the final message that it was the people who mattered so forget about all the mysteries. I think the camps are limited. One camp says that if you hated the end of LOST you will hate 2001 but love the Phantom Menace (because The Phantom Menace "explains" the force). But I do not think either of those are fair comparisons. I did not want the mysteries of Lost "explained" -- I wanted them unified, related to one another. Still mysterious, but connected. 2001 has some very mysterious scenes but it only asks me to accept one: the obelisk appears at moments of major cosmic evolution, and the movie shows us one from the distant past, and one from the near future. The willing suspension of disbelief works awesome for the obelisk, but when I am asked to accept many unconnected mysteries I find this harder to do, if you are claiming you are telling one story. One of the many mysteries on LOST was the ghosts, but even that "one" mystery turned out to be several: some ghosts were actual ghosts, some where a shapeshifter impersonating the dead, some were maybe psychic projections of the living or something like bi-location (Shannon seeing Walt covered in water on the island when he was captured by the others at the same time elsewhere), some were hallucinations (like Hurley seeing his friend from the institute), and some remain inscrutable ("taller Walt" meeting Locke and the end of season 3 -- can't be the Man in Black because he can only take the form of the dead, can't be a ghost because Walt is alive, it is hard to see how it could actually be Walt projecting himself psychically from far away or something, and to say it was a part of Locke's subconscious seems like a stretch).
The people who agree with the creators that the characters have to be the main thing are right; this is a principle of screenwriting. But equally important to screenwriting is the resolution of implicit promises to the audience (that, say, the statue had something to do with the weirdness elsewhere on the island). And frankly, it was actors like Terry O'Quinn and Michael Emerson that made the characters compelling. Juliet's sudden change of heart about the bomb at the end of season 5, very convenient for the writers but not especially compelling from her point of view, hardly suggests that they are taking character as seriously as they claim, though they are not without some tremendous ones. The characters are great at times, but no one is going to confuse this with The Wire, where everyone in the first four seasons is a fully realized human being, and never a device for a writer's agenda.
So how to make sense of LOST. Here is what I think happened. Not a ton of evidence, but it explains a lot of what I wanted explained.
LOST started out as a show about a plane that crashed. Everyone on the plane died. The characters on "the island" were actually in the kind of purgatory that we would later see a version of in the "Alt U" of season 6, except the island, rather than the mainland, was the purgatory. But like they said in Season 6, the dead are all connected and created this place -- the island -- in part out of their subconscious, a place where they had to work things out before they died. This is why when Walt reads about a polar bear one appears -- because the place they are in is a creation of the mind, a collective dream world. This is why children are taken by the others -- because they are innocent and have nothing to work out, they can go straight to the afterlife. This is why the others can take people like the tailies with supernatural force -- because they are supernatural in nature. This is why even though the Others kill people they keep saying they are the "good guys" -- because in the larger scheme, they are. This is why children cannot be born if they are conceived on island -- because this is a place of the dead, not a place for new life. This is why people are healed -- because it is a spiritual place of second chances, and physical limitation has no place here. This is why the island cannot be just accidentally "found," or easily left. Getting to the island is like getting to Hades. The smoke monster, like the Others, is some important force for sorting out people who need to go wherever they need to go. To destroy some, and leave others, still working things out, be (which is why it leaves Locke alone early on). This is why Christian's body is not in the casket in season 1 -- he was dead already at the time of the crash. This is why in the early seasons people "die" (read: move on) when the have worked out their issues, the issues from the past we see in flashback. This is in fact the point of the flashback structure -- to show us what they have to work through on the island. This place has been put together not only by the castaways but also by previous inhabitants. The Dharma station could be the remnant of a group of scientists who died in some kind of science disaster similar to "the incident" and have since moved on to the upper levels of the afterlife. The statue could be the remnant of some much earlier culture who died, who marked the place as the land of the dead with an Egyptian statue. They statue may simply be like the statues that mark the underworld, the products of gods or something.
Lost became a huge hit, a bigger hit than anyone could have predicted. And the fans were obsessive and rabid, and loved to theorize. And the "purgatory" explanation was the explanation that a lot of people landed on. And it was rejected by a lot of other fans as super-lame, because it is frankly, just a notch away from "it was all a dream." And at the end of season 2 or the beginning of season 3 the writers freaked out. They did not want to disappoint this awesome fan base. And they changed the story -- the overall story of the show as a whole. The writers, who said the cages were a metaphor for their lack of direction, worked out, at that point in the show, the deal with ABC for the exact number of episodes till THE END. And that is when they cemented the change. The island, in this new draft, was no longer any kind of purgatory. It was a real island with a lot of weird magic stuff. This invigorated the story hugely, because things that were off the table were suddenly on. And as fans we could feel the new energy. People could find the island without being dead themselves, like the people on the freighter, and the castaways could return to LA for 3 years, then have to come back. Jacob was introduced in the back half of season 3, and became a focus, and the smoke monster became the Man in Black. Some elements of the earlier draft stayed in, like John Locke confronting his father in season 3. In retrospect the Others just went off island and kidnapped him, but I feel like in the first incarnation, he had to confront his father before moving on, just as Jack has to reconcile with his at the end. And notice how JJ Abrams distances himself from the project after it gets started -- the island as purgatory idea was his, I think, and as they moved away from it, he moved away from it.
The creators always said they knew the end at the beginning and after the finale aired they said that was the ending they always had in mind. And I believe them. I think they always envisioned the show ending with Jack's dad taking them all into the upper levels of the afterlife. I just think they initially envisioned this happening from the purgatory of the island, not some Alt U that looked like it was created as the result of the atomic bomb. Two things to notice here.
First, the nuclear bomb they were all standing next to when it went off effectively did nothing, except safely transport everyone into the present day (Juliet was already dying at the time she hit the bomb). Of all the crazy things on the island this is one of the most crazy. I had a hard time understanding what the point of everyone being transported to the 70s was at all. Sawyer and Hurley and that group got stuck there. I assume Jacob sent Jack and Kate etc to join them. But then it seemed like he knew they were all coming back with the bomb together. It fits the idea of the castaways needing to be together for travel that comes up more than once in the show, I guess, but this does not seem like enough. It seemed like the point of the 70s bit was it could have been at the origin on Ben's knowledge of the candidates (THEY were the ones who wrote detailed files on THEMSELVES that the Others had), it seemed important that the group Ben kidnapped at the end of season 2 was the group he knew as a kid (but no -- he forgot all about them), it seemed like we would learn some Dharma secrets, about its origins or something -- but again, no, nothing. It turned out the real reason for everyone to be in the 70s was to create the red herring of the Alt U with the bomb. We start Season 6 thinking we know what the Alt U is, which means we don't question it as people questioned the nature of the island in the first two seasons. And because we don't question its basic nature we don't anticipate, as we did back in the early days of the island, the rug-pull reveal at the end that it is purgatory. The same purgatory the the island was supposed to be in the first draft. This is why season 5 was the weakest LOST season (though I did really like Sawyer and Juliet) -- because, for all their talk about character, it existed solely as a device to wrangle back in the ending that had been abandoned years ago.
Second, and I feel like I have not heard explanation of this one at all -- isn't it strange that the purgatory of the "Alt U" is pretty much heaven? Why does anyone need to remember the island if they are all doing to well, living so happily? Why would anyone need to move on from the already happy lives they have? It is very strange that the Alt U gives us a character arc that takes our heroes on a journey from very happy to blissfully happy. In a big story we expect more change (e.g. Jack goes from being a man of science to a man of faith). It is also very strange that Sayid kills people in this purgatorial realm and still gets to move on with the others. It is very strange that Keamey is there. And it is very strange that Jack so unceremoniously leaves behind a son that was just an illusion. This seems to me to be a remnant of a draft where characters would have gone various directions, after having been tested -- on the island. Some people complained that too much time was spent in the "Alt U," especially before the Desmond episode, but the real problem may have been that it was a years long idea compressed into the subplot of a single season. It was why the final minutes felt a bit clunky -- like the first few episodes of Dollhouse, it is the clunkiness of the writer who is starting a story and does not quite have a handle on everything yet. Because that is exactly what it is. It was written long before they figured out what works and what does not work on this show.
The alternative is to believe that Walt reading about a polar bear and a polar bear appearing is a coincidence. The alternative is to believe that the statue of Taweret (goddess of childbirth) has nothing to do with whatever causes children not to be born (as a result of "The Incident" in the 70s -- an important plot point never explained and more importantly never fixed by the castaways). The alternative is to believe that it is a concidence that someone like Walt who can project like a ghost (?) is brought to an island of ghosts, where there is also a guy who can for an unrelated reason, appear as the dead. It's not that you can't do this -- Warren Ellis's theme on Planetary was "it's a strange world, let's keep it that way." This is exactly what Lost implies the point of the mysteries have been -- these are things that just HAPPEN in the world; whatever happens, happens: psychics, and ghosts, and electromagnetic energy and shapeshifters all that. The world is a weird place. Deal with it. People who liked this theme, I can understand the appeal, but I want those people to understand why it makes me a little crazy. What bugs me is not so much that this is the theme, but that it seems clear to me that this is a much more recently tacked on theme. And the way the show is designed and structured -- because we do not realize the island in all its unconnected madness was simply REAL until a few minutes before the end -- we have no chance to decompress this theme. So it could work theoretically, but the two drafts work against each other the most right here. because they try to slam together two completely different endings into the final 15 minutes. They can sort of pull it off because they are switching between two universes, but they also just don't have time to do both justice.
I still think LOST is the single greatest collection of act breaks (go-to-commerical, go-to-credits, and end-for-the-season scenes) in the history of anything, was able to tell any story they wanted from week to week (let's fix a van, hunt boar, time travel, see ghosts, or do a version of Xena or Alfred Hitchcock presents), and had Michael Emerson and Terry O'Quinn, and stories about Desmond, and these things will make always make it one of my very favorite shows. And while I think something in the concept was fundamentally broken, I also don't wish the original vision had played itself out, because we never would have had Ben Linus become such a huge character, or have the shock of that first flash-forward, or have Desmond realize he is going to wake everyone up, any number of other kickass things. So they only way it gets to be the LOST we know and love is like this. Which means I can't really complain about it -- because if any of my complains were addressed, we would lose something that made the show great, and I would not trade the great moments for anything -- cause my god they were AWESOME.
(A lot of this post comes out of conversations with people in real life and in the comments. Thanks to everyone for helping me work all this out.)