Some things are impossible.
That shouldn't be controversial, but our culture is filled with messages that go against it. Nothing is impossible, you can do whatever your dream, don't give up, just try harder, believe that you can, have faith. Our fiction/mythology is full of people overcoming difficult odds - and impossible is just a harder kind of difficult, even more admirable to inevitably overcome.
And like, that's sometimes a good attitude to have, we've accomplished a bunch of stuff by being persistent and obstinate. "Try harder" can be pretty good advice. But sometimes things that look impossible really are. And if you're attempting something impossible, giving up isn't defeatism - it's the rational thing to do.
(IMPORTANT NOTE: even when something is technically possible, "try harder" can still be bad advice. We know about burnout and depression from working too hard, we know working "crunch time" damages people. Plus it's generally awful to tell someone to work harder from a position of privilege when they're facing obstacles you don't even know OKAY.)
Impossibility is beautiful, because it's so much bigger than possibility. Finding a solution to a problem is a small thing, you can just show it to me. But to claim that there is no solution, that no solution will ever exist, no solution can ever exist, that no amount of ingenuity and effort will solve it - that's enormous.
OKAY what does a genuine impossibility look like?
Guess I'll point you to some classics http://en.wikipedia.org/wiki/Seven_Bridges_of_K%C3%B6nigsberg, http://en.wikipedia.org/wiki/Square_root_of_2, http://en.wikipedia.org/wiki/Squaring_the_circle, http://en.wikipedia.org/wiki/Angle_trisection, you probably know about these already whatever. In general this is something that mathematics does, proving things impossible. And then because humans are bad at accepting impossibilities we still get amateur mathematicians (less charitably, "cranks") trying to square circles and trisect angles because hey the establishment thinks it's impossible but maybe if I just try really hard I'll find something they missed, never mind that it would take a lot less time to verify the proof.
Importantly, these claims are definite but conditional; they say "this is impossible with the methods of Euclidean geometry" or whatever but with different tools you can do the thing. You can't write this as a fraction but sure you can write it some other way, maybe you can build another bridge or a teleporter idk. That doesn't diminish their beauty - like there are infinitely many constructions you can do in geometry and to be able with just a finite amount of work to show none of them square a circle is pretty rad. But yeah it's a thing, the only way we get to be 100% confident that something is true is by tying it down with enough qualifiers. Mathematics is fairly special because we do have those qualifiers, most places we don't and so we can't unambiguously prove impossibilities. Science operates by inventing mathematical models and experimenting to test them against reality - we can prove impossibilities in the model but never be certain whether the world follows the same rules. So while actual definite impossibility exists and is beautiful, it is something alien, we mostly live in an ambiguous world of uncertainty. (Which is not to say things aren't impossible, just we don't get to know for sure.)
Okay this is a blog about games right.
Now games are often power fantasies. Everything in them was placed for us to play with: here's a ledge, you can jump to it / a door, you can open it. Everything you see is possible, it's there for you. If there's a door that's just scenery and can't be opened, that's a failure of the game - and players complain like anything. The "perfect" game is imagined as a wide open space in which anything is possible, that never tells players what they can't do.
Which is fine i guess. But games also have the scope to let things be impossible. They're self-contained systems with fixed rules we can know; so we can prove impossibilities. And I said that's beautiful so okay let's, how?
(Note that the process of creating complex software can obscure what the rules actually are, we can end up with unintended rules - bugs, glitches, exploits. So as @flantz often points out, players end up doing actual science in experimenting to discover the laws of the toy universe, rather than the mathematics of knowing the rules and deducing their implications. In a videogame, the actual rules are what's written on the software, not what the designer or the player had in their head. Board games though, the rules run in the player's minds so they literally are exactly what you think, it's pure mathematics.)
(Lots of videogames do let things be impossible FOR NOW. The metroid thing of areas being inaccessible until you've acquired a new ability. Decent compromise I guess.)
With puzzles, what I usually see people do first is just push buttons, try to solve them without having to think. A good puzzle takes a long time to solve that way but is quickly solved once you think of the right idea, to make you give up and switch your brain on. But if it takes too long to think of the idea you'll just start going through a list of all possibilities, which isn't interesting either.
Now what if it's actually impossible? It can't be solved by button-mashing, you're wasting your time. If the state-space is finite then it can be solved (proven impossible) by exhaustive search, but very slowly - if infinite it can't. A good impossible puzzle would require a long time to be confident it can't be solved by trial and error while being quickly proven impossible once you think of the right idea.
Okay I don't really like puzzles in general and this kind of thinking might lead to things I like more? Looking at a puzzle and not knowing whether it's even solvable, and thus which approach to take, kind of a delicious uncertainty. Trying all possible moves to rule them out is super boring though just as much as trying them all to find the right ones, so that really has to be avoided. But the classic "prove or find a counterexample" topology exam questions are the best so.
Vertex Dispenser was my attempt to put an idea from mathematics into a game somehow more interesting than just make a puzzle like everyone else does. It's this dense knot of ideas, the space you move through in an action game is also the territory you command in a strategy game is also the graph you colour in a puzzle game which feeds resource management for abilities in the action game / the negative space protects the positive and everything's connected. Too weird. Anyway I ended up putting in a bunch of conventional puzzle levels too, I wanted the last one of those to be impossible but was convinced not to. What I did instead was have the level be a collection of ten separate puzzles which you have to solve half of, the rest being impossible - so you have to recognise which ones are possible and solve them while not wasting time on the others.
Maybe that wasn't a good example but here's this idea of recognition. Actually formally proving something is a specialised skill most people lack (it's not scary but it's not instinctive), I'd love to see a game that teaches this but I don't know if you can do it in an interesting way without just being maths, also to verify proofs in software tends to require input in a way that's way less intuitive than what mathematicians usually do so yeah it'll be hard to make a good game out of that (someone do it please). But just requiring recognising impossible situations without proof is still pretty interesting, usually how this works is you find some invariant feature that the allowed moves can't change (parity's a common one), recognition doesn't have the beauty of absolute certainty because a modified trial-and-error approach will still work if you're right most of the time but hey compromise.
Corrypt was getting into this kind of area too - it starts solvable but you can enter unsolvable states and then you have to recognise them and undo, preferably without wasting time on