D&D is the archetypical RPG system. I love it (except for 4th edition which I’ve sworn never to DM again) but Vancian Spellcasting is a bit old hat at this point. Also it’s a highly specific and slightly weird concept. Wizards imprint spells on their brains and forget them after they’re cast. I can see the play balance appeal, it’s a fantastic way to limit the Magic-User and force the player to think ahead carefully when picking a spell list. It definitely worked for The Dying Earth series, but now it’s spread into all kinds of other worlds.
It is a very Hard Magic system (see Sanderson’s First Law for a rundown of Soft vs Hard magic). All of the rules are there and a wizard knows down to the feet or yards exactly how big his fireball will be. And it is exactly the same every single time (discounting for the moment the rare fumble or critical success). For the sake of a coherent game system this is an excellent thing, but for portraying magick as a dangerous, unpredictable, mysterious force it is terrible. I tend to lean towards a Soft style of magic in my world-building, but how does one incorporate that into something as inherently mechanistic and well defined as a set of role-playing game rules?
DCC addresses this issue with insane amounts of randomness. All possible effects of a spell are described in terms of exact rule mechanics but there are so many of them that the Wizard or Elf never knows what’s going to happen when they tell the GM “I’m casting a spell”.
Lamentations of the Flame Princess has less randomness in the spell casting but the severity of the magic fumble tables is really something magnificent to behold, especially the last two Free RPG Day supplements (Vaginas are Magic and Eldritch Cock). Any time a wizard announces that they are casting a spell, all of the other players should reach for a new character sheet and the GM should be slightly worried (and sadistically gleeful) because some of those fumbles can easily end the world or fundamentally re-write reality.
Both are terrific for the gonzo, over-the-top style of play that they are trying to capture, but realism suffers when every magician has a rhino head and seven eyes. I’m looking for something a little less larger-than-life and a little more gritty and down to earth. Something closer to Wagner’s Kane rather than Moorcock’s Elric (Chaos Butterfly anyone?). A spell point system like in Stormbringer or Openquest at least gets rid of the Dying Earth vibe, but perhaps makes things even more mechanistic.
Balancing the level of uncertainty seems to be the key issue. The rules have to be complete and fair, but the more the players are certain of how magic works the less magical it becomes.