The thing is, a setting can't be a living world if it's operating ONLY at the level of the GM's conscious mind and direct thoughts. Not only would the surface level of consciousness be unable to catalog every detail of the world or every characteristic of the npcs/creatures/things in it, but the experience of it from the GM's point of view would be more like an automaton than an organism.
So random tables help the process of that switch, of the GM's own Immersion INTO his living world. They're not exactly necessary, but they can be tremendously useful, for the same reason that randomness in character creation is useful.
Have you ever seen a player, playing in point-buy or other mostly non-random systems (even D&D from 3e onward), make a character, and then you realize that this character aside from some mechanical differences is basically the same guy they were playing last time, and the time before that? Sometimes they may even choose a different class, and yet you'll still get that feeling!
This is because it can often be hard for people to slip fully out of the level of their conscious mind; even if they're trying at all. The Imagination is a tricky thing, and it's easy to choose the path of least resistance, and what that leaves you with is a sense of a character OR a world that's quite flat.
A random table, a well-crafted random table (or other random methods) can help with this sort of thing specifically by introducing an X-factor; something that won't come out of one's immediate intellect. Add that factor, and then figure out how you can accommodate it into your world, and suddenly it forces a level of creativity you wouldn't have had before.
Random tables are a motiveless, neutral tool for bringing out the inner life of a setting.
Currently Smoking: Neerup Poker + Stanislaw Winter Flake