From the perspective of a long-time Lego fan: the brand tie-ins saved the company, and brought us to both a new golden age of creative set design and a return to "build whatever your heart desires from generic, useful bricks".
In the 1990s, the company was foundering, losing popularity and money. They apparently concluded that Today's Youth couldn't cope with complexity and creativity, and shifted to trivial-to-assemble sets with the "specialized, next-to-useless" pieces you bemoan. This was not working, and things got worse, and the company seemed to know no path but doing that more -- trying to make Lego compete with Playstation and Xbox and Nintendo on the game consoles' turf.
But fortunately, the Star Wars deal came along. This gave a needed infusion of money, but more importantly, the more expensive, more complicated sets sold. This let them edge back in to selling other advanced sets, and, timidly at first but I think rather assuredly by now, to sell whole lines focused on creative building. In the early 2000s and late 1990s, you couldn't even buy a bucket of basic bricks. Now you can, and there's Pick A Brick, like the insert catalog from my youth in the 1970s/1980s but expanded to almost 1600 choices. That's awesome!