Long-time SL user here. (I joined in 2005 or 2006 and was in there pretty consistently until mid-2012.)
Linden Labs made a series of poor decisions that got in the way of players & hype. Some of them were understandable or even unavoidable. Others were characterized by laziness. Some were policy decisions that were clearly advertised, while others were technical decisions that had subtle but pervasive effects.
On the technical side, there are two major things you need to know about SL. First is that it was designed in 1999 and a lot of design decisions that made some economic and technical sense in 1999 got baked in (in the same way that email is a crystallization of design decisions that made sense in 1970). A lot of the stability problems come from these decisions – like using a custom reproduction of some TCP features on top of UDP instead of just using TCP, and having a strict one-server-per-65536-meter-square-area rule. A second thing you need to know is that absolutely no effort was made (as far as anybody can tell) to avoid technical debt or keep things maintainable. There was a systematic push for new features, even when old features were completely broken. There were reportedly a certain number of LL employees outsourcing their own jobs to teams of less-competent contractors, because of a bonus structure that made that into a good plan. As a result, there were a lot of ‘fixes’ to major systemic problems and design failures that were essentially just quick hacks, and ultimately they created more bugs or got in the way of whole classes of legitimate user behavior. Anyone who has written anything in LSL that rezzes objects dynamically or interacts with the network will remember running into such limits.
On the user-policy side, despite SL’s reputations, LL has had a series of instances where they’ve lost users by creating vice laws. The first major one that I recall was when gambling was made a violation of the TOS. Gambling was basically the most effective way for somebody to make money on SL, and accounted for some large number of the businesses; while I never saw an estimate of how many people left, it was a lot. Reportedly, LL was worried that crackdowns on internet poker would affect them. Later, they made some zoning changes and limited porn/sex related shops to designated ‘adult zones’. The non-adult zones made up most of the map, and were almost completely emptied out, because one of the biggest draws for virtual reality is being able to engage in fetishes that are literally physically impossible and some large number of people were on SL specifically for that, while many others just sort of followed where the people were. Around the time of the zoning changes, the second life server system was reverse-engineered and cloned by the opensim / open grid system people. At a certain point, LL outlawed ageplay – the act of dressing the avatars of adults in such a way that they resemble children. Everybody who was on SL and had an ageplay fetish moved to OGS, because the open grid had no such rules (in fact, no rules at all – it’s a federation of sims running on individual people’s boxes), was comparably reliable, and had a variety of other features (the set of languages you could script objects in was extended, for instance, and pluggable).
Basically, LL made a series of poor decisions and alienated their own userbase. There was also a hype backlash, immediately after the hype appeared in the first place.
A lot of people liked SL. Some still do. In terms of being able to collaboratively build 3d objects with interesting scriptable behaviors with people across the world and across platforms, it’s still a good option (the only comparable environment I’m aware of for that is OpenCroquet, which isn’t properly peer-to-peer & requires you to code in smalltalk). In terms of places in which to, for instance, transform yourself into an anthropomorphic egg and use a penis the approximate size and shape of an adult deer to get oral sex by a levitating head in a jar, it’s the only system I’m aware of that lets you do that and then immediately afterwards go dancing in an all-ages nightclub. It’s the only system I’m aware of where you can buy a dildo and then program it to take voice commands to follow people around by flying through the air toward their ass, chasing them, and then spew out fireworks when it gets within range.
SL started out as a place characterized by the ease with which people could make self-replicating swarms of dicks, and how common such things are. SL is now a place where both self-replication and dicks are strictly limited, but literally nothing has been added in their place of comparable interest. So the people who wanted to build self-replicating dicks left.