Confidence may come gradually but it can vanish very suddenly.
If a number of investors have enough invested in the system that their investment become worth more to them than maintaining the bitcoin system and some event decreases the confidence in the system, then those investors will attempt to "liquidate" their holdings. Whether they succeed or fail, the impact on the system will likely be terminal since there wouldn't be any entity ready to step in and buy unwanted bitcoins.
If a number of investors have enough invested in the system that their investment become worth more to them than maintaining the bitcoin system and some event decreases the confidence in the system, then those investors will attempt to "liquidate" their holdings. Whether they succeed or fail, the impact on the system will likely be terminal since there wouldn't be any entity ready to step in and buy unwanted bitcoins.