Don't think you should listen to the other guy at face value. The market for your skills is very bad right now. Ensure you don't lose your job, but definitely feel comfortable looking around for something better. When the market feels better and you're getting reached out to a lot, then be more assertive at work.