That's actually not even funny. That's just sad. And the comedian's expression makes me think he knows it. He may have been going more for pathos than comedy and the audience's reaction strengthens the sense of pathos.
My only response to you is "Pretty much". There's definitely a moment where the audience realizes "Aw, we're sad people."