Fix channel close procedure when the peer dies or our handler goes down

Fix channel close procedure when the peer dies or our handler goes down #19647