Residual Connections - Residual Networks (ResNet)

• helps solve the vanishing gradient problem
• can train networks of up to 1000 layers

General Idea

ResNet Block

ResNet Block Variants

What if Input and Output are Not the Same?

Add a linear layer to reshape.