A new approach to modeling sequential data, called Deep Equilibrium Model (DEQ) is introduced. Motivated by the convergence of the hidden layers in existing deep sequence models, the DEQ can find these equilibrium points directly through root finding. The process is equivalent to running an infinite-depth feedforward network, with the added benefits that the equilibrium points can be backpropagated using implicit differentiation. This approach bypasses the depth of the network, and thus uses constant memory.