Errata

These errors were present as of July 28, 2022 but may be fixed in subsequent printings. Please contact us if you find anything else!

• Page 42: in footnote 5, “Epilogue Epilogue” is replaced by “Epilogue”.
• Page 46: in the second paragraph, “λ tanh σ(z)” is replaced by “λ tanh(z)”.
• Page 108: in the second paragraph, “as a network get deeper” is replaced by “as a network gets deeper”.
• Page 124: in the third bullet point, “perturbation stays” should be replaced by “perturbation stay”.
• Page 167: in footnote 16, following from the previous page, “Epilogue Epilogue” is replaced by “Epilogue”.
• Page 213: on the first line of (8.48), the subscripts of the final two activations are reversed: for the first, “α₁;j” is replaced by “j;α₁”, and for the second, “α₂;j” is replaced by “j;α₂”.
• Page 219: on the second line of (8.71), the square bracket should be removed.
• Page 252: on the third line of (10.12), the subscripts “i₁” and “i₂” are both replaced by “i”.
• Page 280: on the third and fourth lines of (10.113), the signs of the four “z” terms need to be reversed. Additionally, on the third line, “ϴ₀₀ δδϴ[2]” should be replaced by “4ϴ₀₀ δδϴ[2]”, adding a missing factor of 4.
• Page 280: on the second line of (10.114), the signs of both “z” terms need to be reversed. Additionally, the factor of “4” at the front of the line should be removed.
• Page 281: in (10.115), the factor of “4” at the front of the final term should be removed.
• Page 281: on the second line of (10.116), the factor of “4” at the front of the line should be removed.
• Page 303: on the final line of (11.37), the subscript “δ₃” is replaced by “δ”.
• Page 307: though not technically an error, on the fourth line of (11.48) it would be nicer to replace both instances of subscript “K” with subscript “G”.
• Page 309: though not technically an error, on the third line of (11.51), it would be nicer to replace the subscript “K” with subscript “G”.
• Page 321: in footnote 14, “Epilogue Epilogue” is replaced by “Epilogue”.
• Page 322: in footnote 16, “Epilogue Epilogue” is replaced by “Epilogue”.
• Page 328: in the top paragraph, “Epilogue Epilogue” is replaced by “Epilogue”.
• Page 336: in the last paragraph, “Epilogue Epilogue” is replaced by “Epilogue”.
• Page 338: in the second term in the second line of (∞.6), there is a missing factor of 1/2.
• Page 339: in the first sentence, the indices “(i₃,δ₄)” are replaced by “(i₃,δ₃)”.
• Page 343: in the second paragraph containing (∞.29), “single input ddNTK” is replaced by “single-input ddNTK”.
• Page 364: in the second paragraph, just before (∞.106), for the ε function the argument “t” is replaced by “s”.
• Page 364: on the first line of (∞.106), for both ε functions the argument “t” is replaced by “s”.
• Page 365: on the first line of (∞.109), for all three ε functions the argument “t” is replaced by “s”. Additionally, the termination index for the sum “t” is replaced by “t-1”.
• Page 426: in the top paragraph, “Epilogue Epilogue” is replaced by “Epilogue”.

We thank Yuxi Liu for the discovery of a number of these errors. Feel free to check out Yuxi’s nice page on PDLT for some additional insights.