1.Discuss error backpropagation
2. Discuss LeCun et al.
2.1. Number of parameters estimated
> number of cases.
2.2. Details of estimation do matter.
2.3. When to stop is also important.
2.4. Network design.
3. Statistical connection.
(When the number of connection parameters is relatively
small,
compared to the sample size)
3.1 Logistic regression
3.2. Nonlinear regression, stochastic approximation
(learning rate problem),
asymptotics
for single hidden-layer netweok
(White, 1989,
JASA, page 1003-1013)
3.3 Projection pursuit regression.
3.4. Dimension reduction model.
3.5. Brillinger's result.