ML Internals
This document contains ML basic internals
:
y' = b + w1 x1 (single feature)
y' = b + w1 x1 + w2 x2 + ... (multiple features)
The kinds of loss functions:
L1 Loss = Sum ( |Actual - Predicted| )
MAE (Mean Abs Error) = L1-Loss / N
L2 Loss = Sum ( |Actual - Predi;qcted| **2 )
MSE (Mean Squared Error) = L2-Loss / N
:
Batch type When weights and bias updates occur
------------------------------------------------------------------------------------------------------
Full batch After the model looks at all the examples in the dataset.
For instance, if a dataset contains 1,000 examples and the model trains for 20 epochs,
the model updates the weights and bias 20 times, once per epoch.
Stochastic After the model looks at a single example from the dataset. For instance,
if a dataset contains 1,000 examples and trains for 20 epochs, the model updates
the weights and bias 20,000 times.
Mini-batch
stochastic After the model looks at the examples in each batch. For instance, if a dataset
contains 1,000 examples, and the batch size is 100, and the model trains for 20 epochs,
the model updates the weights and bias 200 times.