Use Cross Entropy typically
Two problems
Often do both regularization and early stopping to counteract overfitting
ROC curve - Choose the decision threshold based on decision criteria
AUC - Provides total measure of performance across all possible classification thresholds