Asking a question in a lecture

Lecturer: Right, that's everything for today. Any questions before we finish?

You: Yes, sorry, I had a question about something you mentioned earlier.

Lecturer: Sure, go ahead.

You: When you were talking about regularisation, you said it helps with overfitting. I didn't quite follow why adding a penalty makes the model generalise better.

Lecturer: Good question. So basically, the penalty stops the model relying too heavily on any single feature. It forces it to keep things simpler.

You: Okay. So it's a bit like making the model less confident about the training data?

Lecturer: That's a nice way of putting it, yes. It trades a bit of training accuracy for better performance on new data.

You: Right, that makes more sense now. Could I ask a quick follow-up?

Lecturer: Of course.

You: How do you decide how strong the penalty should be? Is there a rule of thumb?

Lecturer: Not really, it's usually tuned with cross-validation. We'll cover that properly in week five.

You: Great, thank you.

Lecturer: And the slides for that are already on Moodle, if anyone wants to read ahead.