Machine Learning

Machine learning is a way of building software that learns a task from examples, instead of following rules a programmer wrote by hand. It powers spam filters, recommendations and photo search, and the core idea fits in one sentence: show the computer enough labelled examples and it finds the patterns for you.

Why hand-written rules break

Say you want a program that sorts tuna into good and bad. The obvious start is to write the rules yourself:

def is_good(tuna):
    if tuna["shiny"]:
        return True
    if tuna["smelly"]:
        return False
    return True

This works on the first few fish you test. Then a shiny tuna turns up that smells off, and the first rule calls it good. You add a rule for that. Then comes a dull tuna that is perfectly fresh, then a firm one with a faint smell. Every fix adds another branch, the branches start to contradict each other, and soon nobody can say which rule wins.

That is the problem machine learning solves. Some tasks are easy for a person to judge but very hard to write down as exact rules: is this email spam, is this a photo of a cat, will this viewer like this video. When the rules would run to hundreds of special cases, it is often easier to collect examples of the right answer and let the computer work out the rules.

Learning from examples instead

In machine learning you don't write the decision logic. You write a program that learns it from data.

The data is a set of examples. Each example has:

• features: the measurable things about it, such as a tuna's colour, shape and smell, each turned into a number;
• a label: the right answer for that example, 'good' or 'bad'.

A learning algorithm reads the examples and builds a model: a mathematical function that takes features in and gives a prediction out. Once the model is built, you give it a tuna it has never seen and it makes a guess.

So there are two phases:

1. Training: the algorithm studies the labelled examples and adjusts the model until its answers match the labels as closely as it can.
2. Prediction, also called inference: the trained model gets new, unlabelled data and guesses the label.

Training is usually slow and done once, or every so often. Prediction is fast and happens every time someone uses the feature.

How a model learns

Under the hood, a model is a set of numbers, often called parameters or weights, that decide how much each feature counts. Before training they are random or set to a default, so the model's guesses are no better than chance.

Training is a loop. The model guesses an example's label, the guess is compared with the real label, and the numbers are nudged so the same mistake is a little less likely next time. Here is one round of that loop, and the start of the next:

Run that loop over thousands of examples, many times, and the errors shrink. The model ends up weighting the features that best predict freshness, which may turn out to be smell and firmness rather than shine. Nobody told it that: it found the pattern in the data.

This is what the video means by 'math, data and practice'. The maths is the rule for nudging the numbers, the data is the examples, and the practice is the loop, repeated.

A worked example in Python

Python's scikit-learn library ships ready-made learning algorithms, so you can train a small model in a few lines. Each tuna below is three scores from 0 to 10: shine, smell and firmness. Look at the fifth one: shiny, but smelly and soft.

from sklearn.tree import DecisionTreeClassifier
 
# Each tuna: [shine, smell, firmness]
X = [
    [9, 1, 8],
    [8, 2, 7],
    [7, 1, 9],
    [3, 8, 2],
    [9, 7, 3],  # shiny but smelly
    [2, 9, 1],
]
y = ["good", "good", "good", "bad", "bad", "bad"]
 
model = DecisionTreeClassifier(random_state=0)
model.fit(X, y)
 
print(model.predict([[8, 2, 8]]))  # ['good']
print(model.predict([[9, 8, 2]]))  # ['bad']

By convention X holds the features and y the labels. fit is the training phase and predict is the prediction phase. A decision tree learns a short series of yes-or-no questions about the features, picking the questions that best separate the labels. Shine never makes it into those questions, because the shiny bad tuna shows that shine alone can't tell good from bad. So the second tuna, shiny but smelly and soft, comes back as bad: exactly the case that broke the hand-written rules.

Six examples is far too few for a real model, but the code looks the same at any size.

More data, better guesses (usually)

The video's last point is that the more examples a model sees, the better it gets. That holds, with conditions:

• The examples must be right. Wrong labels teach the model wrong answers. 'Garbage in, garbage out' applies to machine learning more than almost anywhere.
• The examples must be varied. If every bad tuna in your data is grey, the model may learn 'grey means bad' and wave through a pink tuna that smells awful. The data has to cover the cases the model will meet.
• The gains slow down. Going from a hundred examples to ten thousand often helps a lot. Adding the same number again on top of millions helps far less.

The way to know whether a model is any good is to test it on data it didn't train on. Before training, set aside part of the labelled examples, often around a fifth, as a test set. Train on the rest, then measure how often the model gets the test set right. A model that scores well on its training data and badly on the test set has overfitted: it memorised the examples instead of learning the pattern, like a student who learnt last year's answers rather than the subject.

Where machine learning is used

The video names a few uses, and they all follow the same recipe of examples, features and a prediction:

• Spam detection: trained on emails people marked as spam or not. The features include the words, the links and the sender. The prediction is spam or not spam.
• Recommendations: trained on what people watched, bought or skipped. The prediction is how likely you are to enjoy the next item. The algorithm picking your next cat video is one of these.
• Image recognition: trained on labelled photos. The prediction is what is in a new photo.

Fraud alerts on bank cards, speech-to-text and delivery time estimates work the same way.

Supervised learning and its neighbours

What the video describes, learning from examples that come with the right answer, is supervised learning, the most common kind. Two others you'll meet:

• Unsupervised learning gets no labels and looks for structure on its own, such as grouping customers who shop alike.
• Reinforcement learning learns by trial and error, earning a reward for good actions, as in programs that learn to play games.

Deep learning is machine learning with large neural networks: models with millions or billions of parameters. Large language models are one example, trained on huge amounts of text to predict the next word, and fine-tuning trains one further on a smaller set of examples. The idea underneath is the same loop as above.

When not to use it

Machine learning is the wrong tool when:

• The rules are clear and stable. Working out VAT or checking a postcode's format needs exact logic, not a guess.
• Every decision must be explained. Many models can't easily say why they made a guess, which is a problem for decisions about loans or medical care.
• You don't have the data. No labelled examples means no supervised model. Collecting and labelling data is often the most expensive part of the whole project.

A good habit is to start with simple rules, measure how often they are wrong, and reach for machine learning only when the rules can't keep up.

Common mistakes

• Testing on the training data flatters the model. Always score it on examples it never saw.
• Trusting the labels blindly lets mistakes in. Check a sample by hand before training.
• Training on data that doesn't match real use: a model trained on studio photos struggles with blurry phone pictures.
• Forgetting that a prediction is only a best guess. Design around the cost of being wrong, such as moving a borderline email to the spam folder rather than deleting it.

Key takeaways

• Machine learning builds a model from examples instead of hand-written rules.
• Training nudges the model's numbers until its guesses match the labels; prediction runs the trained model on new data.
• More good, varied examples usually mean better guesses, but bad data teaches bad habits.
• Always test a model on data it didn't train on.
• It isn't magic: it's maths, data and practice.