Machine learning, in one robot

The forward robot squares a number. The backwards robot figures out that squaring is the rule.

2026-05-17

Here is a robot. It has three parts. An input arm on the left, a rule box in the chest, and an output arm on the right. You feed a number into the input arm, the rule box does something to it, and the result comes out the output arm.

That’s the whole metaphor. The only thing that changes between regular programming and machine learning is whether the rule box is something you wrote, or something an algorithm has to figure out from examples.

INPUT RULE x² OUTPUT
Inspired by Deena Abutaha's 2016 input–rule–output teaching robot. Redrawn here so the rule box can change.

The forward robot — this is programming

Set the rule box to x². Feed in a number. Get a number back. We wrote the rule; the machine executes it. Every program you’ve ever used works this way.

But programming isn’t the only way to fill in that box. Imagine the same robot, with the rule slot sealed. You can see the input arm and the output arm — you just can’t see inside. What you do have is the list of pairs the machine produced.

Programming — rule given, output computed
x rule y
1 x² ???
2 x² ???
3 x² ???
4 x² ???
5 x² ???
Machine learning — output known, rule inferred
x rule y
1 ??? 1
2 ??? 4
3 ??? 9
4 ??? 16
5 ??? 25

The left table is the world of regular programming: we declared the rule, and the machine generated the outputs. The right table is the world of machine learning: inputs and outputs known, the rule unknown. Could the rule be x²? Sure. Could it also be |x|·x or exp(2·ln|x|) — all of which agree on these five points? Also sure. Machine learning doesn’t know the rule. It searches for one.

The backwards robot — this is machine learning

To search for a rule given only inputs and outputs, machine learning needs four ingredients.

  1. 1
    An algorithm

    A family of possible rules to choose from. We have to commit to what kind of thing the rule could be before we can search for the best one. A common starting family is y = a·x² + b·x + c — three knobs (a, b, c) that can be turned to give any parabola, including flat lines and tilted lines.

  2. 2
    A loss function

    A number that says how wrong the current guess is. Standard choice: squared error. For every data point, take the difference between what the algorithm predicts and what the robot actually output, square it, average across all the points. Zero means perfect; bigger means worse.

  3. 3
    An optimizer

    A procedure that nudges the knobs so the loss goes down. The classic is gradient descent: compute which direction to move each knob to lower the loss the fastest, take a small step, recompute, repeat.

  4. 4
    Iteration

    Do it thousands of times.

Watch the rule emerge

The widget below runs that loop. The points are what the robot actually output for y = x² at nine input values. The line is the algorithm's current guess a·x² + b·x + c. We start with the knobs nowhere near right (a = -0.5, b = 1, c = 2) — the initial line is a mess. Press Run and watch the loss drop and the line snap to the parabola.

loss —
Predictions from the Estimated Function
Function Parameter Search
iter a b c loss
target: a = 1, b = 0, c = 0

After a few hundred iterations the optimizer settles on roughly a ≈ 1, b ≈ 0, c ≈ 0. It has discovered, without ever being told, that the rule inside the sealed box is 1·x² + 0·x + 0 — which is just x². The robot didn't get smarter. The loop did.

Choosing the family

The robot above had three knobs — a, b, c — because we committed to a quadratic family. That was a lucky guess: the true rule x² lives inside that family. What if we'd picked wrong?

The widget below lets you change three things at once: the true rule the data was generated from, the degree of the polynomial family the algorithm is allowed to search through, and the noise level. Press Run after each change.

loss —
Predictions from the Estimated Function
Function Parameter Search
iter w₁ w₂ w₃ loss
target
degree 3
noise 0.10

Two configurations are worth trying. (1) Pick the sine target and set degree to 1. The algorithm's family is now lines through the origin — one knob, w₁, the slope. There is no line that approximates a sine wave well; the optimizer settles on a gentle positive slope (around 0.77), the least-bad compromise between the wave's rising and falling halves, and the loss bottoms out far above zero. The algorithm did its job; the family was too narrow. (2) Pick the parabola target, crank noise to 0.40 or so, and set degree to 9. Now the family is wildly flexible — there are many degree-9 polynomials that wiggle through every noisy data point. The optimizer finds one. The fit looks great on the training data and would be terrible on any new point. This is overfitting: the algorithm memorized the noise instead of finding the signal.

Choosing the family is a human decision and a hard one. Too narrow and you can't represent the rule. Too flexible and you find a "rule" that matches the noise in your particular sample and generalizes to nothing.

Why this is the whole story

That loop — guess, measure how wrong, nudge, repeat — is the engine underneath every modern machine learning system. What changes between this toy and an industrial model is the size and shape of the rule box.

A linear regression has a handful of knobs. A gradient-boosted tree has a few thousand. A modern image classifier has tens of millions. A large language model has hundreds of billions. The inputs and outputs get more complex too — pixels in, label out; text-so-far in, next-word-probabilities out — but the four ingredients are exactly the same. Algorithm. Loss. Optimizer. Iteration.

What this means

The honest constraint worth keeping in mind: the loop can only find a rule that lives inside the algorithm’s family. Pick a family that’s too narrow and the rule it finds will be the closest bad approximation. Pick one that’s too flexible and it’ll memorize the training data instead of finding the underlying pattern.

Choosing the family used to feel like the obvious place the “machine learning is automated” story broke down — somebody had to pick. That’s less true now. An LLM can read a problem description and pick a reasonable model, often better than a junior practitioner. But the human choice doesn’t disappear; it migrates. The LLM is picking from a menu humans built, using judgment compressed from a corpus humans curated, shaped by an architecture and objective humans designed. The human role in machine learning doesn’t vanish. It just gets harder to see.