What Is Science? It’s Not the Tools You Use

The AI industry has a naming problem.

We often use scientist or researcher to describe anyone who trains a model, evaluates a model, or writes PyTorch code. The tools are sophisticated, the work is technical, and the job titles sound impressive. But none of those things, by themselves, make the work science.

This confusion matters. When we mistake the use of scientific tools for science itself, we undervalue engineering, misunderstand research, and hire for the wrong skills.

A researcher is not a higher-ranking engineer

In a widely discussed post on X, Elon Musk rejected the distinction between “researchers” and “engineers” at xAI, arguing that it disguised a two-tier engineering system.

I agree with part of his diagnosis: researcher should not be an honorific for an engineer who works with models. Research and engineering are different kinds of work, not different ranks in a hierarchy.

But I would not conclude that there are only engineers. Research is real and necessary. We simply need a better way to recognize it.

The distinction is not whether someone writes code, builds models, or works in academia. It is whether the work applies existing knowledge or produces new knowledge.

Model building is not automatically science

Consider a question I once received in an applied scientist interview: How would you handle imbalanced classes in a binary classification problem?

There are several reasonable answers. You might reweight the loss, resample the data, adjust the decision threshold, choose metrics that reflect the real cost of errors, or use a combination of these approaches. The right choice depends on the problem.

It is a useful interview question for testing practical machine-learning judgment. But it is not, by itself, a scientific question. The problem is well known, the standard approaches are documented, and an AI system can summarize them in seconds.

Implementing one of those approaches reliably may still be difficult and valuable. The solution may need to run quickly, securely, efficiently, and at scale. It may need monitoring, fault tolerance, and integration with a much larger system. That is serious engineering—not second-class science.

The same distinction applies to many activities labeled “AI research.” Reproducing a published architecture, tuning hyperparameters, running a standard benchmark, or calling a model API can all be useful work. They become scientific only when they are part of a rigorous attempt to answer something that is not already known.

Two tests for scientific work

To me, scientific work has two essential parts.

1. Ask an important question whose answer is genuinely unknown

Not every unanswered question is a research question. I may not know where I left my keys, but finding them does not expand meaningful knowledge.

A scientific question addresses an uncertainty worth resolving. Its answer should teach us something beyond the immediate task—about a phenomenon, a system, a population, or a class of problems. In applied science, that knowledge may be grounded in a particular organization or product, but the reasoning and evidence should still offer lessons that can transfer beyond a single anecdote.

Good research begins before the experiment. It requires identifying the unknown, explaining why it matters, and framing the question so that evidence can actually answer it.

2. Use a method that can produce a trustworthy answer

An interesting question is not enough. Science also requires a disciplined way of learning from reality.

That means defining terms, forming testable hypotheses, choosing meaningful measurements, considering alternative explanations, quantifying uncertainty, and documenting the work well enough for others to scrutinize or reproduce it. It also means being willing to discover that the original hypothesis was wrong.

Code can support this process, but code is not the process. A model can be an experimental instrument, but the model is not the scientific contribution. The contribution is the new, defensible knowledge produced with it.

A scientific AI question that may not require building a model

Here is an example: How much value does a deployed AI system create for an organization?

That sounds simple, but there is no universal answer and no reliable one-size-fits-all playbook. Does the system save time, or merely move work from one part of a process to another? Does it increase output while reducing quality? Who benefits? Do the gains persist after the novelty wears off? What would have happened without the system?

Answering those questions requires more than a dashboard. A researcher might need to define productivity carefully, establish a credible baseline or counterfactual, separate correlation from causation, measure downstream effects, account for differences in adoption, and test whether the findings hold across teams or over time.

That is science, even if no new model is trained. The object of study is not the model itself but the effect the model has in the world.

Conversely, training a model with an established recipe may be engineering even when the implementation is technically demanding. The tool does not determine the category of work. The question and the method do.

Why the distinction matters

Clearer language helps everyone.

If a team needs someone to productionize a known technique, it should hire and reward excellent engineers. If it needs someone to determine what works, why it works, and under what conditions, it should hire researchers who can frame questions and design credible studies. Many people can do both, and many projects require both, but the capabilities are not interchangeable.

Treating every model builder as a scientist inflates titles while shrinking our idea of science. It reduces research to a set of technical operations and overlooks its hardest part: confronting the unknown in a way that produces evidence others can trust.

Matt Might’s wonderful Illustrated Guide to a Ph.D. depicts human knowledge as a circle. Years of study take a researcher to its boundary; original research makes a tiny dent in that boundary.

That image captures science better than any job title can. Science is not knowing how to use the fashionable tool of the moment. It is finding an important edge of what we know—and pushing it outward.