The largest problem, from a logical standpoint, is that experiments are not always possible. In many cases, we are limited to observational studies. We have to observe the variables as they occur naturally, outside our control, and try to find the relevant correlations.

Under the right circumstances, an observational study can provide evidence of causality. However, it has two major drawbacks in relation to an experiment.

First, it does not involve random assignment to experimental and control groups. So there's a danger that any correlation we find between two variables may be due to some third variable which is not evenly distributed among the groups.

Second, an experiment allows us to control one variable and look for changes in another. If we find them, there is no question which is the cause, which the effect. This question, though, does arise in observational studies. There is always the possibility of confounding variables.