Observational Studies Overestimate

A common claim is that “observational studies consistently over-estimate” and typically that the are more “favourable”.

This seems to be propagated by both EBM proponents and has been mentioned by some philosophers (at least once in JW’s reply to Urbach on randomisation).

It seems to be put forward as an argument for randomisation.

Unless we are taking about publication bias (which I don’t think the people making the statement are {[green right — Jason ]}) I don’t see how this claim can be substantiated—outside of the comparison between RCT estimates and the estimates of observational studies (which, given the topic under discussion is typically what justifies randomisation, is viciously circular).

I know there is some empirical data which refutes this, but I need to find the reference. (not that it needs empirical refutation)

Need to compile a list of who makes the claim:

Hayes 2002: non-random allocation usually results in more optimistic differences between intervention and control than does random allocation—the claim is based on Schulz JAMA 1995;273:408-12

{[green Your opponents deny that their point of view is viciously circular, but I agree with you that it is. This is mentioned in Mackenzie and Grossman. There may be a useful citation or two there. Jason ]}