I am not following your mixture model idea. For every data point you know if it comes from the RCT or observational study. You don't need uncertainty about treatment assignment. What you need is figuring out how to massage observational data to get causal conclusions (e.g. what I think about all day long).

If you have specific observational data you want to look at, email me if you want to chat more.


what I think about all day long

You specialise in identifying the determinants of biases in causal inference? Just curious :) Interesting

2gwern5yNo, the uncertainty here isn't about which of the two studies a datapoint came from, but about whether (for a specific treatment/intervention) the correlational study datapoint was drawn from the same distribution as the randomized study datapoint or a different one, and (over all treatments/interventions) what the probability of being drawn from the same distribution is. Maybe it'll be a little clearer if I narrate how the model might go. So say you start off with a prior probability of 50-50 about which group a result is drawn from, a switching probability that will be tweaked as you look at data. (If you are studying turtles which could be from a large or a small species, then if you find 2 larger turtles and 8 smaller, you're probably going to update from P=0.5 to a mixture probability more like P>0.20, since it's most likely - but not certain - that 1 or 2 of the larger turtles came from the large species and the 8 smaller ones came from the small species.) For your first datapoint, you have a pair of results: xyzcillin reduces all-cause mortality to RR=0.5 from a correlational study (cohort, cross-sectional, case-control, whatever), and the randomized study of xyzcillin has RR=1.1. What does this mean? Now, of course you know that 0.5 is the correlational result and 1.1 is the randomized result, but we can imagine two relatively distinct scenarios here: 'xyzcillin actually works but the causal effect is really more like RR=0.7 and the randomized trial was underpowered', or, 'xyzcillin has no causal effect whatsoever on mortality and it's just a bunch of powerful confounds producing results like RR=0.6-0.8'. We observe that 1.1 supports the latter more, and we update towards 'xyzcillin has 0 effect' and now give 'non-causal scenarios are 55% likely', but not too much because the xyzcillin studies were small and underpowered and so they don't support the latter scenario that much. Then for the next datapoint, 'abcmycin reduces lung cancer', we get a pair look

Open thread, Dec. 21 - Dec. 27, 2015

by MrMind 1 min read21st Dec 2015233 comments


If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.