Mini Review
A Note on Treatment Effect Estimation in Randomized and Observational Clinical Trails
Ying Wang2, Ao Yuan1* and Ming T Tan3
Department of Biostatistics, Bioinformatics and Biomathematics, Georgetown University, USA
Ao Yuan, Department of Biostatistics, Bioinformatics and Biomathematics, Georgetown University, USA
Received Date: July 30, 2025; Published Date: August 19, 2026
Abstract
The randomized clinical trial is used as golden standard, and various classical methods can be used to estimate the treatment effect. When the clinical trial is not randomized it is known that the naive estimate (group sample mean difference) of treatment effect is biased, and causal inference methods is needed to get unbiased estimate. However, in this case it is still not clear for other non-causal classical estimates, such as least squares estimate, regression model (linear or non-linear) estimate, can they implement the treatment effect and give unbiased estimates? Here we look into this question by reasoning and simulation studies. Our conclusions are provided at the end in terms of linear and nonlinear models.
Keywords:Randomized clinical trial (RCT); Inverse propensity scores weighted estimate (IPSWE); Doubly robust estimator (DRE); Naive estimate (NE); Regression coefficient estimate (RCE)
Introduction
The Randomized Clinical Trial (RCT) is used as golden standard, and various classical methods can be used to estimate the treatment effect in this case [1-4]. However, in practice, RCT has its drawbacks, such as time consuming and high costs [5], conflict of interest dangers [6], and ethics and feasibility [7]. Also, sometimes randomization is not possible or difficult to implement strictly. In these cases of observational studies, it is known that the naive estimate (group sample mean difference) of treatment effect is biased [8], and causal inference methods is needed to get unbiased estimate, such as the recently commonly used doubly robust estimator [9-13]. However, in this case it is still not clear for other non-causal classical estimates, such as least squares estimate, regression model (linear or non-linear) estimate, can they implement the treatment effect and give unbiased estimates? Here we look into this question by reasoning and simulation studies.
The Question and Method
Consider the observed data
iid (Y, X , A), i y is response, xi is covariate, and ai is binary
treatment assignment, ai =1/ 0 for treatment/control. The interest
is to estimate

where Y(1) is the potential outcome under treatment, and Y(0) is that under control? It is known that when the treatment assignment is not random or the covariates are not balanced among the treatment groups, or the , ai, s depend on the , xi, s, then


are biased estimates of μ1 and μ0 [8], so is the estimate of mean treatment effect difference

based on the above two estimates, and so causal inference methods are needed to get unbiased estimates.
Now consider the following regression model, for some given link function g (⋅) ,

The question of interest is: in randomized or observational clinical trial, can we get unbiased estimate of Δ using the above model? Our
understanding is that for randomized clinical trial model (1) can be the correct specification and it can be used to estimate Δ unbiasedly.
For observational study, model (1) is incorrect, and the estimate of Δ based on it is biased. In this case, they
xi, s are not iid over n, so the
consistency of
is questionable. In fact, this conclusion applies also to any other data in which some components of the covariates
are not iid.
Commonly Used Estimation Method
Below we list some commonly used estimation method of Δ for randomized and observational clinical trials. The naive and regression coefficient method are valid only for randomized trial. The causal inference methods will provide unbiased estimates of Δ in either randomized or observational study. Here we consider five of them.
Naive Estimator (NE). It just uses the sample mean difference

Regression Coefficient Estimator (RCE): It just uses the estimated regression coefficient
of δ in model (1), i.e., set
.
Inverse Propensity Scores Weighted Estimate (IPSWE) proposed by Horvitz and Thompson (1952) [8]. In this method, it requires a
propensity score model

After obtaining the estimate
, the estimate
is constructed as

Outcome Model Estimator (OME). This estimator is also called g-estimator in Robins (2009) [9]. It requires the outcome model

After obtaining estimate
the estimator
is

Doubly Robust Estimator (DRE), The DRE is a recent development in causal inference. It requires a propensity score model (2) and
outcome model (4). After obtaining estimate
, the DRE
is

The estimator is unbiased if either the propensity score model or the outcome model is correctly specified.
Simulation Studies
Here we simulate data from models (1) and (4), and compute the Naive Estimate (NE), and the Regression Coefficient Estimate (RCE), and the causal estimates IPSWE (3), OME (5) and the DRE (6), and compare the results in corresponding Tables. We consider two cases: case 1), g (⋅) , g1 (⋅ ) and g0(⋅ ) all be identity function, so models (1) and (4) are linear models; case 2), g (⋅) , g1(⋅ ) and g0( ⋅ ) are non-linear link functions. The simulation distinguishes for randomized observational studies.
Simulation Results for Randomized Study
In the randomized study, we considered three different covariate dimensions: p = 3 , p = 5 , and p = 8 . For all three settings, the total sample size was fixed at n = 300 , with n1 =10 0treated subjects and n0 = 200 control subjects. We consider two cases, linear model, g (⋅) be identity, and non-linear model, g (⋅) is non-identity. The treatment assignment was completely randomized, so the average proportion of treated subjects was approximately 0.333 in every setting.
Linear Model: The response variable was generated from the linear model

with Δ0 = 2.0 and error standard deviation 0.5. The true regression coefficients were taken as β0 =( 1.0, − 1.5,0.8)T for the 3-dimensional setting; β0 =(1.0,− 1.5,0.8,1.2, − 0.7 )T for the 5-dimensional setting; and β0 =(1.0,− 1.5,0.8,1.2, − 0.7 )T for the 8-dimensional setting.
For each setting, the simulation was repeated 400 times. Table 1 reports the average estimate and the standard deviation for the five estimators. From Table 1, all five estimators produce estimates very close to the treatment effect Δ0 = 2.0 in all three dimensions. This is expected because the treatment assignment is randomized, so there is no confounding bias. However, there is a clear difference in variability. The Naive estimator has notice-ably larger standard deviations, and its variability increases as the dimension grows. In contrast, RCE, IPSWE, OME, and DRE remain more stable because they ad-just for covariates. Among these adjusted methods, OME, DRE, and RCE give the smallest standard deviations across all three settings, while IPSWE is also close to unbiased but slightly less stable. Overall, the randomized study shows that all five methods are approximately unbiased, but the adjusted estimators, especially OME and DRE, are more efficient than the Naive estimator.
Table 1:Estimation of Δ0 with five methods in the randomized study.

Non-linear Model. In this part, we continue the randomized simulation study by taking the link function as a nonlinear distribution function g(.). The response variable was generated from model (1):

where
. The treatment assignment was randomized, with n = 300 , n 1 =100 , and n0 = 200 . Thus, the treatment indicator
Ai was independent of the covariates Xi .
The index-scale treatment coefficient was set to be δ =1.000 . Since the link function is nonlinear, the treatment effect on the response mean scale is not equal to δ. For each simulated dataset, the response-scale treatment effect was calculated as

The true regression coefficient vectors were taken as β0 =(0.2,1.0,− 1.5,0.8 )T for the 3-dimensional setting; β0 =(1.0,− 1.5,0.8,1.2, − 0.7 )T for the 5-dimensional setting; and
β0 =(0.2,1.0, 1.5,0.8,1.2,− 0.7,0.9,− 1.1 for the 8-dimensional setting.
For each setting, the simulation was repeated 400 times. Table 2 reports the simulation results. In this table, the Naive, IPSWE, OME, and DRE estimators estimate the response-scale treatment effect Δ0 . However, the RCE estimates the index-scale coefficient η directly. Therefore, the RCE column should be compared with η =1.000 , while the other methods should be compared with the response-scale Δ0 . From Table 2, the Naive estimator is close to the response-scale treatment effect Δ0 in all three dimensions. This is reasonable because the data come from a randomized clinical trial, so the treatment assignment is independent of the covariates. The IPSWE estimator gives almost the same results as the Naive estimator because the propensity score is known and constant in this randomized setting.
Table 2:Estimation results with nonlinear g (u) in the randomized study.

The RCE results need to be interpreted differently. In the nonlinear model

the regression coefficient δ ≠ Δ . Therefore, RCE is estimating η, not the response-scale treatment effect Δ0 . The simulation results show that the RCE estimates are very close to η =1.000 in the 3-, 5-, and 8-dimensional settings.
The OME and DRE estimators also estimate the response-scale treatment effect Δ0 , and their estimates are close to the corresponding values. They have smaller standard deviations than the Naive and IPSWE estimators because they use the correct outcome model with the known link function g (⋅) . Overall, in this randomized setting, the Naive estimator works well for the response-scale treatment effect, while RCE should be reported and interpreted as an index-scale estimator.
Simulation Results for Observational Study
In the observational study, we again considered three different covariate dimensions: p = 3 , p = 5 , and p = 8 , with total sample size n = 300 . In this case, the treatment assignment was no longer randomized. Instead, the treatment indicator i A was generated from a propensity score model

where
. The true γ was chosen so that the treated proportion was not close to one half. The average proportions of treated
subjects were approximately 0.296, 0.284, and 0.270 for the 3-, 5-, and 8-dimensional settings, respectively
Linear Model: The outcome model followed the separate regression structure

The error standard deviation was 0.5, and the true intercepts were 0 α0 = 0.5 and α1 = 2.0 , so that the treatment effect was Δ0 =α1 −α0 =1.5.
For the 3-, 5-, and 8-dimensional settings, the true parameters were chosen as follows:

For each setting, the simulation was repeated 400 times. Table 3 reports the average estimate and the standard deviation for the five estimators. From Table 3, the NE is strongly biased in all three observational settings. The RCE is also biased upward, although the bias is smaller than that of the NE because the regression model adjusts for covariates. This shows that when treatment assignment depends on covariates, methods that do not properly account for confounding can give misleading results. By contrast, IPSWE, OME, and DRE all produce estimates much closer to the target value. Among these three methods, OME and DRE are more stable, with substantially smaller standard deviations than IPSWE. Although IPSWE is approximately unbiased, its variability is considerably larger, especially in the 3- and 8-dimensional settings. This is consistent with the general feature of inverse weighting methods, which often suffer from high variance. Overall, the observational study supports the main conclusion of this project. In the presence of covariate-dependent treatment assignment, the NE and RCE are not reliable, whereas IPSWE, OME, and DRE perform much better. Among them, OME and DRE provide the best balance between accuracy and stability.
Table 3:Estimation of Δ0 with five methods in the observational study.

Nonlinear Model
In this part, we consider the observational simulation study with nonlinear out-come models. Different from the previous nonlinear randomized setting, the data in this part were generated from model (4), not from model (1). That is, the treatment and control groups were generated from two separate outcome models,

where
. The treatment assignment was no longer randomized. Instead, i A was generated from the propensity score model

Therefore, the treatment assignment depended on the covariates, and the treated and control groups were not balanced.
For the nonlinear outcome models, g0(⋅ ) and g1( ⋅)were chosen as two different distribution functions. The parameters were chosen so that μ1 > μ0 . The true treatment effect was calculated by formula (5),

Here

For the 3-, 5-, and 8-dimensional settings, the true parameters were chosen as follows:

Since this setting is based on the separate nonlinear outcome models in formula (4), there is no index-scale treatment coefficient δ . Therefore, the RCE is not reported in this table. We compare four estimators: NE, IPSWE, OME, and DRE. For each setting, the simulation was repeated 400 times. Table 4 reports the average estimate and the standard deviation for each estimator. From Table 4, the NE is clearly biased in the observational nonlinear setting. For example, when the dimension is 3, the true treatment effect is Δ0 = 0.260 , but the NE estimate is 0.520. This happens because the treatment assignment depends on the covariates, so the treated and control groups have different covariate distributions. As a result, the simple sample mean difference does not estimate the causal treatment effect. By contrast, IPSWE, OME, and DRE are much closer to the true treatment effect Δ0 . The OME performs well because the outcome model is correctly specified. The DRE also performs well because it uses both the propensity score model and the outcome models. The IPSWE is also close to the target value, although it has larger standard deviations than OME and DRE. Overall, this observational nonlinear study shows that causal adjustment is necessary when the treatment assignment depends on covariates, while the NE is not reliable.
Table 4:Estimation of Δ0 with nonlinear outcome models in the observational study

Concluding Remarks
We examined commonly used regression models and estimates for treatment effect under the randomized clinical trial and observational study, and used simulation studies to confirm our reasonings. Our conclusions are: regression models are generally different for randomized clinical trial and observational study; for the five estimators we examined, some of them can be used for both trials, some are not, which are summarized below.
• For randomized trial with linear model
, all the five estimators give consistent estimate of Δ .
• For randomized trial with non-linear model
, all the estimators except RCE give consistent
estimate of Δ . The RCE estimator
is not an estimator of Δ in this case since typically δ ≠ Δ in non-linear model.
• For observational trial with linear model
and
, the NE and RCE are biased estimates of Δ , and the IPSWE, OME and DRE are unbiased
estimators of Δ.
• For observational trial with non-linear model
and
the NE is a biased estimate of Δ , and the IPSWE, OME and DRE are unbiased estimators
of Δ . In this setting, the RCE is not reported because model (4) does not contain a single index-scale treatment coefficient δ .
References
- Coover JE, Angell F (1907) General practice effect of special exercise. The American Journal of Psychology 18(3): 328-340.
- Neyman J (1923) [1990]. On the application of probability theory to agricultural experiments. Essay on Principles. Section 9. Statistical Science 5(4): 465-472.
- Chalmers TC, Smith H, Blackburn B, Silverman B, Schroeder B, et al. (1981) A method for assessing the quality of a randomized control trial. Controlled Clinical Trials 2(1): 31-49.
- Lachin JM (1988) Statistical properties of randomization in clinical trials. Controlled Clinical Trials, 9(4): 289-311.
- Sanson-Fisher RW, Bonevski B, Green LW, D’Este C (2007) Limitations of the randomized controlled trial in evaluating population-based health interventions. American Journal of Preventive Medicine 33(2): 155-161.
- Bekelman JE, Li Y, Gross CP (2003) Scope and impact of financial conflicts of interest in biomedical research: a systematic review. Journal of American Medical Association 289(4): 454-465.
- Black N (1996) Why we need observational studies to evaluate the effectiveness of health care. British Medical Journal 312(7040): 1215-1218.
- Horvitz DG, Thompson DJ (1952) A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association 47: 663-685.
- Robins J, Rotnitzky A, Zhao L (1994) Estimation of regression coefficients when some of the regressors are not always observed. Journal of the American Statistical Association 89(427): 846-866.
- Rotnitzky A, Lei QH, Sued M, Robins JM (2012) Improved double-robust estimation in missing data and causal inference models. Biometrika 99(2): 439-456.
- Yuan A, Yin A, Tan MT (2021) Enhanced doubly robust procedure for causal inference. Statistics in Biosciences 13(3): 454-478.
- Yin A, Yuan A, Tan MT (2024a) Doubly robust semiparametric estimation for multi-group causal comparisons. Statistics in Biosciences 16: 45-68.
- Yin A, Yuan A, Tan MT (2024b) Highly robust causal semiparametric U-statistic with applications in biomedical studies. International Journal of Bio-statistics 20(1): 69-91.
-
Ying Wang, Ao Yuan* and Ming T Tan. A Note on Treatment Effect Estimation in Randomized and Observational Clinical Trails .Annal Biostat & Biomed Appli. 7(4): 2026. ABBA.MS.ID.000661
Randomized clinical trial (RCT); Inverse propensity scores weighted estimate (IPSWE); Doubly robust estimator (DRE); Naive estimate (NE); Regression coefficient estimate (RCE); iris publishers; iris publisher’s group
-

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.






