Background: Clustered randomized controlled trials are commonly used to evaluate the effectiveness of treatments. Frequently, stratified or paired designs are adopted in practice. Fogarty (2018) studied variance estimators for stratified and not clustered experiments and Schochet et. al. (2022) studied that for stratified, clustered RCTs with large strata. Objective: In stratified and clustered randomized controlled trials, the Hájek estimator (Hájek 1971) can be computed as an estimation of the average treatment effect, and by using inverse probability weighting, which accounts for the potential bias of treatment effect estimation caused by different probabilities of treatment assignment. We aim to construct variance estimators for the Hajek estimator and propose inference methods in clustered randomized experiments. Theoretical Findings: We follow the finite population framework and show that the difference of Hájek estimators for stratified and/or clustered randomized experiments is asymptotically jointly normal. We derive the variance covariance matrix and propose asymptotically conservative sandwich variance estimations of the difference of the Hájek estimators under two common forms of stratification, large stratum designs and designs with many small strata (e.g., paired designs discussed in Pashley & Miratrix (2021)). Importantly, the proposed variance estimator can be applied to designs involving one or more strata that only have a single observation under one or both treatment arms. We also propose methods for hypothesis testing and constructing confidence intervals of the average causal effect. Data Analysis: We evaluate our methods with an application to a real-world field trial in children's health and nutrition. Giles et. al. (2012) conducted a matched-pair cRCT to evaluate the effects of an intervention designed to encourage water consumption among children in afterschool programs in Boston. The intervention, Out of School Nutrition and Physical Activity (OSNAP), involves environmental and policy changes aimed at improving nutrition practices and physical activity. We analyze the data using the proposed method and find that the intervention was associated with a statistically significant positive effect in promoting children's water drinking. Conclusions: Our work provides a valuable perspective that leads to an estimator of the variance that does not depend on the number of small strata. The variance estimator demonstrates effectiveness for inference even for a small sample size. Our proposed method is a good complement to the current methodologies on analyzing stratified and clustered experiments.