External Validation of the Assessment of Different NEoplasias in the adneXa Model Performance in Evaluating the Risk of Ovarian Carcinoma Before Surgery in China: A Tertiary Center Study

Ovarian cancer (OC) is the fourth most common cancer diagnosed in women worldwide.¹ Globally, 313,959 new OC cases and 207,252 deaths were reported in 2020.² The incidence rate in China increases with age after the age of 40 years and reaches its peak at the age of 55 to 59 years.³ Various studies indicated that most women with ovarian carcinoma were diagnosed with advanced disease, associated with a poor prognosis and a low 5-year survival rate of <50%. In comparison, the 5-year survival rate of patients with an early diagnosis of a localized disease could reach 90%.^{4, 5} A key factor in optimizing survival is the combination of early diagnosis and centralized management.⁶ Previous screening trials for OC were unsuccessful in achieving an early diagnosis.⁷ Several studies suggested that a first-line test to screen for OC had a limited role.⁸ However, the risk of OC algorithm, using a series of serum CA125 measurements, achieved higher detection rates with stage-dependent changes in detected cancers than the UK Collaborative Trial of Ovarian Cancer Screening, which used a CA125 cut-off value.^{9, 10} Moreover, recent reports have shown that a multi-marker longitudinal model for the early detection of OC was significantly better than CA125 alone.¹¹

A crucial aspect of managing clinical situations is to make accurate diagnoses. It is vital that patients are referred to a specialist hospital for immediate treatment when they present with an abnormal ovarian mass and need specialist oncology services for OC.

In 2014, the International Ovarian Tumor Analysis (IOTA) team generated a new risk prediction model named the Assessment of Different NEoplasia in the adneXa (ADNEX).¹² It comprises six ultrasonographic and three clinical indicators. The most significant advantage of the ADNEX model is it being the first multi-category model for adnexal masses. It can assess the overall risk of OC and evaluate each subtype. The experienced (level III) ultrasonographers who collected data for the model are equivalent to the level of a consultant with a special interest in gynecological ultrasonography in the UK.¹² Additionally, the models and rules developed and validated by the IOTA group have been validated by less experienced (level II) ultrasonographers.^{13, 14} The correct classification of malignant tumour subtypes is crucial, as treatment differs between subtypes. It is essential to preserve fertility during surgery, especially in young women with borderline ovarian tumors (BOTs), whereas conventional lymphadenectomy is not recommended.¹⁵ Furthermore, treatment of metastatic OC should be based on the primary tumour.¹⁶

This study aimed to externally validate the ADNEX model performance in a tertiary center in China. The better diagnostic performance of the models allows clinicians to select the best treatment options for their patients, thus contributing to better patient prognosis.

Materials and Methods

ADNEX Model

The ADNEX model is freely accessible online at https://www.iotagroup.org/adnexmodel/ and can be downloaded for use in portable applications. Three clinical and 6 ultrasonographic predictors were included in the ADNEX model: age (years), serum CA125 level (U/mL), type of center (an oncology center or other hospital type), maximal diameter of the lesion (mm), the proportion of solid tissue, over 10 cyst locules (yes or no), number of papillary projections (0, 1, 2, 3, and >3), acoustic shadows (yes or no), and ascites (yes or no; Figure 1).¹² The proportion of solid tissue was defined as the ratio of the largest diameter of the maximum solid element to the largest diameter of the lesion. The oncology center was described as a “tertiary referral centre with a dedicated unit for gynecological oncology.”¹² This model could also be used without the serum CA125 level. We calculated the precision performances of the ADNEX model with the serum CA125 level included or excluded.

With all predictive factors entered, the model outcome indicated the probability of a malignant tumour, presented graphically and numerically. Further, the risk of a borderline neoplasm, stages I and II to IV ovarian invasive neoplasm, or metastatic cancer was investigated. We followed the model as described in the initially published version,¹² with no alterations to the equations or coefficients. We also obtained risk estimates using the online calculator on the IOTA website mentioned above.

Results

The level I and III examiners initially retrieved 335 consecutive patients with ultrasonographic adnexal masses managed by surgery. Based on the inclusion and exclusion criteria, 53 patients were excluded for the following reasons: 20 had no histology result (14 with only cytology, 6 without cytology and histology result); 10 underwent surgery >60 days after the ultrasonographic examination; 5 refused to undergo a transvaginal scan (only transabdominal scans were available), 2 were pregnant, and 16 did not undergo surgery because of a decline in the physical condition they could not sustain surgery (n = 2), refusal of surgery for personal reasons (n = 8), or postponement of surgery due to neoadjuvant chemotherapy (n = 10). Therefore, 282 patients were finally included in the analysis (Figure 2). Among them, approximately 100 patients were assessed by a level I operator, and the rest were assessed by a level III operator. Among them, serum CA125 level was missing in 6 (2%). Of the 282 patients, 178 (63.1%) had benign ovarian tumors, and 104 (36.9%) had malignant tumors. The most common benign diagnoses were teratoma (19.5%, 55/282) and endometriomas (11.3%, 32/282). The malignant lesions included 24 (8.5%) BOT, 27 (9.6%) stage I cancer, 46 (16.3%) stages II to IV cancer, and 7 (2.5%) secondary metastatic cancer (Table 1). Age was significantly higher in patients with malignant tumors than in those with benign tumors (P < .001). The median age was 45 years, with 181 (64.2%) premenopausal and 101 (35.8%) postmenopausal women. The serum CA125 level in postmenopausal women was higher than in premenopausal women (Z = −2.814, P = .005), with the ultrasonographic findings showing the former to have a larger malignant tumor diameter, broader solid tissue component, more papillary projections, and ascites (P < .05). Patients with a malignant tumor had almost no acoustic shadow. Table 2 shows the clinical characteristics and ultrasonographic features of the patients.

The concordance between the total predicted risk of malignancy by the ADNEX model with and without CA125 and the observed proportion of malignant tumors is shown in the calibration plots (Figure 3).

The AUC for differentiating between benign and malignant masses by the ADNEX model with CA125 was 0.93 (95% CI, 0.90–0.96) and 0.91 (95% CI, 0.88–0.95) when CA125 was excluded (Figure 4), demonstrating a slightly better performance for the model that included the CA125 (AUC difference: 0.02, 95% CI: 0.002–0.04). Table 3 presents the model performance with risk cut-off points set from 1 to 30%, with and without CA125. The sensitivities of ADNEX models that included CA125 were 99, 95, and 88% at risk cut-off values of 1, 10, and 30%, respectively. The respective specificities were 24, 58, and 82%. Postmenopausal women had a larger AUC than premenopausal women: 0.94 (95% CI, 0.87–0.98) versus 0.91 (95% CI, 0.86–0.95) with CA125 and 0.90 (95% CI, 0.83–0.95) versus 0.88 (95% CI, 0.83–0.93) without.

The ADNEX model displayed good discrimination between the tumor subtypes when categorized as benign, borderline, stage I invasive, stages II to IV invasive, and secondary metastatic tumors (Table 4). For instance, near-perfect differentiation between benign and stage II to IV tumors was achieved in models that included CA125 (AUC, 0.99; 95% CI, 0.98–1.00). In contrast, performance was worse in differentiating borderline from phase I ovarian carcinoma (AUC, 0.53; 95% CI, 0.37–0.69). Compared to other groups, the ADNEX model without CA125 mainly had a lower AUC for stage II to IV tumors, particularly when compared to secondary metastatic cancers. The AUC for the model with CA125 was 0.82 (95% CI, 0.71–0.93), and that without CA125 was 0.51 (95% CI, 0.31–0.72).

Discussion

Our study suggests that level I and III sonographers may help differentiate benign and malignant masses at presentation, achieving levels very similar to those obtained by the experienced sonographers in the original validation study by the IOTA group.¹² The model sensitivity and specificity in our external validation study for defining malignancy with a 10% cut-off were 95.2% (95% CI, 89.1–98.4) and 57.9% (95% CI, 50.3–65.2), respectively, compared to 96.5 and 71.3% in the original study.¹² It can be seen that the sensitivity was comparable to that of the original study; however, the specificity was lower than that of the primary research. When the cut-off value was 30%, the sensitivity and specificity were 87.5 and 82.0%, respectively. In contrast, a lower false-positive rate, and a much higher cut-off value for malignancy (30%) may be more appropriate at a tertiary center.²⁶ Similarly, for some centers, a higher cut-off value (eg, 30%) was preferred, and they considered that having a high specificity may be more critical. This limits the number of false-positive results, where patients with benign lesions are referred to oncologists. Contrastingly, various hospitals tend to prefer lower cut-off values (eg, 5 or 10%), and having high sensitivity is probably the most crucial, which restricts the number of patients with false negatives. Thus, the optimal cut-off value might vary depending on the center type as different countries have different medical care systems, protocols for referrals, and clinical features of the patients.

It was impossible for us to determine the best cut-off value for conservative treatment as only patients treated with surgery were included in this study. In our study, based on the ultrasound diagnosis and the patient's symptoms, the sonographers recommend surgery or conservative treatment. However, ultimately, it is the treating clinician, together with the patient, who decides on the treatment strategy. Therefore, the recommended management and the actual may differ. Recent studies have been shown to evaluate all patients with adnexal masses, regardless of whether they were treated surgically or conservatively. The results suggest that the ADNEX model with or without CA125 is the best model for differentiating between benign and malignant adnexal masses.²⁷ In our study, a total of 32 patients received advice on conservative treatment, and we will continue to follow up and pay attention to the changes in the condition of these patients. Therefore, in the future, we will carry out the long-term follow-up of conservatively treated patients to validate the model in a larger patient population to better distinguish between benign and malignant tumors. Moreover, we admit that this study was challenging, because there may be no information for patients who have been followed up, and various factors such as surgery during the follow-up period will be excluded.

This is an external ADNEX model validation study for OC in a tertiary center in China. Furthermore, the validation was performed by levels I and III ultrasonographers, whereas previous reports collected ultrasonographic scan data from experienced level II or III examiners.^{12, 13} Several limitations of this study could be summarised as follows. First, this was a single-center study. Such a design might lead to possible sampling variations between centres and limit the applicability of the outcomes for other fields. Second, this study included several patients with missing CA125 values. We used a single random interpolation to deal with the missing values, recognizing that these were not necessarily the true value. However, the 24 participating centers in the original ADNEX model study had malignancy prevalence between 0 and 66%,¹² indicating that the results are likely to be scalable. Furthermore, models are expected to perform differently in different centers and countries. Nevertheless, it should be noted that this study had some strengths. The reference diagnoses were robustly selected, including only cases with histological features. However, this may also be viewed as potentially flawed as the option to treat the mass conservatively was not included in the study. The serum CA125 was measured in all patients using the same assay kit, thus avoiding potential bias. Our study relied on a centralized histopathology examination to avoid possible bias, especially in distinguishing borderline from benign or stage I tumors, which is an ongoing challenge for pathologists.

The levels of ultrasonography experience in this study (I and III) were categorization based on recommendations made by the EFSUMB.²⁰ These findings were supported by the Royal College of Radiologists.²⁸ It is worth noting that it is difficult to define precisely the levels, which might overlap at the boundaries. Therefore, we acknowledge that this approach had some limitations.

An individualized diagnosis of ovarian masses is possible with the ADNEX model by describing the type of malignancy in detail. The proper differentiation between benign and malignant adnexal masses is one of the most critical steps in choosing the appropriate treatment. Presurgical detailed diagnosis with the ADNEX model could assist in selecting the most appropriate surgical approach (laparoscopic or cesarean) or in focusing directly on the original site of the malignancy if metastases occur. The ADNEX model could be used to calculate relative risk ratios for comparison with the background risk of individual patients.²⁹

Conclusions

To conclude, this study showed that the ADNEX model was the highly accurate performing model in discriminating, calibrating, and clinical applicability. External validation is a key step in the development of any diagnostic model. Undoubtedly, the ADNEX model could potentially change management decisions for patients with an adnexal mass and is a valuable tool for clinical applications. Further studies on the characteristics of the IOTA ADNEX model in a broader range of patients managed conservatively are needed.