Article Text

Download PDFPDF

Do data from child protective services and the police enhance modelling of perinatal risk for paediatric abusive head trauma? A retrospective case-control study
  1. Patrick Kelly1,2,
  2. John M D Thompson3,
  3. Santuri Rungan4,
  4. Shanthi Ameratunga5,
  5. Timothy Jelleyman6,
  6. Teuila Percival2,7,
  7. Hinemoa Elder8,
  8. Edwin A Mitchell9
  1. 1 Te Puaruruhau, Starship Children’s Health, Auckland, New Zealand
  2. 2 Paediatrics: Child and Youth Health, University of Auckland Faculty of Medical and Health Sciences, Auckland, New Zealand
  3. 3 Paediatrics: Child and Youth Health, University of Auckland, Auckland, New Zealand
  4. 4 Community Child Health, Sydney Children’s Hospitals Network Randwick and Westmead, Sydney, New South Wales, Australia
  5. 5 School of Population Health, University of Auckland, Auckland, New Zealand
  6. 6 Department of Paediatrics, Waitemata District Health Board, Takapuna, New Zealand
  7. 7 Kidz First Children’s Hospital, Counties Manukau District Health Board, Auckland, New Zealand
  8. 8 School of Graduate Studies, Te Whare Wānanga o Awanuiārangi, Auckland, New Zealand
  9. 9 Paediatrics, University of Auckland, Auckland, New Zealand
  1. Correspondence to Dr Patrick Kelly; patrickk{at}adhb.govt.nz

Abstract

Objectives First, to investigate whether there is a relationship between a family being known to child protective services or police at the time of birth and the risk of abusive head trauma (AHT, formerly known as shaken baby syndrome). Second, to investigate whether data from child protective services or police improve a predictive risk model derived from health records.

Design Retrospective case control study of child protective service and police records.

Setting Nine maternity hospitals.

Participants 142 consecutive cases of AHT admitted to a tertiary children’s hospital from 1991 to 2010 and born in one of the nine participating maternity hospitals. 550 controls matched by the date and hospital of birth.

Outcome measure Abusive head trauma.

Results There is a relationship between families known to child protective services or police and the risk of AHT. Notification to child protective services: univariable OR 7.24 (95% CI 4.70 to 11.14). Involvement with youth justice: univariable OR 8.94 (95% CI 4.71 to 16.95). Police call-out for partner violence: univariable OR 3.85 (95% CI 2.51 to 5.91). Other violence offence: univariable OR 2.73 (95% CI 1.69 to 4.40). Drug offence: univariable OR 2.82 (95% CI 1.63 to 4.89). However, in multi-variable analysis with data from perinatal health records, notification to child protective services was the only one of these variables to remain in the final model (OR 4.84; 95% CI 2.61 to 8.97) and had little effect on overall predictive power. The area under the receiver operating characteristic curve was 89.5% (95% CI 86.6 to 92.5) using variables from health data alone and 90.9% (95% CI 88.0 to 93.7) when notification was added.

Conclusions Family involvement with child protective services or police is associated with increased risk of AHT. However, accessing such data at the time of birth would add little predictive power to a risk model derived from routine health information.

  • child abuse
  • shaken baby syndrome
  • intimate partner violence
  • perinatal care
  • primary prevention

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • This study is a case-control study examining risk factors for paediatric abusive head trauma, using data collected well before the outcome of interest took place.

  • This study examined data from multiple sources, matching police and child protective service records for families with perinatal health records for new-born babies.

  • This was a retrospective study, so it was not possible to control the quality and consistency of data collection across health, police and child protective services.

Introduction

Paediatric abusive head trauma (AHT), formerly known as shaken baby syndrome, is an inflicted injury to the skull or intracranial contents of a young child.1 2 It is a significant cause of death and disability1 3 with major long-term social and economic consequences.4 5

It would clearly be best, if possible, to take steps to prevent AHT before it occurs. Because AHT often occurs in response to crying, current prevention strategies focus on teaching all new parents about the dangers of shaking and how to cope with a crying baby.6 7

However, it is likely that age and crying are not the only risk factors for AHT. Studies identify a variety of risk factors for other forms of child abuse, some of which have also been identified in cohort studies of AHT.8 It seems reasonable to suggest that there may be circumstances in which the risk of AHT is increased. If those could be identified, there may be benefit in interventions targeted at those circumstances and/or at specific families where such circumstances exist.

Targeted interventions are common in prevention strategies for other forms of child abuse. One example is regular home visits in early childhood (‘home visiting’),9 where families qualify for visits after a risk assessment often including criteria such as a history of child abuse, intimate partner violence (IPV), substance abuse or criminal justice involvement.10 11 Recently, New Zealand economists used public benefit and child protection records for 57 986 children to develop a general predictive risk model for child abuse. The outcome variable was defined as a ‘substantiated report of maltreatment by the age of 5 years’. The authors suggested this model could be used to target home visiting at those most likely to benefit. Predictor variables included (among many others) ‘child protection service reports for other children’, ‘substantiated physical or sexual abuse before age 16 years’, ‘partner has criminal record’, ‘police family violence reports’ and ‘youth justice referrals for partner before age 16 years.”12

In a recent multi-centre case-control study, we used variables from routinely collected perinatal health records in an attempt to construct a model that could predict the risk of AHT. However, those records contained little or no information on the possible risk factors outlined above.

The purpose of this study was, therefore, to obtain data from sources outside the health system which respond to child abuse and adult criminality, to investigate whether there is a relationship between those data and the risk of AHT and to determine whether incorporating such data would improve the ability of primary healthcare providers to assess risk in the perinatal period. Because AHT has a median age at diagnosis of 5 months,13 it seemed appropriate to focus our investigation on information which might be obtainable at or before the time of birth.

Methods

Setting

New Zealand has a population of 4.7 million with approximately 60 000 births per year. Starship Children’s Hospital is in the largest city, Auckland (population 1.4 million). The incidence of AHT is estimated at 34/100 000 births.13 14

Study design and population

This study was developed from a retrospective case-control study where data were obtained from perinatal records. That study is described in detail elsewhere.8 Briefly, cases were admitted to Starship Children’s Hospital from 1991 to 2010 and met four criteria: (1) age <2 years; (2) intracranial injury and/or skull fracture; (3) AHT diagnosed through a rigorous multi-disciplinary process and reported to statutory authorities13; (4) born in one of nine participating maternity hospitals in the North Island of New Zealand.8 The study population included these cases and four controls for each case randomly selected from babies born on the same day in the same maternity hospital. In 2016, controls were checked with the Ministry of Health and statutory authorities to confirm that they did not sustain AHT up to the age of 5 years.8

In the current study we obtained data from the Birth Certificate, Oranga Tamariki—Ministry for Children (the statutory child protective services agency) and police. Each agency serves the entire country and records their data in a national electronic database. Because these databases are not normally accessed by health professionals, we were unsure what data they could provide. Early in study design, investigators conferred with experienced statutory social workers and police officers to ensure we understood how their data were collected and structured. Decisions were then made as to which variables could be extracted from each database.

Patient involvement statement

Patients were not involved in the development of the research question, the design of the study or in recruitment to or conduct of the study.

Data collection

In 2011 and 2012, health data were collected retrospectively by study investigators directly from maternal and child perinatal records.8 Data were collected for this study in 2013 by a separate investigator blinded to perinatal data and to the case or control status of the baby. The same sources of data and methods of assessment were used for both cases and controls.

The name and date of birth of the mother and baby were provided to Births, Deaths and Marriages, who returned the name and date of birth of the father from the Birth Certificate.

One investigator (SR) was trained by Oranga Tamariki and given access to their record system. Using the names and dates of birth of the mother, father and baby the investigator searched for any data recorded prior to the baby’s birth. Data were collected on the mother, the father, the mother’s partner(s), the father’s partner(s), siblings and step-siblings. Data included whether they ever had a notification to the statutory child protective services agency, an investigation or a determination by a statutory social worker that abuse had occurred (‘substantiation’). Data on the type of abuse exist only for substantiations. These were collected, as well as data on the type of statutory response (eg, custody or guardianship) and adolescents referred to the youth justice system (juvenile delinquency). We searched for concerns about alcohol, drugs, IPV or mental health. However, these were documented so rarely that they were discarded from analysis. Data included a count. For example, we recorded not only notification, but how many notifications.

Police were provided with the names and dates of birth of the mother, father, baby and specified individuals associated with them in child protective service records: mother’s partner(s), father’s partner(s), siblings and step-siblings. Police searched their database for data related to IPV (a call-out to the home, charges or convictions); charges or convictions for other violence, alcohol or drugs; and mental health concerns. Mental health concerns were documented so rarely that they were discarded from analysis. Data included a count. For example, we recorded not only call-outs for IPV, but how many call-outs.

For both databases, data were searched for as far back in time as they could be found. It therefore included data dating back to the birth of the parents, if electronically recorded.

Data from Births, Deaths and Marriages were recorded electronically and transferred into Microsoft Excel. Data from child protective services and police were recorded on separate proformas for each baby and entered in Microsoft Excel by two investigators.

Statistical analysis

For variables from child protective services or the police analysed in this study, there were no identifiable missing data. Subjects were either in their database, or they were not.

Data were analysed in SAS software (SAS V.9.4, Cary, North Carolina, USA). Data were first tested for difference in frequency between cases and controls in simple cross-tabulations and logistic regressions, not accounting for the matched design. The χ2 test was used for categorical data and t-tests for continuous data. Variables with very low frequency in the dataset were removed. Conditional logistic regression estimated univariable ORs and 95% CIs for variables of interest. This is a logistic regression in which each case was matched with their specified controls by date and hospital of birth.

Variables significant at p<0.05 on univariable analysis were included in multivariable analysis of two groups (child protective services, police). Each group was reduced stepwise, backwards and forwards, to ensure consistency and examine how variables influenced each other.

Variables which remained significant in each group at p<0.05 were retained, combined and reduced stepwise, backwards and forwards.

Variables which remained significant at p<0.05 were added to the existing model derived from perinatal health records. Variables which remained significant at p<0.05 were retained.

The case control design enabled us to control for potential confounders such as age and community characteristics. However, matching in the design can introduce confounding in the analysis.15 We analysed the data using both matched (conditional) and unmatched (unconditional) logistic regression. Results consistent across both methods are more likely to be robust. Both are provided in the tables so readers can judge for themselves. Also, it is useful to describe the performance of a predictive model by the area under the Receiver Operator Characteristic (ROC) curve. The ROC plots sensitivity against specificity across the entire distribution of the two populations (cases and controls). The area under the curve (AUC) is one measure of how well the model distinguishes between the two populations. However, a ROC can only be determined from the results of unconditional logistic regression.

The proportion of the total variability of the outcome (AHT) that could be accounted for by our model was assessed using a pseudo R-squared statistic (Nagelkerke’s R 2).

Results

There were 142 cases of AHT and 550 controls. Participant details are provided elsewhere.8 The results for variables from the Birth Certificate, child protective services and the police included in logistic regression are presented in table 1.

Table 1

Variables included in logistic regression (conditional and unconditional analysis)

Data from health records

The previous study found that nine variables from health records were significantly associated with the risk of AHT: maternal age, maternal ethnicity, ‘other social history’ (a catch-all for any explicit documented concern about the social circumstances), partner status, whether the mother took supplements during pregnancy, ‘unknowns’ (missing data concerning booking for antenatal care, other social history, partner status and substance abuse), prolonged rupture of membranes (for more than 48 hours before delivery), gestational age and type of feeding at hospital discharge. Those variables are discussed in detail elsewhere.8

Data from the Birth Certificate

As previously described,8 the mean age of mothers of cases was 4 years less than controls and each year of age reduced the risk of AHT (table 1).

The mean age of fathers of cases was 5 years less than controls and they were less likely to appear on the Birth Certificate. However, both variables were accounted for in group multivariable analysis by controlling for maternal age.

Data from child protective services

Fourteen variables were associated with the risk of AHT on univariable analysis (table 1). Counting the number of events had no effect on significance, except for notifications to child protective services. As the number of notifications increased, so did the risk of AHT. Using no notification as the reference, the OR for AHT of one notification was 3.32 (95% CI 1.20 to 9.23) and for two or more notifications was 5.55 (95% CI 2.79 to 11.03).

In multivariable analysis of these 14 variables from child protective services, six variables remained significant: involvement of the father or mother in youth justice; notification to child protective services; substantiation and three subtypes of substantiated abuse (neglect, sexual abuse, behaviour or relationship difficulties—but not physical abuse).

Data from the police

Five variables were associated with the risk of AHT on univariable analysis (table 1). Counting the number of events (eg, the number of call-outs for IPV) did not substantially alter their significance. In multivariable analysis of these five variables from the police, one remained significant: call-out to the home for IPV.

Combined analysis of child protective services and police data

In combined multivariable analysis of the seven significant variables from both groups, two remained significant: involvement in youth justice and notification to child protective services.

Final model

When these two variables were incorporated into the existing model, only one remained significant: notification to child protective services. All previous perinatal variables remained significant, with very little change in point estimates.

The final model therefore included ten variables which remained significant whether analysed conditionally or unconditionally (table 2).15

Table 2

Variables remaining in the final model (conditional and unconditional analysis)

The AUC of the model derived from health data alone was 89.5% (95% CI 86.6 to 92.5).8 The AUC of the new model was 90.9% (95% CI 88.0 to 93.7)—effectively the same. Similarly, Nagelkerke’s R 2 remained much the same. For the model derived from health data alone it was 33.1% (for both conditional and unconditional regression), and for the new model it was 34.5% (conditional) or 35.5% (unconditional).

Discussion

In this study, although we confirmed that there is a relationship between data known to child protective services or police at the time of birth and the risk of AHT, our principal finding was that accessing such data at the time of birth would add little predictive power to a risk model derived from routine health information.

The strengths of this case-control study are that it used a robust method for studying a rare condition and a comprehensive approach to data collection and analysis, incorporating data from multiple sources which have not previously been combined in the investigation of AHT. There are several limitations. First, it was retrospective, so we had no control over the quality of the data. Second, data from child protective services and police may not be sensitive indicators of risk. For example, one finding of our original study was the risk associated with formula feeding in the first week of life (table 2). We hypothesised that this might reflect confounding factors which hinder breastfeeding such as lack of support,16 history of abuse as a child or IPV.8 17 Although data from child protective services and police provided no evidence to support these hypotheses, neither could they disprove them. It is well recognised that most child abuse is not reported to child protective services and police.18 19 Similarly, although one New Zealand study using information collected by health researchers described an association between IPV during pregnancy and AHT,20 it is not unexpected that we were unable to replicate that finding. Only 24% of IPV in New Zealand is reported to police,21 a finding consistent with low rates of reporting of IPV to police in all jurisdictions studied across the world.22 Finally, although our model can discriminate between two populations (as suggested by the AUC), it explains only one third of the variation between those populations (as suggested by the pseudo R2 statistic). Statistical associations between AHT and a variable do not necessarily mean that the variable will identify individuals more or less likely to experience AHT. Factors associated with very high ORs may still turn out to be unhelpful as individual-level predictors.23 Our current model is not good enough to guide clinical practice or policy. Neither the model, nor any of the variables it contains, can be used to guide clinical interventions in specific families. More work is needed to replicate our findings, investigate other potentially relevant variables and examine possible confounders which may underlie or explain some of the variables in our model.

Despite these limitations, our data are consistent with the literature in one important respect: the mere fact of notification to child protective services is an indicator of risk, regardless of the outcome.24 25 Notification is an action with many possible outcomes: no further action (in New Zealand this is the outcome for approximately 50% of notifications and is a decision often taken at the national call centre without further assessment),26 investigation (a highly variable process in which a social worker determines whether abuse has occurred, a positive determination being known as ‘substantiation’)27 and a decision whether a child is in need of care and protection.28 If a child is identified to be in need of care and protection, there may be formal proceedings specified in legislation: Family Group Conference or court orders.29 We found no evidence that any of these responses had any significant additional effect. Notification itself was the principal indicator of risk. However, although the OR was high by traditional epidemiological standards, it is not surprising that it added little to the predictive value of our model. Because of the limitations mentioned above, a variable with an apparently strong independent association with the outcome (estimated by OR) will often not contribute meaningfully to predictive accuracy.23

Our findings stand in contrast to the research mentioned in our introduction, which argued for the value of risk modelling for child abuse using data from the public benefit system and child protective services.12 Despite access to large amounts of data, that research had serious limitations. These included the assumption that ‘substantiation’ is a valid outcome variable; the risk of bias inherent in the exclusion of families outside the public benefit system; the potential for breach of privacy and stigmatisation without evidence for benefit; and the possibility of unintended consequences if their model was used to allocate interventions by influencing or overriding frontline clinicians.27 30 In addition, their final model included 132 separate variables, many of which did not differ significantly between cases and controls. Our study used a much more tightly defined outcome variable, excluded no sector of the population, was more parsimonious (achieving a higher AUC with fewer variables) and ended by reinforcing the value of data already routinely collected by health professionals. The interpretation of such data by health professionals to guide health interventions would involve no breach of privacy.

One key finding of our earlier study was a strong association between AHT and ‘unknowns’: missing perinatal health data on ‘booking’ (whether the mother registered for antenatal care), other social history (any recorded concern about the social situation), partner status (married, de facto or single) and substance abuse (defined as engagement with alcohol and drug services).8 In our current study, child protective services and police data had no effect on the strength of this association, suggesting no direct relationship between these ‘unknowns’ and families known to child protective services or the police. The absence of such a relationship may suggest that these families at risk are overlooked by both health professionals and statutory authorities. The hypotheses put forward in our earlier paper (minimal or fragmented antenatal care, poor relationships with healthcare providers or reluctance to ask difficult questions of high-risk families)31–33 deserve further research.

This study tested the hypothesis that combining information from perinatal health records with information from child protective services and the police would enhance the ability of primary healthcare providers to predict (and therefore possibly prevent) AHT. We found little evidence to support that hypothesis. We suggest that our findings will be generalisable to other countries. Our method was robust, many variables in our model are consistent with other literature on risk factors for AHT,8 and the limitations of police and child protective services data (described above) are well-recognised internationally.18 19 22 With respect to AHT, we suggest that if the quality and consistency of perinatal healthcare could be improved, it is the health system which may hold the key to identifying families most likely to benefit from early intervention. Such improvements could include: routine and universal enquiry during pregnancy for matters of possible relevance such as alcohol and drug abuse, IPV, unplanned pregnancy and untreated mental illness; routine and universal follow-up of families with missing data or poor engagement with antenatal care; and routine and universal access to evidence-based early intervention programmes when matters of concern are identified by health providers.

Acknowledgments

The authors thank Oranga Tamariki, the New Zealand Police and the Registrar of Births, Deaths and Marriages, who allowed us to access their data. We thank Detective Sergeant Neil Holden, Officer Commanding, Auckland City Police Child Protection Team and Craig Newby, Auckland District Health Board Oranga Tamariki Liaison Practice Leader, for their assistance with defining the data to be obtained, trialling the time required for this research and training Dr Rungan in searching the Oranga Tamariki database. We thank Jess Wilson for her assistance with statistical analysis.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.

Footnotes

  • Contributors PK and EAM conceived the study. SR and PK collected and entered the data. JMDT, PK and EAM analysed the data. PK drafted the manuscript. All the authors contributed to the study design (PK, EAM, SR, JMDT, SA, TJ, TP and HE), acquisition of data (SR, PK) or analysis and interpretation of data (JMDT, PK, EAM and SA); revised the manuscript critically (PK, EAM, JMDT, SA, TJ, HE, TP and SR) and approved the version to be published (PK, EAM, JMDT, SA, TJ, HE, TP and SR).

  • Funding This work was supported by the Starship Foundation which funded the employment of Dr Rungan as a Child Protection Research Fellow and by Cure Kids which supported (in part) Professor Edwin Mitchell and Associate Professor John Thompson.

  • Competing interests None declared.

  • Ethics approval Approval was gained from the New Zealand Health and Disability Ethics Committee (13/NTA/51), the Oranga Tamariki Research Access Committee, the Police Research and Evaluation Steering Committee, the Registrar-General of Births, Deaths and Marriages and the Privacy Commissioner.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Anonymised data may be shared upon request from the corresponding author (ORCID 0000-0002-8813-8877). A data sharing agreement will be generated and all data will be anonymised prior to sharing.

  • Patient consent for publication Not required.