Risk Score for Predicting Adolescent Mental Health Problems Among Children Using Parental Report Only: The TRAILS Study



      To construct a risk score for adolescent mental health problems among children, using parental data only and without potentially stigmatizing mental health items.


      We prospectively derived a prediction model for mental health problems at age 16 using data from parent report on 1676 children aged 11 from the general population. Mental health problems were considered present in the top 15% scores on the combined Achenbach ratings. The model was validated in a separate cohort (n = 336) children. A risk score was constructed for practical application.


      In the derivation cohort, 248 (14.8%) had mental health problems at follow-up. Predictors in the final model were gender, maternal educational level, family history of psychopathology, math achievement at school, frequently moving house, severe disease or death in the family, parental divorce, and child frustration level. The model was well calibrated, showed good discriminatory power (area under the curve 0.75; 95% confidence interval 0.72–0.78), and validated well. The risk score stratified children in classes of risk ranging from 6.6% to 52.2%.


      A risk score based on parent-reported data only and without mental health items accurately estimated the 5-year risk of adolescent mental health problems among children from the general population. Children with high risk may benefit from further monitoring or intervention. The risk score may be particularly suitable when parents want to circumvent an explicit discussion on possible mental health problems of their child.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Merikangas K.R.
        • He J.P.
        • Burstein M.
        • et al.
        Lifetime prevalence of mental disorders in US adolescents: results from the National Comorbidity Survey Replication—Adolescent Supplement (NCS-A).
        J Am Acad Child Adolesc Psychiatry. 2010; 49: 980-989
        • Kessler R.C.
        • Berglund P.
        • Demler O.
        • et al.
        Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication.
        Arch Gen Psychiatry. 2005; 62: 593-602
        • Patel V.
        • Flisher A.J.
        • Hetrick S.
        • McGorry P.
        Mental health of young people: a global public-health challenge.
        Lancet. 2007; 369: 1302-1313
        • Wang P.S.
        • Berglund P.
        • Olfson M.
        • et al.
        Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication.
        Arch Gen Psychiatry. 2005; 62: 603-613
        • Jensen P.S.
        • Goldman E.
        • Offord D.
        • et al.
        Overlooked and underserved: “action signs” for identifying children with unmet mental health needs.
        Pediatrics. 2011; 128: 970-979
        • Toumbourou J.W.
        • Stockwell T.
        • Neighbors C.
        • et al.
        Interventions to reduce harm associated with adolescent substance use.
        Lancet. 2007; 369: 1391-1401
        • Lynch F.L.
        • Hornbrook M.
        • Clarke G.N.
        • et al.
        Cost-effectiveness of an intervention to prevent depression in at-risk teens.
        Arch Gen Psychiatry. 2005; 62: 1241-1248
        • Durlak J.A.
        • Wells A.M.
        Evaluation of indicated preventive intervention (secondary prevention) mental health programs for children and adolescents.
        Am J Community Psychol. 1998; 26: 775-802
        • Cuijpers P.
        • Smit F.
        Subthreshold depression as a risk indicator for major depressive disorder: a systematic review of prospective studies.
        Acta Psychiatr Scand. 2004; 109: 325-331
        • Heflinger C.A.
        • Hinshaw S.P.
        Stigma in child and adolescent mental health services research: understanding professional and institutional stigmatization of youth with mental health problems and their families.
        Adm Policy Ment Health. 2010; 37: 61-70
        • Crone M.R.
        • Bekkema N.
        • Wiefferink C.H.
        • Reijneveld S.A.
        Professional identification of psychosocial problems among children from ethnic minority groups: room for improvement.
        J Pediatr. 2010; 156: 277-284
        • Theunissen M.H.
        • Vogels A.G.
        • Reijneveld S.A.
        Work experience and style explain variation among pediatricians in the detection of children with psychosocial problems.
        Acad Pediatr. 2012; 12: 495-501
        • Achenbach T.M.
        Manual for the Child Behavior Checklist/4-18 and 1991 Profile.
        University of Vermont, Burlington, Vt1991
        • Huisman M.
        • Oldehinkel A.J.
        • de Winter A.
        • et al.
        Cohort profile: the Dutch “TRacking Adolescents’ Individual Lives” Survey—TRAILS.
        Int J Epidemiol. 2008; 37: 1227-1235
        • Achenbach T.M.
        Manual for the Youth Self-Report and 1991 Profile.
        University of Vermont, Burlington, Vt1991
        • Van der Ende J.
        • Verhulst F.C.
        Informant, gender and age differences in ratings of adolescent problem behaviour.
        Eur Child Adolesc Psychiatry. 2005; 14: 117-126
        • Amone-P’Olak K.
        • Ormel J.
        • Oldehinkel A.J.
        • et al.
        Socioeconomic position predicts specialty mental health service use independent of clinical severity: the TRAILS study.
        J Am Acad Child Adolesc Psychiatry. 2010; 49: 647-655
        • Vogels A.G.
        • Crone M.R.
        • Hoekstra F.
        • Reijneveld S.A.
        Comparing three short questionnaires to detect psychosocial dysfunction among primary school children: a randomized method.
        BMC Public Health. 2009; 9: 489
        • Ormel J.
        • Oldehinkel A.J.
        • Ferdinand R.F.
        • et al.
        Internalizing and externalizing problems in adolescence: general and dimension-specific effects of familial loadings and preadolescent temperament traits.
        Psychol Med. 2005; 35: 1825-1835
        • Oldehinkel A.J.
        • Hartman C.A.
        • De Winter A.F.
        • et al.
        Temperament profiles associated with internalizing and externalizing problems in preadolescence.
        Dev Psychopathol. 2004; 16: 421-440
        • Donders A.R.
        • van der Heijden G.J.
        • Stijnen T.
        • Moons K.G.
        Review: a gentle introduction to imputation of missing values.
        J Clin Epidemiol. 2006; 59: 1087-1091
        • Schafer J.L.
        Multiple imputation: a primer.
        Stat Methods Med Res. 1999; 8: 3-15
        • Marshall A.
        • Altman D.G.
        • Holder R.L.
        • Royston P.
        Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines.
        BMC Med Res Methodol. 2009; 9: 57
        • Royston P.
        • Moons K.G.
        • Altman D.G.
        • Vergouwe Y.
        Prognosis and prognostic research: developing a prognostic model.
        BMJ. 2009; 338: b604
        • Harrell Jr., F.E.
        • Lee K.L.
        • Mark D.B.
        Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
        Stat Med. 1996; 15: 361-387
        • Steyerberg E.W.
        • Eijkemans M.J.
        • Habbema J.D.
        Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis.
        J Clin Epidemiol. 1999; 52: 935-942
        • Chatfield C.
        Model uncertainty, data mining, and statistical inference.
        R Stat Soc Ser A. 1995; 158: 419-466
        • Hanley J.A.
        • McNeil B.J.
        The meaning and use of the area under a receiver operating characteristic (ROC) curve.
        Radiology. 1982; 143: 29-36
        • Altman D.G.
        • Vergouwe Y.
        • Royston P.
        • Moons K.G.
        Prognosis and prognostic research: validating a prognostic model.
        BMJ. 2009; 338: b605
        • Steyerberg E.W.
        • Borsboom G.J.
        • van Houwelingen H.C.
        • et al.
        Validation and updating of predictive logistic regression models: a study on sample size and shrinkage.
        Stat Med. 2004; 23: 2567-2586
        • Maguire J.L.
        • Kulik D.M.
        • Laupacis A.
        • et al.
        Clinical prediction rules for children: a systematic review.
        Pediatrics. 2011; 128: e666-e677
        • Goodman R.
        The Strengths and Difficulties Questionnaire: a research note.
        J Child Psychol Psychiatry. 1997; 38: 581-586
        • Jellinek M.S.
        • Murphy J.M.
        • Little M.
        • et al.
        Use of the Pediatric Symptom Checklist to screen for psychosocial problems in pediatric primary care: a national feasibility study.
        Arch Pediatr Adolesc Med. 1999; 153: 254-260
        • Jaspers M.
        • De Winter A.F.
        • de Meer G.
        • et al.
        Early findings of preventive child healthcare professionals predict psychosocial problems in preadolescence: the TRAILS study.
        J Pediatr. 2010; 157: 316-321
        • Harland P.
        • Reijneveld S.A.
        • Brugman E.
        • et al.
        Family factors and life events as risk factors for behavioural and emotional problems in children.
        Eur Child Adolesc Psychiatry. 2002; 11: 176-184
        • Hill J.
        Early identification of individuals at risk for antisocial personality disorder.
        Br J Psychiatry. 2003; 44: S11-S14
        • Burt K.B.
        • Hay D.F.
        • Pawlby S.
        • et al.
        The prediction of disruptive behaviour disorders in an urban community sample: the contribution of person-centred analyses.
        J Child Psychol Psychiatry. 2004; 45: 1159-1170
        • Pitzer M.
        • Jennen-Steinmetz C.
        • Esser G.
        • et al.
        Prediction of preadolescent depressive symptoms from child temperament, maternal distress, and gender: results of a prospective, longitudinal study.
        J Dev Behav Pediatr. 2011; 32: 18-26
        • Ragan P.V.
        • McGlashan T.H.
        Childhood parental death and adult psychopathology.
        Am J Psychiatry. 1986; 143: 153-157
        • Luecken L.J.
        Attachment and loss experiences during childhood are associated with adult hostility, depression, and social support.
        J Psychosom Res. 2000; 49: 85-91
        • Maccabe J.H.
        Population-based cohort studies on premorbid cognitive function in schizophrenia.
        Epidemiol Rev. 2008; 30: 77-83