Phenotypic divergence between individuals with self-reported autistic traits and clinically ascertained autism

Sarah M. Banker, Miles Harrington, Matthew Schafer, Soojung Na, Matthew Heflin, Sarah Barkley, Jadyn Trayvick, Arabella W. Peters, Abigaël A. Thinakaran, Daniela Schiller, Jennifer H. Foss-Feig, Xiaosi Gu
Nature Mental Health
Icahn School of Medicine at Mount Sinai, New York, NY, USA

Table of Contents

Overall Summary

Study Background and Main Findings

This study investigated the differences between clinically-ascertained individuals with ASD and individuals recruited online based on self-reported autistic traits. Key findings include a lack of significant relationship between self-reported and clinician-rated autistic traits in the ASD group (b = 0.025, P = 0.251), and differences in social behavior, with the ASD group showing reduced ability to exert social influence in the social controllability task and less affiliative behavior in the social navigation task. The online high-trait group reported significantly higher levels of social anxiety (F(2,163) = 59.80, P = 3.33 x 10^-20) and AVPD symptoms (F(2,163) = 107.84, P = 1.46 x 10^-30) compared to both the ASD and low-trait groups.

Research Impact and Future Directions

The study provides valuable insights into the differences between clinically-ascertained individuals with ASD and individuals recruited online based on self-reported autistic traits. It clearly demonstrates that while self-reported traits can be informative, they do not always align with clinician-administered assessments and may not accurately reflect observable social behavior. The study makes a strong case for caution when using online self-report measures for autism research, particularly when drawing conclusions about the ASD population as a whole.

The study's findings have practical implications for both research and clinical practice. For researchers, it highlights the need to carefully consider the limitations of online recruitment and self-report measures, particularly in the context of ASD. It suggests that online studies should be used in conjunction with, rather than as a replacement for, traditional lab-based research. For clinicians, the study reinforces the importance of using a multi-faceted approach to assessment, incorporating both self-report and observational measures, and being aware that high self-reported autistic traits may be indicative of other conditions, such as social anxiety or AVPD.

While the study provides valuable guidance, it also acknowledges key uncertainties. The reliance on a single self-report measure (BAPQ) and the lack of a direct measure of insight are limitations that need to be addressed in future research. The study also suggests that future research should explore the use of additional trait measures and self-reported diagnoses in online studies, as well as investigate potential platform differences in social profiles.

Critical unanswered questions remain, such as the extent to which the findings generalize to other populations and the specific mechanisms underlying the observed discrepancies between self-report and behavior. The methodological limitations, particularly the reliance on a single self-report measure and the lack of a direct measure of insight, do not fundamentally undermine the study's conclusions, but they do highlight the need for further research to replicate and extend these findings. The high range of IQ scores in the in-person sample is another limitation that affects the generalizability of the conclusions.

Critical Analysis and Recommendations

Clear Research Question (written-content)
The abstract clearly states the research question, comparing in-person recruited, clinically-assessed individuals with autism to online-recruited individuals. This is important because it immediately informs the reader of the study's focus and target populations.
Section: Abstract
Problem Statement: Self-Report Limitations (written-content)
The introduction effectively introduces the core problem of relying on self-report measures for ASD research in online settings. This is crucial because it highlights the potential for misrepresentation and the need for validation of online methods.
Section: Introduction
Key Finding: Discrepancy Between Self-Report and Clinician Ratings (written-content)
The Results section clearly reports the lack of a significant relationship between self-reported ASD traits (BAPQ) and clinician-rated traits (ADOS) within the in-person ASD group (b = 0.025, P = 0.251). This is a key finding because it challenges the assumption of agreement between these measures and suggests they may capture different aspects of ASD.
Section: Results
Clear Presentation of Social Controllability Results (written-content)
The Results section effectively presents the results of the social controllability task, showing that the ASD group rejected a smaller percentage of high offers in the controllable condition and perceived less control. This is important because it indicates a reduced ability to exert social influence, a key aspect of social interaction.
Section: Results
Consideration of Social Anxiety and AVPD (written-content)
The Discussion acknowledges the potential role of social anxiety and AVPD in the online high-trait group. This is important because it considers alternative explanations for the observed differences and suggests that self-reported autistic traits in the general population may reflect generalized social avoidance rather than autism-specific difficulties.
Section: Discussion
Detailed Participant Recruitment Description (written-content)
The Methods section clearly describes the participant recruitment process for both online and in-person samples, providing detailed eligibility criteria. This is crucial for transparency and reproducibility of the study.
Section: Methods
Add Subheadings for Clarity (written-content)
The Results section lacks subheadings, making it harder to follow the flow of the findings. Adding subheadings would improve the clarity and organization of the Results section, making it easier for readers to navigate the different types of results.
Section: Results
Balance Discussion of Online Research Limitations and Benefits (written-content)
The Discussion does not fully balance the discussion of online research limitations and benefits. Including a more explicit acknowledgment of the potential benefits of online research would provide a more nuanced perspective.
Section: Discussion

Section Analysis

Abstract

Key Aspects

Strengths

Suggestions for Improvement

Introduction

Key Aspects

Strengths

Suggestions for Improvement

Results

Key Aspects

Strengths

Suggestions for Improvement

Non-Text Elements

Table 1 | Group demographic information
Figure/Table Image (Page 3)
Table 1 | Group demographic information
First Reference in Text
See Table 1 for demographic characteristics of each group.
Description
  • Overview: Table 1 presents the demographic information for three groups of participants: a 'Clinical ASD' group, a 'High-trait' group, and a 'Low-trait' group. These groups are compared across several demographic variables to assess the differences in their group compositions.
  • Sample Sizes: The table includes the sample size ('n') for each group, which is 56 for each of the three groups (Clinical ASD (n=56), High-trait (n=56), Low-trait (n=56)).
  • Age Statistics: Age is reported as the mean (average) along with the standard deviation (a measure of the spread or variability) for each group. For example, the Clinical ASD group has a mean age of 28.07 years with a standard deviation of 8.53 years. The other groups similarly report their mean age and standard deviation.
  • Gender Breakdown: Gender is reported as the percentage of women, men, non-binary individuals, and those who did not report their gender in each group. For instance, in the Clinical ASD group, 30.3% are women, 51.8% are men, and 14.3% are non-binary.
  • Sex Breakdown: Sex, reported as the percentage of female and male participants, is also included. For example, in the Clinical ASD group, 51.8% are female and 48.2% are male.
  • Ethnicity: Ethnicity is described using percentages for different ethnic groups such as American Indian or Alaska Native, Asian, Black, White, and Other. For example, in the Clinical ASD group, 0% are American Indian or Alaska Native, 10.7% are Asian, 19.7% are Black, 57.1% are White, and 12.5% are Other.
  • IQ and Cognitive Ability: The table also includes the mean and standard deviation for IQ (Intelligence Quotient) for the Clinical ASD group (Mean: 112.38, Standard Deviation: 16.26) and a measure of Cognitive ability for the High-trait and Low-trait groups (Mean: 7.40, Standard Deviation: 3.42 and Mean: 7.02, Standard Deviation: 3.53, respectively). IQ scores are designed to represent a person's reasoning and problem-solving abilities, with the average score set at 100 and the standard deviation at 15.
  • Employment Status: Employment status is reported as the percentage of employed and unemployed individuals, with some participants not reporting their employment status. For example, in the Clinical ASD group, 48.2% are employed and 51.8% are unemployed.
  • Household Income: Household income is categorized into income brackets such as 10-50k, 50-100k, and >100k, with percentages reported for each group. For example, in the Clinical ASD group, 42.9% have a household income between 10-50k.
  • Education Level: Education level is reported as the percentage of individuals with different levels of education such as graduate school, college, some college, high school, some high school, and no high school. For example, in the Clinical ASD group, 17.9% have a graduate school education.
  • Statistical Significance: The table includes a column labeled 'Group difference,' indicating whether there were statistically significant differences between the groups for each demographic variable. The statistical significance is determined using Kruskal-Wallis tests (for continuous variables) and Chi-squared tests (for categorical variables). A p-value is reported to indicate the likelihood that the observed difference occurred by chance, where a p-value less than 0.05 is commonly considered statistically significant.
Scientific Validity
  • Essential Demographic Information: The table provides essential demographic information, which is crucial for understanding the composition of each study group and assessing the generalizability of the findings. The range of demographic variables is comprehensive and relevant.
  • Appropriate Statistical Tests: The use of appropriate statistical tests (Kruskal-Wallis and Chi-squared) for different types of variables is methodologically sound. However, the specific post-hoc tests used to determine pairwise group differences following a significant omnibus test are not explicitly stated, which should be clarified.
  • Lack of Effect Sizes: The inclusion of p-values allows the reader to quickly identify statistically significant group differences. However, reporting effect sizes alongside p-values would provide a more complete picture of the magnitude of these differences.
  • Inconsistent Cognitive Measures: While the table provides IQ scores for the Clinical ASD group and cognitive ability scores for the other groups, these measures are not directly comparable. Ideally, the same cognitive assessment should be used across all groups to allow for more meaningful comparisons. If this was not possible, justification for the use of different measures should be provided.
Communication
  • Clear Reference: The reference to Table 1 in the Results section is clear and appropriately placed, guiding the reader to the relevant information about the participant groups.
  • Potential for Overwhelm: The table provides a comprehensive overview of the demographic characteristics, but the sheer volume of information might overwhelm some readers. Strategic use of bolding or shading could highlight key differences between groups.
  • Lack of Visual Emphasis on Significance: While the table includes statistical test results (p-values), more visual cues could enhance the immediate understanding of significant group differences. For example, using asterisks to denote statistical significance directly in the table would be beneficial.
Fig. 1 | Trait comparisons. a, The ASD (n = 56 participants) and high-trait...
Full Caption

Fig. 1 | Trait comparisons. a, The ASD (n = 56 participants) and high-trait (HT; n = 56 participants) groups had comparable levels of self-reported autistic traits (measured via BAPQ; two-sided pairwise comparisons using estimated marginal means, with confidence intervals and P values adjusted for multiple comparisons using the Tukey method: t(111) = -0.28, P=0.957, estimated difference = -0.026, 95% CI [-0.25, 0.19], Cohen's d = -0.05; mean ASD: 3.82, mean HT: 3.85, mean low-trait (LT): 2.11).

Figure/Table Image (Page 4)
Fig. 1 | Trait comparisons. a, The ASD (n = 56 participants) and high-trait (HT; n = 56 participants) groups had comparable levels of self-reported autistic traits (measured via BAPQ; two-sided pairwise comparisons using estimated marginal means, with confidence intervals and P values adjusted for multiple comparisons using the Tukey method: t(111) = -0.28, P=0.957, estimated difference = -0.026, 95% CI [-0.25, 0.19], Cohen's d = -0.05; mean ASD: 3.82, mean HT: 3.85, mean low-trait (LT): 2.11).
First Reference in Text
As anticipated owing to how the groups were defined, the three groups differed in their self-reported autistic traits, as measured by BAPQ scores (F(2,163) = 232.86, P = 1.66 × 10-48, Npartial² = 0.74; Fig. 1a).
Description
  • Overall Focus: The caption refers to 'Fig. 1a', which is a component of a larger figure (Fig. 1) that presents trait comparisons across different groups. This specific component focuses on comparing the levels of self-reported autistic traits between an Autism Spectrum Disorder (ASD) group, a High-Trait (HT) group, and a Low-Trait (LT) group.
  • Measurement Tool: The caption specifies that self-reported autistic traits were measured using the Broad Autism Phenotype Questionnaire (BAPQ). The BAPQ is a questionnaire designed to quantify autistic-like traits in individuals, even if they don't have a formal ASD diagnosis.
  • Sample Sizes: The number of participants in the ASD and High-Trait groups is explicitly stated as n = 56 for each group, indicating that there were 56 individuals in each of these groups.
  • Statistical Analysis: The statistical analysis used to compare the groups is described as 'two-sided pairwise comparisons using estimated marginal means, with confidence intervals and P values adjusted for multiple comparisons using the Tukey method.' This means that the researchers compared each pair of groups (ASD vs. HT, ASD vs. LT, HT vs. LT) to see if their means were different, and they used a method (Tukey's) to correct for the fact that they were doing multiple comparisons, which can increase the chance of finding a significant difference just by chance. The 'estimated marginal means' are the average scores for each group, adjusted for any other variables in the model.
  • Statistical Results: The results of the comparison between the ASD and HT groups are presented as follows: t(111) = -0.28, P=0.957, estimated difference = -0.026, 95% CI [-0.25, 0.19], Cohen's d = -0.05. Here, 't(111) = -0.28' refers to the t-statistic, a measure of the difference between the two groups' means relative to the variability within the groups, with 111 degrees of freedom (related to the sample size). 'P=0.957' is the p-value, indicating that there is a 95.7% chance of observing the data (or more extreme data) if there is truly no difference between the groups. 'Estimated difference = -0.026' is the estimated difference in the means of the two groups. '95% CI [-0.25, 0.19]' is the 95% confidence interval, which provides a range of values within which the true difference between the group means is likely to fall. 'Cohen's d = -0.05' is a measure of effect size, quantifying the size of the difference between the two groups' means in standard deviation units.
  • Mean BAPQ Scores: The mean BAPQ scores for each group are also provided: mean ASD: 3.82, mean HT: 3.85, mean low-trait (LT): 2.11. These scores represent the average level of self-reported autistic traits in each group, according to the BAPQ.
Scientific Validity
  • Statistical Support: The claim that the ASD and HT groups had comparable levels of autistic traits is supported by the statistical analysis provided, which shows a non-significant difference (P = 0.957).
  • Appropriate Post-Hoc Testing: The use of appropriate post-hoc tests (Tukey method) for multiple comparisons strengthens the validity of the conclusion, as it controls for the increased risk of Type I error.
  • Effect Size Provided: Providing the effect size (Cohen's d = -0.05) is valuable, as it quantifies the magnitude of the non-significant difference. A small effect size further supports the conclusion that the groups are similar in their self-reported autistic traits.
Communication
  • Concise Summary: The caption concisely summarizes the key finding that the ASD and high-trait groups showed comparable levels of self-reported autistic traits. The inclusion of group abbreviations (ASD, HT, LT) aids in quick comprehension.
  • Potential Overload of Statistical Detail: The caption includes a substantial amount of statistical detail. While comprehensive, this might overwhelm some readers. Moving some of the detailed statistical information (e.g., degrees of freedom) to the main text or a footnote could improve readability.
  • Effective Cross-Referencing: The reference to 'Fig. 1a' in the Results section is appropriate and helps the reader quickly locate the relevant visual representation of the data.
Fig. 1 | Trait comparisons. b, c, Investigation into traits of other disorders...
Full Caption

Fig. 1 | Trait comparisons. b, c, Investigation into traits of other disorders characterized by social impairment revealed that, compared with both other groups (n = 56 participants each), the high-trait group (n = 56 participants) self-reported a higher level of social anxiety (two-sided mixed-effects model with random intercept for matched pair ID: F(2,163) = 59.80, P = 3.33 × 10-20, Npartial2 = 0.42; mean ASD: 35.39, mean HT: 46.43, mean LT: 19.21 (b)) and avoidant personality disorder (AVPD) symptoms (two-sided mixed-effects model with random intercept for

Figure/Table Image (Page 4)
Fig. 1 | Trait comparisons. b, c, Investigation into traits of other disorders characterized by social impairment revealed that, compared with both other groups (n = 56 participants each), the high-trait group (n = 56 participants) self-reported a higher level of social anxiety (two-sided mixed-effects model with random intercept for matched pair ID: F(2,163) = 59.80, P = 3.33 × 10-20, Npartial2 = 0.42; mean ASD: 35.39, mean HT: 46.43, mean LT: 19.21 (b)) and avoidant personality disorder (AVPD) symptoms (two-sided mixed-effects model with random intercept for
First Reference in Text
The groups differed in their social anxiety symptoms (F(2,163) = 59.80, P = 3.33 × 10-20, Npartial² = 0.42; Fig. 1b), such that the high-trait group had higher scores (indicating more symptoms) than both the low-trait group (t(110) = 10.87, P = 5.72 × 10-14, estimated difference = 27.3, 95% CI [21.4, 33.3], Cohen's d = 2.06) and the ASD
Description
Scientific Validity
Communication
Fig. 1 | Trait comparisons. matched pair ID: F(2,163) = 107.84, P=1.46×10-30,...
Full Caption

Fig. 1 | Trait comparisons. matched pair ID: F(2,163) = 107.84, P=1.46×10-30, Npartial² = 0.57; mean ASD: 20.09, mean HT: 23.80, mean LT: 11.36 (c)).

Figure/Table Image (Page 4)
Fig. 1 | Trait comparisons. matched pair ID: F(2,163) = 107.84, P=1.46×10-30, Npartial² = 0.57; mean ASD: 20.09, mean HT: 23.80, mean LT: 11.36 (c)).
First Reference in Text
The pairwise group differences for AVPD traits follow the same pattern as social anxiety: the high-trait group had higher scores (indicating more symptoms) than both the low-trait group (t(110) = 14.58, P = 2.27 × 10-14, estimated difference = 12.50, 95% CI [10.42, 14.58], Cohen's d = 2.70) and the ASD group (t(111) = -4.18,
Description
  • Overall Focus: The caption describes part of Figure 1, specifically component '(c)', which presents results related to Avoidant Personality Disorder (AVPD) symptoms across three groups: ASD, High-Trait (HT), and Low-Trait (LT). The sample size for each group is implicitly stated as n = 56, as it is mentioned in prior captions for Figure 1.
  • Statistical Analysis: A mixed-effects model with a random intercept for matched pair ID was used for the analysis. A mixed-effects model is a statistical technique that allows for the analysis of data with both fixed effects (effects that are of direct interest) and random effects (effects that account for variability in the data). In this case, the random intercept for matched pair ID controls for the fact that some participants were matched, meaning their data points might be more similar to each other than to data points from other participants.
  • Statistical Results: The results of the mixed-effects model are presented as F(2,163) = 107.84, P=1.46×10-30, and Npartial² = 0.57. 'F(2,163) = 107.84' refers to the F-statistic, a measure of the variance between group means relative to the variance within groups, with 2 and 163 degrees of freedom. 'P=1.46×10-30' is the p-value, indicating extremely strong statistical significance. 'Npartial² = 0.57' is partial eta-squared, a measure of effect size, indicating the proportion of variance in AVPD symptoms that is explained by group membership, after controlling for other variables.
  • Mean AVPD Scores: The mean AVPD scores for each group are provided: mean ASD: 20.09, mean HT: 23.80, mean LT: 11.36. These scores represent the average level of self-reported AVPD symptoms in each group.
Scientific Validity
  • Appropriate Statistical Model: The use of a mixed-effects model with a random intercept for matched pair ID is appropriate, given the study design with matched participants. This approach accounts for the non-independence of observations within matched pairs.
  • Strong Statistical Evidence: The reported F-statistic, p-value, and partial eta-squared provide strong evidence for a significant group difference in AVPD symptoms. The effect size (partial eta-squared = 0.57) indicates a large effect.
  • Incomplete Pairwise Comparison Details: The reference text correctly points out that the pairwise group differences follow a similar pattern to social anxiety. However, the statistical details of these pairwise comparisons are not fully provided in the caption, requiring the reader to consult the main text.
Communication
  • Clear Summary: The caption effectively summarizes that the figure component (c) displays the results of a mixed-effects model analysis for Avoidant Personality Disorder (AVPD) traits, including the F-statistic, p-value, and partial eta-squared. The group means are clearly presented, which allows for a quick comparison of AVPD symptoms across the groups.
  • Assumed Knowledge: The use of abbreviations is consistent with prior captions, which aids in reader comprehension. However, the caption assumes that the reader is familiar with the meaning and interpretation of mixed-effects models and associated statistics.
  • Missing Key Finding: The caption could be improved by briefly stating the key finding (e.g., 'High-trait group reported highest AVPD symptoms') to provide more context.
Fig. 1 | Trait comparisons. d, In the in-person ASD group (n = 56...
Full Caption

Fig. 1 | Trait comparisons. d, In the in-person ASD group (n = 56 participants), there was no relationship between clinician-rated autistic traits measured via ADOS (mean = 13.85) and self-reported autistic traits measured via BAPQ (two-sided general linear model: b = 0.025, s.e.m. = 0.02, t(51) = 1.16, P = 0.251, 95% CI [-0.018, 0.067], Npartial² = 0.01).

Figure/Table Image (Page 4)