home site map our company advisory board contact us update profile
| 
| 
| 
| 
| Ann Glaser, Editor in Chief |
| News & Information » News » Articles » Interviews » Ask the Expert » Submissions |
| » 800.242.5183» Email Us |
| Home | Articles | Article Details |
  |
|
8/23/2004 Assessing Grammatical Morpheme Production Using Elicited Sampling Brenda L. Beverly & Holly Goodnoh | |
| Brenda L. Beverly University of South Alabama Mobile, AL Holly Goodnoh The Gregory Kistler Treatment Center for Children, Inc. Fort Smith, AR Abstract Language sampling is a well-known, performance-based assessment procedure. However, routine use by speech-language pathologists (SLPs) is limited. This is unfortunate, because assessment in educational settings warrants periodic use of language sampling. Furthermore, investigations of grammatical morphology using language sampling have revealed important differences between children who are specifically language impaired (SLI) compared to age-matched, typically developing (TD) peers and younger, TD children matched for mean length of utterance (MLU). Correct production of three grammatical morphemes, possessive -s, past tense -ed and 3rd person singular -s, was compared in traditional free-play language samples, and in contextualized, elicited samples. Four children with SLI and three MLU-matched TD children participated. Results revealed significant differences favoring the younger TD children in the elicited format but no group differences for free-play samples. The number of obligatory contexts within the 100-utterance free-play samples was insufficient for possessive -s and past tense -ed. Only contexts for the 3rd person singular -s occurred with sufficient frequency to consider the free-play samples valid. Elicited procedures are preferable to free-play language samples for assessing production of grammatical morphemes with children believed to be language-impaired. Elicited probes can be easily designed by clinicians, less time-consuming to administer and interpret, and provide an adequate number of opportunities to assess morphological performance in a valid manner. Finally, elicited sampling can be a central part of dynamic, functional assessments in the educational setting. Introduction Speech-language pathologists are well-versed in their responsibilities to assess children’s speech, language and communication skills for identification, goal development, progress measurement and accountability (ASHA, 1997). Assessment options include; standardized tests, developmental scales, performance-based or criterion-referenced measures, and less formal behavioral observations (Miller, 1981; Moore-Brown & Montgomery, 2001; Paul, 2001). Training programs in speech-language pathology include classroom and clinical practice opportunities directed toward these assessment options. SLPs in-training are instructed in the advantages and disadvantages of these procedures. Despite this, it is our impression that standardized test administration continues to dominate most assessments. This practice is understandable, but unfortunate. Performance-based assessment, including language sampling, can be an important component of assessment and management in the school setting (Moore-Brown & Montgomery, 2001). Language sampling procedures can assist clinicians in overcoming limitations from standardized tests. For example, the annual review for IEP purposes requires a progress assessment. However, annual evaluation with repeated standardized tests is not an indicator of children’s progress toward educational outcomes in response to treatment, as many standardized tests measure decontextualized and fragmented behaviors (Moore-Brown & Montgomery, 2001). Furthermore, repeated standardized testing cannot adequately measure children’s progress in response to treatment without consideration of effects due to practice and regression toward the mean (e.g., Dikmen, Heaton, Grant, & Temkin, 1999). These factors are not often addressed by clinicians. In lieu of standardized assessment, performance-based measures including language sampling can be used to measure functional change. There also exists a research-driven rationale for the use of language sampling with children suspected of language impairment. That is, researchers have sought to identify a clinical marker for specific language impairment (SLI; Rice 2000). SLI is defined as significant language impairment in the absence of deficits in hearing, cognition, neurological status, and social-emotional functioning (Leonard, 1998; Stark & Tallal, 1981). A clinical marker for SLI would allow investigators and clinicians to confidently differentiate impaired individuals for clinical and research purposes. Aspects of grammatical performance currently hold the most promise for a clinical marker, because investigations of grammatical morphology using language sampling have revealed important differences between children with SLI compared to their typically developing (TD) age-matched peers and younger TD children matched for mean length of utterance (MLU; e.g., Leonard, Eyer, Bedore, & Grela, 1997; Oetting & Horohov, 1997; Rice, Wexler, & Cleave, 1995; Steckol & Leonard, 1979). Rice (2000) explained that grammatical morpheme production may be an effective clinical marker for children with SLI learning to speak English, because by age five TD children show near adult levels of grammatical morpheme performance. Rice reported that a group of five-year-old children with SLI fell below 60% accuracy for grammatical morpheme production compared to TD five year olds who performed above 80% accuracy. Only one child from each group would have been misidentified using a composite of several grammatical morphemes, including past tense -ed and 3rd person singular –s. Although a clinical marker has yet to be defined, clinicians and clients can benefit from implementing language sampling for assessing children’s production of grammatical morphology. Sampling tools for naturalistic assessment of language production have been widely available since the 1970’s (e.g., Lee & Canter, 1971; Miller, 1981; Retherford Stickler, 1987; Tyack & Gottlesben, 1974, Weiner, 1984). Miller (1981) provided an overview of sampling and analysis procedures for application in clinical settings. Although he focused on free-speech samples, he addressed elicited sampling. Elicited sampling was presented as a supplemental procedure for probing children’s productions of various forms and functions. Elicited samples were recommended for morphological and syntactic segments that do not occur frequently. This included relatively rare forms, such as passive voice, and forms that are not obligated, such as modals (i.e., can, will, could, may). Elicited samples were recommended to increase sample size for specific communicative functions, such as asking or responding to wh-questions. Various authors have identified contexts facilitative for eliciting specific forms and functions (see for example, Miller, 1981, and Paul, 2001). Miller cautioned that elicited procedures were experimental, lacked normative results, and should be used in conjunction with free-speech language samples, or when the determination of a child’s developmental level was not the primary purpose. These cautions seem less relevant today because performance-based methods of assessment are recognized as valuable components of comprehensive evaluations and performance criteria have been established (e.g., Owens, 2001; Retherford, 2000). Investigations of grammatical morpheme production by children with SLI revealed effects of language sampling procedures. For example, Leonard et al. (1997) found significant differences in free-play language samples compared to elicited probes. Specifically, traditional free-play language samples indicated no significant differences in production of 3rd person singular -s or articles (i.e., a, the) between children with SLI and MLU-matched peers. However, children with SLI performed more poorly than younger MLU-matched children when these morphemes were tested in elicited samples. Increasingly, researchers have employed elicited samples when investigating grammatical morpheme production and SLI (e.g., Cleave & Rice, 1997; Oetting & Horohov, 1997; and Rice, Wexler, & Hershberger, 1998). Free-play language sampling and elicited probes were included in a recent study of grammatical morpheme production by children with SLI and younger, MLU-matched TD children (Goodnoh & Beverly, 2002). Results for three grammatical morphemes in these two contexts are presented below. We hypothesized that younger TD children would show greater percentages of correct use for all three grammatical morphemes. No differences between sampling contexts were predicted. Method Participants Data represent a subset from other investigations, collected by the first author, in the area of grammatical morpheme production by children with SLI (Beverly & Williams, in press; Goodnoh & Beverly, 2002). The number of eligible children who participated in both free-play and elicited samples was small. Participants included; four children with SLI, mean age of 59 months, and three TD children with a mean age of 44 months, matched for gender and MLU (+/- 0.2 morphemes). Mean MLUs for the SLI and TD groups were 4.28 and 4.47, respectively. An independent samples t-test revealed non-significant differences for MLU, t = .478, p = .653. Children with SLI had expressive language standard scores < 85 on the Preschool Language Scale – 3 (PLS-3; Zimmerman, Steiner, & Pond, 1992) and MLUs >1 SD below age-level expectations based on 100 complete and intelligible utterances analyzed with Systematic Analysis of Language Transcripts (SALT; Miller & Chapman, 2000). Phonological deficits, nonverbal cognitive deficits, hearing impairment, and neurological and social/emotional problems were ruled out for all participants. See Table 1 for a summary of participant characteristics. Language assessment was conducted to verify group assignment, SLI versus TD. Language age-equivalent scores are reported with caution, for descriptive purposes. Table 1: Participant Description  Procedures Experimental tasks consisted of a free-play language sample and an elicited probe task. Free-play samples were collected during clinician-child play with a variety of toy sets (barn and animals, train and car set, dolls and accessories). From the free-play sample, 100 consecutive utterances that were complete, intelligible and spontaneous were transcribed and analyzed with SALT to determine MLU. MLU was used for pre-experimental eligibility and matching purposes. The behaviors of interest, percentages of correct use in obligatory contexts for the target morphemes possessive -s, past tense -ed, and 3rd person singular -s, were coded independently by the investigators using the typed transcripts. Inter-examiner reliability for transcripts was greater than 85%, and inter-examiner reliability for morpheme coding was greater than 90% item-by-item agreement. Participants were administered three elicited probe tasks, created by the second author, to assess correct production of the three target morphemes. For each morpheme, there were 10 opportunities for productions in a meaningful context. Questions and open-ended sentences, similar to those used on some standardized tests, were scripted to accompany hands-on materials and a storybook context. The first task, “animal friends,” provided opportunities for possessive -s through role-play with animal characters. Six cut-out pictures of animals with corresponding, color-matched toys and foods were used. The child was told that the goals were to feed the animals and help them find their toys. For example:
The second task, the “clumsy chef” story was designed to elicit past tense -ed. The second author wrote and illustrated a brief, original story about a clumsy chef. The main characters carried out a series of goal-directed actions, using 10 verbs ending in vowels or liquids – play, pour, pull, spill, glue, stir, try, smell, color, pay. Actions were presented on one page in the present tense. Then, on the page following, the completed action was depicted and an open-ended sentence was used to elicit the past tense -ed, as follows:
The third task, designed to test 3rd person singular -s, involved play with a dollhouse and accessories. The investigator modeled the target morpheme in a sample utterance and then asked questions requiring 3rd person singular -s productions to describe the functions for household objects. For example:
During the elicited tasks, children were reinforced for responding with verbal and tangible reinforcers. Responses were followed by the investigator’s model of the target morpheme, for example:
Thus, participants had investigator models for target morphemes throughout the task regardless of the correctness of their responses. Results Grammatical morpheme means and standard deviations for both groups in both sampling conditions are presented in Table 2. Figures 1 and 2 depict results for free-play language samples and elicited probes, respectively. An omnibus analysis of variance (ANOVA), 2 (Group) X 3 (Morpheme) X 2 (Context) with repeated measures on Morpheme and Context, was computed initially. Despite the relatively low statistical power with few participants, significant main effects were revealed for Morpheme, F = 9.954, p = 0.025, and Group, F = 7.505, p = .041. One interaction was significant, Morpheme X Context, F = 7.828, p = .038. Table 2. Percentage of Correct Grammatical Morpheme Production by Children with SLI and MLU-Matched Controls in Free-Play and Elicited Samples  Figure 1. Percentages of correct use in obligatory contexts for the three target morphemes in free-play language samples by four children with SLI and three MLU-matched TD children  Figure 2. Percentages of correct use for the three target morphemes in elicited probes by four children with SLI and three MLU-matched TD children  Possessive -s Obligatory contexts in free-play samples for possessive -s ranged from 0 to 3 total opportunities. Both free-play and elicited samples revealed higher accuracy for correct use of possessive -s by younger TD children than the group with SLI. Performance for possessive -s by both groups was greater in the elicited samples compared to the free-play samples. In the elicited samples (see Figure 2), TD children outperformed children with SLI, means of 100% and 45%, respectively. All three TD children achieved mastery (defined here as better than 80% correct use) in the elicited context, but only one of the four children with SLI achieved mastery. Independent samples t tests revealed non-significant group differences for the free-play language sample, t = 1.025, p = .352, but differences approaching statistical significance for the elicited sample, t = 2.515, p = .054. Note that these p values are two-tailed. Past Tense -ed Like possessive -s, free-play revealed few opportunities for past tense -ed production; 0 to 2 opportunities in 100 utterances per child. Results from free-play language samples revealed 0% correct production by children with SLI and only 33% accuracy by TD children. Likewise, neither group achieved mastery levels for past tense -ed in the elicited probes; however, both groups had higher means for the elicited condition. In the elicited sample (see Figure 2), TD children produced past tense -ed more often (60% accuracy) than children with SLI (13% accuracy). An independent samples t test was nonsignificant for the two groups’ correct percentages of past tense -ed in the free-play condition, t = 1.000, one-tailed p = .423. (Levene’s Test indicated significance, such that equal variances were not assumed). Significant differences, however, were found between groups in the past tense -ed elicited probe, t = 3.053, one-tailed p = .028. Third Person Singular -s For 3rd person singular -s, TD children had a mean of nine obligatory contexts per free-play sample and children with SLI had a mean of 14 contexts per sample. The children with SLI had a mean of 55% correct use of 3rd person -s in free-play samples, and the TD children had 93% accuracy in free-play (see Figure 1). Unlike the other target morphemes, TD and SLI groups had decreased mean percentages of accuracy in the elicited samples compared to free-play, 35% and 87%, respectively. Two of three TD children showed mastery for 3rd person singular -s in the elicited probes, but no child with SLI attained mastery. Significant group differences based on independent samples t tests were found for the free-play sample, t = 2.692, p = .043, and for the elicited probe, t = 2.688, p = .043. Discussion: Grammatical morpheme production by children with SLI and their MLU-matched peers was assessed in two sampling procedures; free-play and elicited contexts. Means for the three target morphemes – possessive -s, past tense -ed, and 3rd person singular -s – were higher for the group of younger TD children than means for the group of children with SLI. Despite this, free-play language samples did not reveal statistically significant differences in grammatical morpheme production between impaired and un-impaired children. The elicited probes, in contrast, revealed significant differences favoring the group of younger TD children. Thus, elicited probes were more effective than free-play language samples for assessing children’s grammatical morpheme production. Perhaps, these findings are not surprising to researchers of grammatical morpheme production, however, the clinical implications are considerable. Foremost is the concern that free-play language samples may not provide a valid assessment of grammatical morpheme production. Given the time-consuming nature of free-play language sample collection and analysis, this limitation would seriously undermine frequent use for the purposes of morphosyntactic assessment. Specifically, too few obligatory opportunities for production of possessive -s and past tense -ed occurred in the 100-utterance free-play samples. Participants in this investigation had fewer than four contexts, a minimum established in many research investigations (e.g., Cleave & Rice, 1997; Beverly & Williams, in press). For 3rd person singular -s, the free-play samples arguably had sufficient contexts for clinical purposes. Five out of seven participants had at least 10 obligatory contexts. No child had fewer than six opportunities and one had as many as 21 obligatory contexts. This issue of sample size for valid assessment is particularly problematic for clinicians; after all, language samples for clinical purposes are likely to consist of fewer than 100 utterances. Thus, contextualized probes designed to elicit a minimum of 10 productions are more valid estimates than free-play language sampling for assessing some morphemes. Interestingly, individual children who had 100% production for any of the target morphemes in free-play sampling also had above mastery percentages of accuracy in the elicited samples. However, children who had zero correct productions of possessive -s or past tense -ed, had varied performances in elicited samples, that could only have been predicted if group assignment was known. For example, one TD participant, Subject Two, had 0% correct use for possessive -s and past tense -ed from the free-play sample, but in the elicited samples she had 100% and 60% correct, respectively. Conversely, three of the four children with SLI had 0% accuracy for possessive -s and past tense -ed in free-play, and they continued to have 0% to 50% accuracy in the elicited samples. Therefore, free-play samples for possessive -s and past tense -ed, given an insufficient number of obligatory contexts, would lead clinician’s to erroneous conclusions about children’s ability to produce these morphemes. Another benefit of elicited sampling is its use as a dynamic assessment procedure. Unlike standardized tests or traditional language sampling procedures that sample a behavior in time, dynamic assessment is used to assess speech-language and communication skills as the examiner interacts with the child or mediates the task (Moore-Brown & Montgomery, 2001). During dynamic assessment activities, a behavior is tested, taught, and tested again, to assess the child’s ability to learn the target skill. Elicited probes are compatible with this purpose. For example, our 10-item activities could provide quick pre-test and post-test measures in conjunction with a mediated activity designed to teach the desired behavior. Alternatively, the examiner can mediate the child’s attempts to produce the target behavior while administering the elicited probe. After all, consistent feedback was provided to participants in this investigation through investigator modeling of the target morpheme in context after every response attempt, correct or incorrect. In fact, participants’ correct productions for possessive -s and past tense -ed increased in elicited probes compared to free-play samples. This was true for both groups, but it did not result in children with SLI performing at mastery levels. We conclude that dynamic use of elicited probes can increase clinicians’ confidence regarding the validity and reliability of their assessments, while providing important information about the modifiability of children’s communication skills and the level of teacher/clinician support needed to affect a change. Another advantage of elicited probes is the relative ease with which they can be created and administered. The possessive -s and 3rd person singular -s probes used in this investigation were quickly assembled with common materials found in our clinic. Our original, illustrated story with carefully controlled verbs for past tense -ed assessment could not be re-created, but readily available children’s storybooks could be equally effective for probing past tense production. Miller (1981) presents several protocols, including elicitation procedures for wh-questions, tag questions, negation, reflexive pronouns, and word order in agent-action-recipient utterances. Paul (2001) describes the use of elicited sampling procedures and other criterion-referenced methods across several developmental stages. Her text includes user-friendly protocols for probing preposition use, past/future tense, age-appropriate verb use, pragmatic functions, and comprehension of vocabulary associated with classroom directions, just to name a few. One limitation requiring attention is the assessment of standard English morphology with individuals from non-mainstream backgrounds. Known morphosyntactic differences in dialects must be considered before undertaking assessment of grammatical morphology production, and certainly, cultural/linguistic factors will need to be dealt with before a grammatical clinical marker can be established for SLI. This limitation leads us to further endorse elicited sampling over free-play sampling. Dynamic assessments, which can include elicited samples, are recommended when assessing individuals from culturally and linguistically different backgrounds because they do not rely on children’s prior knowledge. In conclusion, investigation of grammatical morpheme production using both traditional free-play language sampling and elicited samples revealed important advantages of elicited procedures. Elicited samples, not free-play language samples, revealed significant differences in production of three grammatical morphemes by younger MLU-matched TD children compared to children with SLI. Only the elicited samples provided an adequate number of opportunities for producing some grammatical morphemes, thereby increasing validity of assessment. Furthermore, ease of creation and flexible administration make elicited sampling an effective, performance-based assessment tool for routine use by speech-language pathologists. References: American Speech-Language-Hearing Association (ASHA). (1997). Preferred practice patterns for the profession of speech-language pathology. Rockville, MD: Author. Beverly, B. L., & Williams, C. C. (in press). Present tense “be” use in young children with SLI: Less is more. Journal of Speech, Language, and Hearing Research. Cleave, P. L., & Rice, M. L. (1997). An examination of the morpheme BE in children with specific language impairment: The role of contractibility and grammatical form class. Journal of Speech, Language, and Hearing Research, 40, 480-492. Dikmen, S. S., Heaton, R. K., Grant, I., & Temkin, N. R. (1999). Test-retest reliability and practice effects of expanded Halstead-Reitan neuropsychological test battery. Journal of the International Neuropsychological Society, 5(4), 346-356. Goodnoh, H., & Beverly, B. L. (2002, November). Grammatical morpheme patterns as MLU increases in children with SLI. Paper presented at the annual convention of the American Speech-Language-Hearing Association, Atlanta, GA. Lee, L. L., & Canter, S. (1971). Developmental sentence scoring: A clinical procedure for estimating syntactic development in children’s spontaneous speech. Journal of Speech and Hearing Disorders, 36, 315-338. Leonard, L. B. (1998). Children with specific language impairment. Cambridge, MA: The MIT Press. Leonard, L. B., Eyer, J., Bedore, L., & Grela, B. (1997). Three accounts of the grammatical morpheme difficulties of English-speaking children with specific language impairment. Journal of Speech, Language, and Hearing Research, 40, 741-753. Miller, J. F. (1981). Assessing language production in children: Experimental procedures. Baltimore, MD: University Park Press. Miller, J. F., & Chapman, R. S. (2000). SALT: A computer program for the systematic analysis of language transcripts [computer software]. Madison, WI: Language Analysis Laboratory, Waisman Center, University of Wisconsin. Moore-Brown, B. J., & Montgomery, J. K., (2001). Making a difference for America’s children: Speech-language pathologists in public schools. Eau Claire, WI: Thinking Publications. Oetting, J. B., & Horohov, J. E. (1997). Past-tense marking by children with and without specific language impairment. Journal of Speech, Language, and Hearing Research, 40, 62-74. Rice, M. L. (2000). Grammatical symptoms of specific language impairment. In D. V. M. Bishop & L. B. Leonard (Eds.), Speech and language impairments in young children: Causes, characteristics, intervention and outcome (pp. 17-34). Philadelphia, PA: Taylor & Francis. Rice, M. L., Wexler, K., & Cleave, P. (1995). Specific language impairment as a period of extended optional infinitive. Journal of Speech and Hearing Research, 38, 850-863. Rice, M. L., Wexler, K., & Hershberger, S. (1998). Tense over time: The longitudinal course of tense acquisition in children with specific language impairment. Journal of Speech, Language, and Hearing Research, 41, 1412-1431. Owens, R. E. (2001). Language development: An introduction, 5th Edition. Boston, MA: Allyn and Bacon. Paul, R. (2001). Language disorders from infancy through adolescence: Assessment and intervention, 2nd Edition. St. Louis, MO: Mosby. Retherford, K. S. (2000). Guide to analysis of language transcripts, 3rd Edition. Eau Claire, WI: Thinking Publications. Retherford Stickler, K. (1987). Guide to analysis of language transcripts. Eau Claire, WI: Thinking Publications. Stark, R. E., & Tallal, P. (1981). Selection of children with specific language deficits. Journal of Speech and Hearing Disorders, 46, 114-122. Steckol, K. F., & Leonard, L. B. (1979). The use of grammatical morphemes by normal and language-impaired children. Journal of Communication Disorders, 12, 291-301. Tyack, D., & Gottlesben, R. (1974). Language sampling, analysis, and training: A handbook for teachers and clinicians. Palo Alto, CA: Consulting Psychological Press. Weiner, F. (1984). Computerized language sample analysis [Computer software and manual]. State College, PA: Parrot Software. Zimmerman, I. L., Steiner, V. G., & Pond, R. E. (1992). Preschool Language Scale, 3rd Edition. San Antonio, TX: Psychological Corporation. Author Note Portions of this paper were presented in November 2002 at the annual convention of the American Speech-Language-Hearing Association, Atlanta, GA. We gratefully acknowledge the participation of the cooperating children and their families. We thank Rebecca McCullen for assisting with illustrating the storybook elicited task. | |