Project VOICES

Video Coding: Dyads Selection

With the addition of new undergraduate students and part-time personnel, a primary focus is to continue accelerating the coding process of dyadic interactions within videos.

With the assistance of undergraduate students and part-time staff, overall progress has increased from 25% to 38% of videos being completely coded. That count increases to 48%, if videos in progress or partially coded are included within the count. Dyadic coding continues to be the area where we seek to grow and maintain a large team given that completion of a full 3-hour video requires 4-6 hours of active coding time.

Video Coding: Content

Piloting and beta testing of the content coding system for identification of content instructional moments in the videos has continued to make significant process.

In February 2023, the team met with content experts Dr. David Purpura and Dr. Jessica Whittaker to obtain feedback on the developed coding system. The team was advised to look at additional state standards and cross reference similar coding schemes, to ensure that content moments had not been overlooked. Five additional states were randomly selected to be included in the standards review. Additionally, two coding schemes/manuals were pulled and examined: Individualizing Student Instruction Classroom Observations Coding Manual and Early Learning Math and Science Observation Scheme (ELMS). Review of additional state standards and established coding schemes increased confidence in the newly developed content coding manual. Standards from other states continued to fit within topic areas established and there’s significant overlap in coding schemes, while also finding areas of improvement. Once these additional revisions have been made, a final draft will be sent to Drs. Purpura and Whittaker for another feedback round.

Two additional part-time employees were added to the content team in January. New members have completed in-person training on content-focused interactions and the coding scheme.

Currently, team members are becoming reliable and must pass reliability at 85% or higher. Once reliability is passed the team reviews selected videos that comprise the random 15% that need to be coded. To begin, 77/516 videos have been randomly selected.

Language Sample Coding

Two additional staff members joined the “picking and snipping” team, to further accelerate the task of synchronizing the audio and video timing and to select and “snip” the needed 5-minute samples from the raw 3-hour audio files.

Personnel were required to complete detailed and rigorous training before working independently. The self-paced training taught staff how to pick from the videos and make the snips in Audacity. Reliability is determined by team members matching times to the master document created by the team lead. Team members were required to pass reliability at 80% or higher.

Snips were picked from the videos in three passes for centers, book reading, and circle time. The first pass involved copying all video and audio times from the initial Project Voices to ensure that any snips made weren’t duplicated in Voices II. The second pass used the video coding data from the initial Project Voices to determine if there were any additional times for the setting. Those times were then selected, and the snip made in Audacity. The 3rd pass made a final pass through the observation videos to determine if there were any additional times for the setting that was not accounted for in the video coding data.

As of March 2023, the first and second pass has been completed for centers, book reading, and circle time. A third pass is in progress for centers and book reading. As of March 2023, 374/774 (48%) centers snips have been picked and snipped, 249/516 (48%) of book readings have been picked and snipped, and 332/516 (64%) of circle time have been picked and snipped.

Teacher Audio Transcription

Undergraduate research students are assigned to the transcription team each semester. The lab has a high retention rate, with students opting to return for more than one semester.

Each semester, new transcription team members complete a self-paced 4–6-hour training offered by SALT. SALT offers built-in short quizzes at the end of each module for transcribers to test their knowledge. After completing the training transcribers must pass reliability at 80% or higher. Transcribers working with teacher audio listen to and transcribe a 5-minute snip from a project teacher. Transcribers working with child audio must listen to and transcribe two audio recordings from project children.

As of March 2023, 364/774 (47%) of centers snips have been transcribed, 174/516 (34%) of book reading snips have been transcribed, and 238/516 (46%) of circle time has been transcribed.

Child Audio Transcription

The child team is responsible for correcting transcripts from the initial Project Voices and adding language codes. All team members on the team must complete training for transcription and pass reliability as described in the transcription section. Afterwards they complete self-paced training for all coding schemes and are provided with instructions for correcting transcripts. Reliability for coding must be passed at 85% or higher and reliability for correction must be passed at 80% or higher.

As of March 2023, the child team consists of two volunteers who just began the training and reliability phase.

Trade Book Genre Coding

The genre coding team sorted books from the first grant based on genre (i.e., narrative, informational, and mixed). In addition to sorting based on genre, additional instruction foci of the text were coded (I.e., alphabet, math, science, and social studies). As of February 2023, coders have completed 100% of the 596 books. To complete this coding, a new coding scheme was created.

The coding manual was given to new coders to introduce them to the purpose of the task. The manual detailed the process of finding information on each text to identify appropriate genre. Other important components of the manual include descriptions of what each code entails, examples, and a decision tree for determining the genre.

Coders have completed coding 100% (n = 596) books. Preliminary descriptive data is as follows: there are 334 narrative, 86 informational, and 140 mixed genre texts with 16 falling into the other category and 20 being unidentifiable.

Hand-coding Basic Language Team

The initial pass of hand coding, now termed Basic Language coding, will include hand codes applied to teacher transcriptions from Center, Circle, or Book reading. The basic language team has been divided into two separate teams Basic Language Team 1 (BL 1) and Basic Language Team 2 (BL 2), with each specializing in a small subset of codes.

BL 1 will code child directed speech, open-ended questions, abstract nouns, and grammaticality, while BL 2 will code child directed speech, taught vocabulary, mental state verbs, and elaborated noun/verb phrases.

Training materials which include self-guided training with built-in practice and knowledge checks and a separate reliability test out were finalized in April. Coders must receive a reliability score of 85% or higher to continue coding on the team.

Hand-coding Elicitations Team

This team will code aspects of preschool teacher communication with children. Specifically, this team intends to code for the presence of teachers’ elicitations, expansions, and extensions of child responses. Researchers intend to train assistants by providing definitions and rationale for each code and information about both inclusionary and exclusionary criteria.

Researchers will create a reliability calculator to determine that assistants reach 90% reliability.

Coding for elicitations, expansions, and extensions will begin once the codebook is finalized and researchers train assistants to accurately and reliably code for each factor. The codebook is about 90% complete.

Dissemination

Manuscripts

• Chen, S., Phillips, B. M., & Dong, S. (submitted). Unpacking the language teaching belief-practice alignment among preschool teachers serving children from low SES backgrounds. Teachers and Teacher Education. Manuscript submitted for publication.
• Fumero, K., Wood, C., & Phillips, B. M. (submitted). Supportive language strategies for preschool dual language learners: Associations with early language outcomes. Early Childhood Research Quarterly. Manuscript submitted for publication.
• Phillips, B. M., Funari, C., Oliver, F., Berrien, J., Burris, P. W., & Mesa, M. P. (2022). Joint contributions of teacher’s pedagogical content knowledge and book reading to preschooler’s growth in language skill. Reading and Writing, 35(8), 1815-1838.
• Phillips, B. M., Oliver, F., Tabulda, G., Wood, C., & Funari, C. (2020). Preschool teachers' language and vocabulary knowledge: Development and predictive associations for a new measure. Dyslexia, 26(2), 153-172.

Dissertation

• Fumero, K. (2022). Supportive language strategies for preschool dual language learners: Associations to early language outcomes (Publication No. 28967513). [Doctoral dissertation, Florida State University]. ProQuest Dissertations & Theses Global. https://www.proquest.com/dissertations-theses/supportive-language-strategies-preschool-dual/docview/2703937921/se-2

Conference Publications

• Chen, S., Phillips, B. M., & Dong, S. (2018, April 13 - 17). Language instruction in preschools serving children from disadvantaged backgrounds: Teachers' beliefs, practices, and challenges. [Conference Presentation]. American Educational Research Association Annual Meeting, New York, NY, United States.
• Fumero, K., Wood, C., & Phillips, B. (2022, November 17 - 19). Language facilitation strategies for preschool dual language learners: Associations to early language outcomes. [Technical session]. American Speech Language Hearing, New Orleans, LA, United States.
• Funari, C., Mesa, M., Phillips, B., Ritchie, J., & Oh, Y. (2022, April 21 - 26). What do preschool teachers do all day? [Virtual Roundtable Presentation]. American Educational Research Association Annual Meeting, San Diego, CA, United States. https://doi.org/10.3102/1888848
• Phillips, B. M., Oliver, F. S., Mesa, M. P., & Funari, C. R. (2019, March 21 - 23). The relations between preschool teachers' background, classroom time allocation and quality of language instruction. [Poster presentation]. Society for Research in Child Development Biennial Meeting, Baltimore, Maryland, United States.
• Phillips, B. M. & Wood, C. (2023, March 23 - 25). Use of connectives: Exploring the development of an early emerging academic language feature among preschool children. [Conference presentation]. Society for Research in Child Development Biennial Meeting, Salt Lake City, UT, United States.

VOICES II Overview

Five Florida State University researchers have received a $1.7 million award funded by the Institute of Education Sciences to improve language in preschool classrooms.

Professor Beth Phillips, Associate Professor Sonia Cabell, Professor Yaacov Petscher (Florida Center for Reading Research), Professor Carla Wood (Communication Disorders), and Assistant Professor Lakeisha Johnson (Communication Disorders) lead a team of researchers in coding preschool classroom videos, teacher-specific audio recordings and child-specific audio recordings.

Early childhood education research robustly supports the critical importance of children’s language skills for their adjustment to and success in school for their long-term achievement, particularly in reading. Unfortunately, research also finds that the quality of classroom language environments is weak to moderate. Despite studies indicating that the language teachers use with children matter for their language learning, research suggests not all aspects of teacher talk are equally effective. There is a gap between our clear understanding of the critically urgent need to support language development and current knowledge of how best to do so for all children.

VOICES II seeks to identify aspects of teachers’ language in the classroom that are uniquely and meaningfully associated with children’s language development. These features must be clearly identified to bring about specific changes in teachers’ behavior to make a difference.

VOICES II represents secondary analyses – plus new coding of video and audio recordings – of data collected in VOICES I, funded by the Spencer Foundation. VOICES II builds on the work of VOICES I by bringing a substantially expanded focus on teachers’ language use in diverse instructional contexts at three time points. VOICES I included 7 language variables while VOICES II includes 21 language variables. VOICES II uses over 86,000 measurements across 86 teachers coupled with language data for over 580 children, to better understand the role of teachers’ talk in predicting growth in children’s critically important language skills.

VOICES I Overview

Project VOICES, or VOICES I, was a five-year grant funded by the Lyle Spencer Award from the Spencer Foundation to Drs. Phillips and Wood. VOICES I was designed to assess preschool children’s vocabulary and language skills, which are highly influential to their future academic success. VOICES I gathered information through teacher questionnaires, interviews, classroom observations, and individual child assessments that measure language skills for three school years (2015-2016; 2016-2017; and 2017-2018). This data led to a better understanding of how the general classroom environment and other classroom instructional activities influenced the development of children’s language skills.

VOICES I primarily investigated teacher and classroom characteristics predicting language environments and child outcomes, focusing on teacher’s knowledge and on diversity in preschool sites. Data was collected three times during the school year on teacher practices and child language to compose one of the largest samples of naturalistic language data from both teachers and children. VOICES I collected over 86,000 measurements across 86 teachers coupled with language data for over 580 children. VOICES I focused on designing and validating a teacher knowledge measure (TALK-P) and a classroom rating scale against a limited version of teacher-talk coding, and on characterizing the structure of child language with both standardized and naturalistic data.

VOICES I began the search to identify which teachers, with which characteristics, and in which contexts are or can do better for children ages 3-5, and what teachers themselves need to succeed in this endeavor.

In the News

FSU professors get $1 million grant to study preschool improvement
Tallahassee Democrat