Title: Challenges of Measurement in Community Research

"Empowering Choices" and "Transitions" are 3-year projects conducted in partnership between the Institute on Disability and Human Development at UIC and four Public High Schools serving low-income minority youths with severe disabilities. The goal of these projects is to improve academic retention and to prepare low-income minority youths with severe disabilities for competitive employment after graduating from high school. One component of the intervention includes classroom training that teaches students how to set goals and recruit the help of others in the goal attainment process. The development of goal setting, action planning, and help recruiting are integral intervention skills. Role playing exercises are conducted to measure students' acquisition of these behavioral skills. In addition, at-risk and engaging behavior is evaluated to assess the effect of the intervention on the students' behaviors. The panel intends to discuss the challenges in conducting role-plays, scoring the role-plays, and collecting at-risk and engaging behavior data.

During year one of the intervention twenty-five students from four Chicago Public High Schools received intervention training. Role-play assessments were used to measure participants' ability to recruit help from others and handle responses to their help requests. Specifically, students' ability to action plan (if potential helper agrees to help) and handle rejections (if potential helper refuses to help) were assessed. Challenges in conducting role-plays, such as engaging participants, will be discussed.

After the role-plays were conducted in the classroom, 2 trained research assistants scored them. The research assistants began by reviewing the original scoring instrument. Results obtained from this sample with an earlier version indicated that the student's ability to action plan and handle rejections increased after the intervention. However, the research team felt that the scoring scheme could be improved to more accurately reflect behaviors displayed during the role-plays. The role-play score sheets were therefore modified with the goal of getting more valid and reliable behaviors. The panel will discuss the challenges in developing a scoring instrument to quantify behavior data.

Another phase of the intervention attempts to promote school engagement. To evaluate intervention effects in this area, the at-risk behaviors and engaging behaviors of high school students participating in the intervention are assessed. At-risk behaviors are defined as excused and unexcused absences, tardiness, truancy, detentions, and suspensions. Engaging behaviors include participating in club meetings, sports practices, and volunteer activities, holding a part-time job, and staying after school to get extra help. During year two of the intervention attendance and discipline records for 146 students from four Chicago Public High Schools were reviewed to collect at-risk data, while in person or over the phone interviews were conducted to collect engaging behavior data. A trained research assistant collected these data over a four-month period. The difficulties of accessing the school records data, the differences in methods of collecting and keeping records among the participating schools, and the difficulty of tracking student participants for interviews will be discussed.

SHARE YOUR IDEAS!

BACK TO LIST OF ABSTRACTS