Results Overview

Oklahoma Schools Attuned 2002 Program Evaluation

September 30, 2002

Education Evaluation Associates

Norman, Oklahoma

Summary and Recommendations

for a quick look, and an overview, please visit http://www.beyondutopia.net/akom/at-a-glance.htm

An end-of-year survey conducted in May 2002 of teachers and students who have participated in the Oklahoma Schools Attuned program, as well as an in-depth look at student records and case studies of attuned students suggests the following:

• Students who have been a part of a Schools Attuned school with deep depth of coverage will be able to make great improvements in overall school performance, sometimes as much as 35% gain in overall scores.
• Students starting at a lower point scholastically, and who have had continuous contact with teachers who have been through All Kinds of Minds training have the largest increase.
• Teachers who understand the All Kinds of Minds program are able to implement the concepts more effectively, and develop personalized plans for accommodation.
• Teachers are most comfortable with strategies and accommodations to help students with attention and social cognition problems.
• Parental and family support can contribute to the success of All Kinds of Minds, but even without it, it is effective.
• Teachers perceive themselves as empowered by All Kinds of Minds strategies, which results in a higher belief in one’s ability to effect change.
• Teachers believe in the soundness of the principles in All Kinds of Minds, and believe that implementing them will yield results.
• Teachers believe in the soundness of the All Kinds of Minds recommendations, and are willing to incorporate them in their daily teaching.
• Key factors for success of the program are a) teacher attitude; b) understanding of the AKOM approach and materials; c) access and availability of AKOM materials; d) perceived support system (ongoing training, online resources, liaisons); administrative support (school board, principal).

A survey of teachers and administrators having participated in All Kinds of Minds training for the first time suggests the following:

• 88% are planning to take extra time to identify students who may benefit from the Schools Attuned approaches;
• 91% of teachers believe that the Schools Attuned program helps the teacher better understand how to be more effective in teaching or managing programs or schools that include at-risk students;
• 91.5% believe that Schools Attuned can help develop strategies for each of the students so they can use their strengths to succeed in all aspects of academic life;
• 87% have greater confidence in their ability to be effective as a teacher in the upcoming year, due to the Schools Attuned program;
• 90% have new insight into why favorite teaching approaches are effective.

Teachers indicated they would like to have the following:

• Ongoing teacher training in the All Kinds of Minds philosophies and methods;
• Summer training up to four weeks in length, with an in-depth approach to implementing All Kinds of Minds philosophies and methods (40%);
• Readily available materials to help with planning, attuning, and conceptualization of key terms and approaches, either downloadable via the Internet or through a contact person (91%);
• Resources that will help develop strategies for building and improving student motivation (86%);
• Resources to help students “preview” (91%);
• Resources to help students develop “response inhibition” (92.5%);
• Resources to help students with pacing and self-regulation (92%);
• Resources and information on self-monitoring for students (93%).

Based on preliminary program evaluation results, it is highly recommended that the following action steps be taken:

1. Expand the program to include more teachers, with the goal that eventually all teachers in Schools Attuned will have had at least rudimentary training, and access to All Kinds of Minds materials;
2. Expand the base of All Kinds of Minds liaisons to provide ongoing support as the program expands;
3. Increase the availability of support materials, including books and Internet-downloadable materials, including forms;
4. Expand teacher-parent programs so that all
5. Use the All Kinds of Minds program to help identify schools that could qualify for funding under various federal programs, or through philanthropic organizations.

Index to all summary statistics:

http://www.beyondutopia.net/akom/results-overview/

Full qualitative report:

http://www.beyondutopia.net/akom/OKSA-qualitative.htm

Group 1: Attuned Students and Non-Attuned Students in Schools Attuned schools

Random sampling of students in Oklahoma Schools Attuned program schools: Enid Roosevelt, Enid McKinley, Tulsa Monte Cassino, Tulsa Holland Hall, Tulsa Carnegie.

Assessment methodology: Instruments / questionnaires for students and teachers. Instruments received from Dr. Swartz, Chapel Hill. Implementation and data collection directed by independent evaluators.

Timing of Assessment: May 2002

Grades 2 – 4:

Result: An atmosphere conducive to learning, with self-confidence, and a belief in their own abilities to succeed both academically and socially. Summary results displayed at: http://www.beyondutopia.net/akom/ase-random/self-efficacy-random-samples.htm

85% or more of all students in grades 2 – 4 believed the following things about themselves:

• That they can become better readers
• That they can become better at solving math problems
• That they can make friends
• That they can organize themselves when they need to work
• That they can become better students

Grades 5 and up:

Result: An environment that encourages positive attitudes and believe they are able to self-regulate and that academic success is achievable. Summary results displayed at: http://www.beyondutopia.net/akom/self-regulation/se-self-regulation.htm

60% or more of all students surveyed believed that they are able to:

• Keep attention focused
• Learn important concepts from texts or readings
• Study effectively for tests
• Manage time in order to have ample study time for class
• Keep from being distracted by others

Students are concerned about issues involving time management, social pressure, performance in front of others, and teacher opinion. Summary results are displayed at:

http://www.beyondutopia.net/akom/student-course-concerns/student-course-concerns.htm

Their ideas about why academic success occurs is mixed, with attributions ranging from intrinsically interesting material to mentoring. Summary results are displayed at:

http://www.beyondutopia.net/akom/causal-attribution/causal-attribution.htm

Group 2: Attuned Students

A randomly selected group of 40 students, ranging from 2^nd grade to 8^th grade, who have been through the attuning process in accordance with the Schools Attuned program. The students are from Monte Cassino, where virtually all teachers have participated in Schools Attuned training, and the Schools Attuned program has been in place for at least four years.

Assessment methodology: Instruments / questionnaires for students and teachers. Instruments received from Dr. Swartz, Chapel Hill. Others developed by independent evaluators. Implementation and data collection directed by independent evaluators.

Timing of Assessment: May – July 2002

Quantitative Results:

Students beginning with less than 3.50 (out of 4.0) overall performance score (a combined score incorporating all areas covered in the students’ grade reports, as well as Iowa scores)

Net change in overall score: 8.30 % positive increase

All students (including students beginning with less than 3.50 (out of 4.0) overall performance score (a combined score incorporating all areas covered in the students’ grade reports, as well as Iowa scores)

Net change in overall score: 3.92 % positive increase

Qualitative Results:

For full qualitative results, click here: http://www.beyondutopia.net/akom/OKSA-qualitative.htm

• Teachers demonstrated a high degree of willingness to identify students needing help, and to develop and implement a strategy for helping them succeed socially and academically;
• Teachers demonstrated a high degree of understanding of the concepts in the All Kinds of Minds approach, with particular high degrees of understanding in the areas of attention and social cognition;
• Teachers demonstrated a high degree of creativity in developing alternative learning approaches which built upon the students’ strengths;
• Teachers exhibited positive traits, with almost complete elimination of such negative traits or behaviors as labeling, punitive attitudes, impatience, etc. (Not surprising, considering the teachers self-reported. At the very least, results demonstrate that they were self-aware and conscious of negative traits and behaviors.).
• Positive results in academic and non-academic areas of endeavor were found in the majority of students evaluated. Success was based on teacher evaluation, grades.
• Teachers attributed positive results to the implementation of accommodations, interventions, and approaches recommended by the All Kinds of Minds’ Schools Attuned program, in addition to a multi-pronged, flexible approach incorporating extra-programmatic items including medication and counseling.
• Teachers in the Schools Attuned environment showed a high level of willingness to work with students on an individual basis, and to dedicate blocks of time to one-on-one mentoring, coaching, and tutoring.

Note: Because the teachers self-reported results, it is difficult to determine whether or not an individual teacher’s conception of self was shared by the student. While a teacher may have thought of herself or himself as caring, involved, and pro-active, the student’s perception could have been different. In fact, the students progress or lack of “progress” or “improvement” is a judgment call from the point of view of the teacher. A lack of progress, as reported by the teacher, could have been, in fact, resistance on the part of the student who did not like the approach. Because of the element of doubt that student resistance or perception dissonance could cause, it is important to follow up with interviews of students who have been attuned. Their responses could suggest that the program is even more effective than the teachers believe. It could also reveal when and how students’ perception of teachers differs from the teacher’s own self-concept.

Group 3: Teachers with Experience in the Schools Attuned Program

Targeted random sampling of teachers with Schools Attuned training, plus experience: Enid Roosevelt, Enid McKinley, Tulsa Monte Cassino, Tulsa Holland Hall, Tulsa Carnegie.

Assessment methodology: Instruments / questionnaires for students and teachers. Instruments received from Dr. Swartz, Chapel Hill. Implementation and data collection directed by independent evaluators.

Timing of Assessment: May 2002

Teacher Self-Efficacy in schools with less than 100% saturation (not all teachers in the school system have participated in AKOM training; nevertheless one can expect sharing of ideas and spillovers of attitudes):

• Good attitudes toward teaching and belief in one’s ability to make a difference
• High confidence in being able to identify student needs
• High confidence in being able to develop a response to student needs
• High degree of willingness to protect students from public humiliation
• Less confidence in ability to recognize underlying problems.

Summary results posted on Internet at: http://www.beyondutopia.net/akom/teacher-self-efficacy.htm

Teachers familiar with the All Kinds of Minds program, who have implemented strategies for at least one year believe that it is most helpful in areas of social cognition and in the constructs used in learning (attention, memory, time management, etc.)

Teachers have most confidence in the All Kinds of Minds program to help students with issues pertaining to:

• Attention
• Social Cognition

They feel most confident in areas of

• Strengthening Strengths
• Accommodations
• Interventions
• Progress and Monitoring

The have the most confidence in helping the student in:

Social Cognition: Relationships with Peers

Social Cognition: Relationships with Adults

Behavioral Problems

Teachers believe, overall that the All Kinds of Minds strategies are most helpful in the following areas:

social issues: very helpful	83.2 on a scale of 100
constructs (attention, memory, etc.): very helpful	75.7 on a scale of 100

Summary of results available through the Internet at: http://www.beyondutopia.net/akom/teacher-end-of-year/

Group 4: Teachers and Administrators with first-time training in the Schools Attuned Program

Assessment methodology: Instruments / questionnaires for students and teachers. Instruments developed by independent evaluators. Implementation and data collection directed by independent evaluators.

Timing of Assessment: August – September 2002

Instrument #1: http://intercom.virginia.edu/SurveySuite/Surveys/teacher-august-02/index2.html

Instrument #2: http://intercom.virginia.edu/SurveySuite/Surveys/oksa-attention-1

Instrument #3: http://intercom.virginia.edu/SurveySuite/Surveys/oksa-3-growth-strategies

A survey of teachers and administrators having participated in All Kinds of Minds training for the first time suggests the following:

• 88% are planning to take extra time to identify students who may benefit from the Schools Attuned approaches;
• 91% of teachers believe that the Schools Attuned program helps the teacher better understand how to be more effective in teaching or managing programs or schools that include at-risk students;
• 91.5% believe that Schools Attuned can help develop strategies for each of the students so they can use their strengths to succeed in all aspects of academic life;
• 87% have greater confidence in their ability to be effective as a teacher in the upcoming year, due to the Schools Attuned program;
• 90% have new insight into why favorite teaching approaches are effective.

Teachers indicated they would like to have the following:

• Ongoing teacher training in the All Kinds of Minds philosophies and methods;
• Summer training up to four weeks in length, with an in-depth approach to implementing All Kinds of Minds philosophies and methods (40%);
• Readily available materials to help with planning, attuning, and conceptualization of key terms and approaches, either downloadable via the Internet or through a contact person (91%);
• Resources that will help develop strategies for building and improving student motivation (86%);
• Resources to help students “preview” (91%);
• Resources to help students develop “response inhibition” (92.5%);
• Resources to help students with pacing and self-regulation (92%);
• Resources and information on self-monitoring for students (93%).