Frequently Asked Questions
About the survey
How was the Panorama Student Survey developed?
The Panorama Student Survey was developed under the leadership of Dr. Hunter Gehlbach, associate professor and associate dean at the UC Santa Barbara's Gevirtz Graduate School of Education and director of research at Panorama Education. Dr. Gehlbach and his research team at the Harvard Graduate School of Education followed a rigorous survey development process consisting of literature reviews, focus groups, feedback from experts around the county, cognitive interviews with students, and multiple rounds of piloting and refining.
Dr. Gehlbach and his team developed the instrument based on modern principles of survey design. These best practices include:
- Wording survey items as questions rather than statements
- Avoiding "agree-disagree" response options that may introduce acquiescence bias and instead using verbally labeled response options that reinforce the underlying topic
- Asking about one idea at a time rather than using double-barreled items (e.g., "How happy and engaged are you?")
- Using at least five response options to capture a wider range of perceptions
- Phrasing questions with positive language rather than using reverse-scored or negative language, which students tend to have trouble understanding
Who can participate in the survey? Will all teachers and schools participate?
Students statewide in grades 3-12 will have the opportunity to take the Panorama Student Survey in the fall. Students must have been enrolled continuously in a class for at least three (3) weeks with the same teacher in order to participate in the survey just as teachers must have provided direct instruction to their students for a minimum of three (3) weeks.
What does the Panorama Student Survey measure?
The Panorama Student Survey measures student perceptions of teaching and learning, culture and climate, and student experience in the classroom and school. Student perceptions of their classroom experience helps identify a teacher's strengths and areas for teacher growth. Schools and teachers can use the results to set specific priorities for differentiated professional development and coaching support, to design effective teaching practices and more engaging learning environments, and to take actions that will improve student experience in school.
The Panorama Student Survey contains two main categories of scales:
- Scales about the Classroom and Teaching: Measures student perceptions of teaching and learning in a particular class.
- Scales about the School: Measures student perceptions of teaching and learning at the school in general.
Within these two categories, Panorama offers several scales – or sets of questions – that measure specific topics (e.g., School Climate, Teacher-Student Relationships).
What are the survey topics?
Panorama has a total of 19 scales that measure student perceptions of teaching and learning in a particular class or at the school in general. The following scales have been selected for the HIDOE Panorama Student Survey:
Scales about the Classroom and Teaching
-
Pedagogical Effectiveness
Perceptions of the quality of teaching and amount learning students experience from a particular teacher.
Classroom Climate
Perceptions of the overall social and learning climate of the classroom.
Classroom Rigorous Expectations
How much students feel that a specific teacher holds them to high expectations around effort, understanding, persistence, and performance in class.
Classroom Engagement
How attentive and invested students are in class.
Scales about the School
-
School Belonging
How much students feel that they are valued members of the school community.
-
Valuing of School
How much students feel that school is interesting, important, and useful.
-
School Safety
Perceptions of student physical and psychological safety while at school.
How many questions or survey items are there per survey format?
The HIDOE Panorama Student Survey numbers of questions or survey items, per survey format are listed below:
- Upper Elementary (grades 3-5): 38 survey items
- Secondary (grades 6-12): 44 survey items
In addition to the HIDOE Panorama Student Survey questions/items, all students (grades 3-12) will be asked to respond to the School Quality Survey (SQS) safety dimension questions. There are six (6) SQS items.
What kinds of survey questions are asked?
Each of Panorama's survey scales includes 4-7 items that measure one specific construct or topic. Survey items were developed based on modern principles of survey design. These best practices include:
- Wording survey items as questions rather than statements
- Avoiding "agree-disagree" response options that may introduce acquiescence bias and instead using verbally labeled response options that reinforce the underlying topic
- Asking about one idea at a time rather than using double-barreled items (e.g., "How happy and engaged are you?")
- Using at least five response options to capture a wider range of perceptions
- Phrasing questions with positive language rather than using reverse-scored or negative language, which students tend to have trouble understanding
The Panorama Student Survey items use response options that reinforce the central focus of the question. For example, a question like "How much do you enjoy your class?" might have the following responses to keep respondents focused on the idea of "enjoy" as they think about their response:
Do not enjoy at all
| Enjoy a little bit
| Enjoy somewhat
| Enjoy quite a bit
| Enjoy a tremendous amount
|
An example question for each survey scale are included below:
Scales about the Classroom and Teaching
-
Pedagogical Effectiveness
How often does this teacher give you feedback that helps you learn?
-
Classroom Climate
In this class, how much does the behavior of other students hurt or help your learning?
-
Classroom Rigorous Expectations
How often does this teacher take time to make sure you understand the material?
-
Classroom Engagement
How often do you get so focused on class activities that you lose track of time?
-
Classroom Teacher-Student Relationships
If you walked into class upset, how concerned would your teacher be?
Scales about the School
-
School Belonging
How connected do you feel to the adults at your school?
-
Valuing of School
How much do you see yourself as someone who appreciates school?
-
School Safety
How likely is it that someone from your school will bully you online?
Student survey samples:
What information is there regarding the reliability and validity of the Panorama Student Survey?
Panorama developed the Panorama Student Survey as the first major survey instrument with the following essential properties:
- Educator-focused design, including survey scales that equip teachers with feedback they can use to improve practice and enable educators to monitor student attitudes, beliefs, and values that are predictive of important outcomes;
- Theoretically-grounded, empirically-based design process that meets or exceeds standards of academic scholarship;
- Adherence to best practices in survey design;
- Allowing schools to customize the survey to their specific needs and teaching frameworks while retaining validity and reliability; and
- Providing the survey instrument to any educator interested in improving pedagogical practice and student outcomes for free.
The Panorama Student Survey was developed by a team of researchers at the Harvard Graduate School of Education under the direction of Dr. Hunter Gehlbach. For more information, review Panorama's Validity Brief:
https://panorama-www.s3.amazonaws.com/files/panorama-student-survey/validity-brief.pdf
How is survey data be used?
The primary purpose of the survey is to provide valuable reflective information for educators who are working to improve the classroom and school learning environment. Teacher reflections of the Panorama Student Survey results are used to inform a teacher's Core Professionalism. Teachers may reflect upon their individual or school-level results as appropriate to their role in the school. Teachers should consult and collaborate with their evaluator on the acceptable documentation method of reflection. Data is also aggregated to provide school, complex- and state- level feedback. These data can be used to highlight priorities, track improvement and evaluate programs.
Survey logistics
Which teachers will not be surveyed?
These teachers in the following roles will not be surveyed:
- Teachers with students who all take the HSA-Alt assessment.
- Teachers with no Employee ID.
- Educational Assistants.
- Student Teachers.
- Part Time Teachers (PTTs).
- Visiting lecturers.
Teachers who are responsible for delivering direct classroom instruction and monitor student performance for three (3) or more weeks will administer the survey.
How are survey dates, teachers and classes selected?
Classes for teachers were electronically pre-selected using class data loaded into the infinite campus system. During the "SBT Set-Up", SBT coordinators should work with their school-based to check all teacher rosters to ensure the following:
- Each teacher has at least one class of students to complete the survey for teachers delivering direct classroom instruction and monitoring student performance.
- For teachers with smaller class sizes, classes are selected in the following order of preference:
- Class has a recommended minimum of ten (10) students for rostering. (See No. 3 below.)
- If there are no classes that meet the recommended minimum of ten (10) students for rostering, the teacher has students from different classes who meet the three (3)-week direct instruction time are combined into one class to achieve the ten (10) student recommended minimum requirement (it is
required that teachers of small class sizes
roster at least
ten (10) students; if a teacher receives fewer than ten (10) student responses, no report will be generated for that teacher.).
Example: Teacher A has a first period with five (5) students, a second period with four (4) students and a third period with six (6) students. The survey coordinator should combine all periods for a student total that ensures meeting the recommended minimum roster of ten (10) student participants on the day of the survey administration. - Even
if teachers are not able to roster the recommended minimum of ten (10) students, their
rosters may be submitted and these teachers may have their students participate in taking the Panorama Student Survey. However, these teachers will not receive a report if student responses for them do not meet the minimum of ten (10) required.
How is student confidentiality protected?
All paper surveys have individual identifiers (QR codes) and a cover page with student names. The identifying cover page may be disposed or placed in the sealed envelope. Each student is to place his/her completed survey into the provided blank envelope and seal it before submitting envelope to the proctor.
What are the differences between the Elementary (3-5) and Secondary (6-12) Panorama survey versions?
The Upper Elementary (3-5) survey (44 items) has fewer items than the secondary and uses language and vocabulary that is accessible to most students in these grades. The Secondary (6-12) survey (51 items) is designed for older students and includes more items and age-appropriate language.
Is the survey administered in paper or online format?
Schools decide on the format that they will be administering, either (but not both) paper or online for grades 3-12. The selection is determined by the SBT prior to roster verification (RV).
How long does the survey take to complete?
Usually, 30-40 minutes is ample time to complete the entire process for the survey, for all levels, including material distribution and instruction.
How is the survey administered to students in different age groups?
The surveys are administered to the whole class in grades 3-12. Students in grades 3–12 follow a standard protocol that typically involves the teacher administering the survey with the students reading the survey themselves. Specific details and instructions are outlined in separate administration instructions and scripts for the survey levels.
Will parents be notified before the survey is administered?
Yes, an informational letter in English that has been translated into the required 14 languages is sent home by the school informing parents of the survey activities. The survey requires passive consent, meaning that parents respond to this letter only if they do not want their children to participate in the survey. A completed and signed form must be returned to the school by the designated time (prior to the survey administration window) to opt out of participation.
Will parents have the opportunity to review the survey?
Yes. Hard copies of the Panorama Student Survey will be on file in school offices. These can be reviewed by parents, teachers, and other stakeholders onsite. Survey are not allowed off school premises.
Can a teacher/proctor read the items to a student who is having difficulty understanding the survey?
Teachers/proctors may do the following:
- Read sentences or individual words.
- Define words.
- Read items requested by student(s).
Which teacher will students survey in an inclusion classroom?
If both general education and special education teachers provide direct instruction, contribute to the students' grades and monitor students' progress, both teachers will be surveyed. Each teacher will roster the same class but administer the surveys separately on different days.
What accommodations can be made for students with special needs participating in the survey?
Students will be provided with the documented accommodations identified in their IEPs during the survey administration window. Students who qualify for the HSA-Alt assessment are exempt from participation.
Are students who are language or vocabulary challenged exempt from the survey administration?
No, all students in grades 3-12 will be administered the Panorama Student Survey.
What are the considerations for non-English Proficient (NEP) and Limited English Proficient (LEP) students when completing the survey?
The Panorama Student Survey has been translated into the 14 required languages for Hawaii and English: Hawaiian, Cebuano, Chinese (Simplified and Traditional), Chuukese, Ilocano, Japanese, Korean, Marshallese, Samoan, Spanish, Tagalog, Tongan and Vietnamese. If there is a request for a language translation not identified in the required languages list, it is recommended that schools can contact their respective complex or district personnel for assistance. The student may complete his/her survey using the online English or online translated home language surveys. Students will be provided with the same documented accommodation as they receive for regular classroom instruction.
Can the teacher or proctor read questions or survey items in English if the class is taking the Hawaiian language survey?
Yes, the survey instructions, scripts and items may be read in English to students who are completing the survey in Hawaiian by the teacher or proctor.
Will the Hawaiian language survey packets include the translated scripts and instructions?
No, scripts and instructions are not translated in Hawaiian.
How do you survey a classroom, if the main teacher is out on leave or there isn't a regular teacher?
Surveys are to be administered to classes where students have been with the regular classroom teacher for a minimum of three (3) weeks. If a teacher meets all participation criteria, is registered in the Infinite Campus system as the classroom teacher, participates in roster verification (RV), receives a class pack correctly addressed to him/her, but is out during the administration window, the teacher's class will be surveyed by the substitute or proctor. Students are to be reminded that they are completing their survey for their classroom teacher not the substitute teacher. Please contact your survey coordinator or the EES Help Desk (808-586-4072) for additional clarification.
Can and should a long term substitute be surveyed?
A long-term sub may be surveyed given
ALL of the following conditions:
- Have an Employee ID;
- Taught the class for a minimum of three (3) weeks;
- Projected to be with the rostered class until the end of the school year; AND
- Roster a recommended minimum of ten (10) or more students eligible to take the survey.
Substitute teachers' Panorama Student Survey results will not be used for evaluation purposes but are helpful in enhancing their classroom implementation strategies, support overall school efforts and information that is welcomed by their administration team. Student responses to the survey for a long-term sub will be used to calculate the school's Strive HI school climate measure.
Survey results
Will individual teacher survey information be made publicly available?
No, data for individual teachers are confidential and not publicly released except to their principals in the school report.
How will teachers and principals use survey results to inform professional learning? How do principals help teachers understand the survey results and encourage teachers to apply their learning to classroom practice?
The Panorama Student Survey captures key dimensions of classroom life and teaching practice as students experience them. Surveys can deliver valid, reliable and detailed insights on teaching and learning. Using the Panorama Student Survey, educators have the ability to measure student perceptions in the following areas:
-
Pedagogical Effectiveness
Perceptions of the quality of teaching and amount learning students experience from a particular teacher.
Perceptions of the overall social and learning climate of the classroom.
-
Classroom Rigorous Expectations
How much students feel that a specific teacher holds them to high expectations around effort, understanding, persistence, and performance in class.
How attentive and invested students are in class.
-
Classroom Teacher-Student Relationships
How strong the social connection is between teachers and students within and beyond the classroom.
Findings can inform and promote discussions on school quality and whole-school priorities, as well as focus teacher professional development and student engagement projects. The survey measures enable decision-makers at every level to focus priorities and track progress; helping to ensure investments in professional development and school improvement produce positive results. Principals and teachers will have access to Panorama's resource library and Playbook in the online platform where reports are viewed. Principals are empowered to use the data and resources provided by Panorama Education, Inc. in professional development sessions to encourage improvements in teachers' classroom practice. Teachers are also able to use the data and resources provided by Panorama Education, Inc. to reflect on their classroom practices and formulate/implement processes and strategies to improve the classroom experience for students.