Observe
Data Tip #4: "Make Data Observations: Before Identifying Solutions, Get All the Facts on the Table"
Set aside assumptions, and focus on just the 'data facts' before leaping to explanation and interpretation."
(Love, Nancy et al. The Data Coach’s Guide to Improving Learning for All Students, 2008.)
Teachers are natural problem solvers. When we see evidence in our data that groups of students are underachieving, we are anxious to find solutions. But data analysis is most effective if a team takes the time to observe and record as many details as possible about what the data reveal. The Using Data process advocates a “hold your horses” mindset that can help teachers to better pinpoint a student learning problem before jumping to explanations, interpretations, and quick-fix solutions.
Observe is the third stage in a 4-phase dialogue process* that guides deep discussion toward uncovering accurate meaning from the data. (See more information about Step 1: Predict and Step 2: Capture Predictions.)
The Observation phase of the 4-phase dialogue process requires strong discipline! Assign a group dialogue monitor to avoid moving the discussion too quickly to “because” and “we should.”
Observations might start with phrases such as “I notice that…, I see that…, I’m struck by…., I’m surprised that…”
Sample Observations
What makes a good observation statement? Here are some questions to guide you to make refined and specific data observation statements:
- Does each statement communicate a single idea about student performance?
- Is the statement short and clear?
- Does the statement incorporate numbers (the data)?
- Does the statement focus just on those direct and observable facts contained in the data, without explanation or inference?
- Does the statement use relevant data concepts such as mean, median, mode, range, or distribution?
Depending on the type of data you are looking at, the observations might resemble the examples below.
Sample aggregate data observation:
I notice that at the school level, student performance in math increased from Year 1 to Year 2 (44 percent to 47 percent) and then declined in Year 3 (to 33 percent).
Sample disaggregate data observation:
I see that in the most recent year of data at the school level, 44 percent of sixth-grade African American students performed at the lowest performance level in English language arts, compared with 36 percent of Hispanics and 35 percent of white students.
Sample student work data observation:
I’m surprised that our regular education and special education students had the same difficulty with the vocabulary used in this open response science question.
Follow these action steps to discover all the facts your data can reveal as your data team makes observations.
Action Steps
- First, gather your data team members. They might be a grade-level or vertical team, a subject-area department, or your school leadership team.
- Together, study a visual representation of the data you want to analyze. Allow some quiet “think time” to allow members to digest and make sense of what they see. Provide some think-time prompts, such as:
--What important points seem to pop out?
--What are some patterns and trends that are emerging?
--What seems surprising or unexpected? - Share observations about the data. Stick to “just the facts.” A round robin brainstorming strategy works well when making data observations. It encourages all data team members to look closely at the data and have a voice at the table.
- Capture each observation on chart paper. Continue the process until all possible observations are surfaced and captured.
After capturing a complete set of observations, now the team is ready to generate possible explanations for what they observed. Our next Data Tip will discuss Step 4 in the 4-phase dialogue process: Making Inferences.
Written by: Diana Nunnaley, Director
Mary Anne Mather, Facilitator TERC's Using Data
