Data Kitchen Tools

Every kitchen needs good cookware and tools. These are the tools that we use in the Graduate College of Education (GCOE) to help us work with our data and process improvement.

Data Dig Directions:

  • First, take some time to closely examine and analyze the data set(s) together.
  • Follow the protocol, but be flexible where needed.
  • Comment in every box.
  • Send your notes to the associate dean – we will bring these back to the next data dig for review.
Table with data protocol indicating steps and discussion prompts
Step Discussion Prompts 

Start with questions: What is our goal? What are we trying to learn?

Is this a new focus, or are you looking at something you changed and want to evaluate?

For example:

  • Program learning outcome
  • Teaching performance expectation
  • GCOE focus area – general or detailed
  • A change made previously
  • Other goals

What do you see and notice?

Make descriptive comments without judgment or interpretation.

“I wonder why…”

“This compared with that…”

“I notice that…”

“One pattern I see is…”

What hypotheses or explanations do you have about what you see? What alternative explanations exist?

Share interpretations of what was noticed based on your experience or knowledge of the issue. Solicit as many possible alternative explanations as possible:

One hypothesis is…

The data seemed to suggest…

One reason for ____ might be _____...

Another reason might be…

On the other hand…

Even though it seems _____, it might be _______.

What questions does this raise for you?

Do you need additional data?

Is there information the team would like to gather?

Any individuals they would like to follow up with to learn more?

I wonder …

One question I have is…

At first, I thought, but now…

What were one or two big thoughts that came out of the conversation?

 Write down your thoughts and then share them among the team/group.

What change might we make and why?

Using your insights, what does the next iteration look like?

Be specific about what you want to change.

Where do you want to move the needle?

Include a time frame.

What data will you use to know whether this was an improvement?

If you’re looking at a change you made already, what data was helpful? What did you see there?

What data could you look at as part of this process?

How might the data change to reflect improvement?

San Francisco State University Data

Bay Area and National Data

Commission on Teacher Credentialing (CTC)

Chancellor's Office Data

Educator Quality (EdQ) Center

California State University (CSU) Data

Institutional Research & Analyses

CSU Students Success Dashboard

The data below is part of the CSU Students Success Dashboard effort. A few dashboards are presented in this section, filtered by college, major and department depending on the selected chart. It includes undergraduate and graduate students' data extracted from sources such as the Enrollment Reporting System (ERS). For more information, please refer to CSU's Enrollment Reporting System Data Dictionary. All the links below require a CSU account.

The education unit develops and implements a comprehensive continuous improvement process at both the unit level and within each of its programs that identifies program and unit effectiveness and makes appropriate modifications based on findings.

Table with a description of CTC common standard four and a description of the requirements.
Common Standards Required Documentation

  • The education unit develops and implements a comprehensive continuous improvement process at both the unit level and within each of its programs that identifies program and unit effectiveness and makes appropriate modifications based on findings.

  • The education unit and its programs regularly assess their effectiveness in relation to the course of study offered, fieldwork and clinical practice and support services for candidates.

  • Both the unit and its programs regularly and systematically collect, analyze and use candidate and program completer data as well as data reflecting the effectiveness of unit operations to improve programs and their services.

  • The continuous improvement process includes multiple sources of data including 1) the extent to which candidates are prepared to enter professional practice; and 2) feedback from key stakeholders, such as employers and community partners about the quality of the preparation.

Graphic depiction of the unit assessment system, including the roles of responsibilities of personnel in the unit and programs.
Annotated list of data sources included in the assessment cycle, including those submitted in annual data reporting and those that are not.

Multi- year unit assessment cycle schedule specifying the unit assessment activities; when they occur and who is responsible for collecting, analyzing and determining modifications.

Annual data submission, analysis and feedback (located in the data warehouse and does not require resubmission) will be reviewed.

Survey Data including CTC sponsored surveys as well as local survey data and/or exit interview data as appropriate.

Process Mapping Protocol (created by the High Tech High GSE Center for Research on Equity and Innovation, Retrieved 3/7/23)

The purpose of this protocol is to better understand the process leading to a particular outcome and identify potential breakdown points where we should focus our improvement efforts.

 

Roles:

  • Interviewee: the person being interviewed who can provide a helpful perspective on the process
  • Interviewer/Facilitator: the person who interviews the interviewee and who facilitates Step 3
  • Process Mappers: one to two people who map the process while listening to the interview

 

Norms:

  • Resist Solutionitis… get your map out first, then interrogate it
  • Share the Air… step up, step back
  • Seek to Understand, not Confirm

 

Step 1: Identify your End Point (5-7 min.)

Before mapping the process, you need to articulate the endpoint (i.e. goal) you are after. 

  • Individually brainstorm (2 min) possible endpoints for the process you want to map. See if you can express it in one short sentence. Examples: A student secures an internship. A student applies to a 4-year college. A teacher plans a project that integrates math.
  • Whip: each person shares one endpoint and the whip continues until all ideas have been shared.
  • Choose one or create a new one (without getting hung up on the perfect wording). 

 

Step 2: Create the Map (10-15 min.)

Using the roles above, the interviewer interviews the interviewee to understand the process (i.e. the reality) leading up to that goal, while others map the process on paper. 

Helpful questions/sentence frames:

  • Start with: So if X is your goal, where do you begin?
  • Then what?
  • Listen for decision points. Is this a decision point? What happens if… (yes)? What happens if… (no)?
  • End by asking: “What was most challenging about the process you just described? What changes could we make to address that challenge?”

 

Step 3: Interrogating your Map & Identifying Change Ideas (15-20 min.)

The interviewee may no longer be present. The rest of the team engages in a discussion about what they heard, starting with a whip, where each person shares one thing that struck them from the conversation. Looking over the process maps, discuss the following as a group:

  • What are we learning about this process? 
  • Was there anything important that we heard that is missing on the process map? (capture it)
  • Where/how might this process breakdown, especially for students from traditionally marginalized groups? 
    • Put an X over those places in your map where the process could breakdown. 
  • What might we do (i.e. change ideas) to improve this process?
    • Write change ideas on your map by the breakdown points.

 

This protocol has been created by the High Tech High GSE Center for Research on Equity and Innovation.