What are we collecting data for?
Reflexively engaging with data collection in education.
We are drowning in a sea of data and starving for knowledge - Sydney Brenner
Listen to the podcast segment of Ideology in Education (at 6:21) based on this blog post for the Teacher’s Education Review below 👇.
Data burdens.
The start of a new teaching year.
As a year 7 mathematics teaching team it was decided that we would begin each topic with an electronic pre-test and end with a post-test (both identical to one another). These tests were designed by the company who developed the textbook that we used for year 7 mathematics and were aligned specifically to the layout of the content explored in each chapter of the textbook. The idea was that by completing tests before and at the conclusion of a topic, teachers would be able to (hopefully!) see the growth of their students over the course of their teaching program. At the beginning of the year, I remember having a graduate teacher excitedly email me through their meticulously designed Excel spreadsheet, with columns for pre-test and post-test results, along with a column dedicated to showing the percentage growth of students. The percentage growth cell would change colour to brighter shades of green as the percentage difference increased.
Fast forward to Term 3.
I no longer collect this data.
My colleague’s beautiful Excel spreadsheet has a number of empty columns now.
Over the first two terms I found that:
Importing and analysing the data became laborious.
Students were getting sick of sitting so many tests.
Students were often away during pre-test or post-test sessions.
Even with explicit expectations, students would often either rush through the tests to complete them or not put in their best effort.
The growth results were often not surprising.
But most importantly, it was never really clear exactly what teachers were to do with this data or how this practice aligned with our goals as a school. Sure, I could see growth in my students in numerical form, but wasn’t this something I could already judge from my interactions with students in the classroom, along with my formative and summative assessment practices? Besides a performative insurance on the effectiveness of my practice, this data seemed benign. Essentially, the collection of this data became a burden on my practice. So, being a teacher that must make a sea of decisions based on judgments of what is going to be of most benefit for my, as well as my student’s time, it was time to call it in.
Data is the new gold.
This is the way things seem to go in schools, right?
There are many policy-makers, leaders, and teachers captured by the idea that if we simply had more data, our educational outcomes would be greater, teachers would be better and social inequalities in education would lessen (Fox & O'Connell, 2016; Lingard et al., 2015). Businesses, institutions, and organisations are implored to see data as the “new gold” (O'Halloran & D'Souza, 2020), with education systems following suit, deifying data as a pure representation of educational quality (Hardy & Lewis, 2017). Faith in data, along with the desire to collect increasing volumes of data, has arguably led to the development of what the educational philosopher Gert Biesta (2010) describes as “measurement culture” (p. 10), where league tables of performance in a rather narrow range of educational outcomes have come to profoundly impact policy and practice. In the Australian context, the National Assessment Program - Literacy and Numeracy (NAPLAN) highlights this trend, which “has encouraged whole-school practices that are productive of, and highly responsive to, performance data” (Hardy & Lewis, 2017, p. 2). Furthermore, NAPLAN results are often used to present crises in need of urgent education reform (Australian Broadcasting Network, 2023). Teachers are told that they need data in order to teach well. For example, Victoria’s Department of Education (2021) states that “methodical analysis of assessment data provides the evidence a practitioner needs to improve teaching and learning for the group and individuals within it” (para. 1, emphasis added). Data collection is routinely utilised as an important step in the teaching and learning improvement process across Australian states and territories, along with moves in Australian initial teacher education programs to develop data-literate teachers (McDowall et al., 2021).
It is no wonder then that school leaders and teachers feel a compulsion to collect growing amounts of data of their students, including their performance and growth in relation to prescribed curriculum outcomes. Little time, however, seems to be given to examining whether such data practices are in fact desirable and in the best interests of our schools and communities.
Bending back.
Within the context of Australian education, data is often collected as a means of determining performance of students in meeting prescribed outcomes (Department of Education, 2023). Data collection for these purposes feeds into the prevailing narrative that considers performance, competition and effectiveness as ideologically representative of “good” education (Greentree, 2021). Sadly, the way that data is used to chastise or congratulate education systems has led to some rather “perverse” effects, where performance data is “gamed” in order to preserve performative perceptions of the “good” school (Lingard & Sellar, 2013).
But is this really what we want happening in our schools?
Engaging in a process of reflexivity, whether individually or collectively, may serve as an important tool to support educational leaders, teachers and schools to align their data practices with their contextual school values and needs. Reflexivity is a particular kind of reflection that transcends technical aspects of a teacher’s work to examine the values directing educational practice (Ryan & Webster, 2019). Although reflection is often interested in questions of practical matters (that is, “how well does our data predict student achievement?”), reflexivity involves considering the values upon which our practices are conducted (“who benefits from our data collection practices?”). Reflexivity is the mental “bending back” upon one’s practice and beliefs (Archer, 2009, p. 2), which requires the individual “to consider themselves in relation to their (social) contexts and visa versa” (Archer, 2007, p. 4). By examining the ways in which structural forces, such as ideologies, impact on the practice of schools and teachers, reflexivity can support the possibility for alternative actions (ibid.).
A reflexive attitude to data collection asks probing questions, such as:
Why are we collecting data?
What data is being collected?
Who are we collecting data for?
What are we collecting data for?
Microsoft Teams has a class function that teachers can access for their classes, titled “Insights”. Microsoft states that:
Insights in Microsoft Teams uses at-a-glance data views to help educators track their students’ Teams activity, from assignment turn-in to engagement in class conversations. With spotlighted datapoints and visualizations to keep educators informed on trends in student activity and growth, the Education Insights dashboard is designed to save you time in planning, giving feedback to students, and providing help where needed (Microsoft, 2023, para. 1).
Discovering this function some time ago, my initial reaction was positive. I thought to myself, “what a great way to see how students are engaging with the content that I was spending so much time preparing”. However, shortly after I realised the kind of data that I was able to collect on my students made me feel a little uneasy. I took some time to reflect on the why, what and who of this data collection practice, which revealed some uncomfortable truths for me. I found that this data served mostly as another way for me as the teacher to gain control, to have mastery over the actions of my students.
There is not necessarily a single answer to this. Different leaders and teachers will have varying responses, depending on their context.
I don’t expect that others see this function in the same way that I do. It is the disposition of reflexivity that is important here, as each will come to varying conclusions depending on their contexts and beliefs surrounding the collection of data in this way. It is about considering whether our data collection practices have “simply been absorbed unconsciously or fatalistically from the dominating ideologies of society and whether they are authentically chosen by us because we have critically judged them to be of value” (Ryan & Webster, 2019, p. 73, emphasis added).
It is taking the time to interrogate the why, what and who of our data collection practices so that we may ensure that we’re aligning them in such a way to achieve desirable goals in education.
No data will be able to make this decision for us.
Till next time,
References
Archer, M. S. (2007). Making Our Way Through the World : Human Reflexivity and Social Mobility. Cambridge University Press.
Archer, M. S. (2009). Conversations about Reflexivity. Taylor & Francis Group. http://ebookcentral.proquest.com/lib/deakin/detail.action?docID=465428
Australian Broadcasting Network. (2023). One in three school students not meeting numeracy and literacy expectations. Australian Broadcasting Corporation. Retrieved 8th November from https://www.abc.net.au/news/2023-08-23/one-in-three-school-students-not-meeting-naplan-expectations/102763936
Biesta, G. (2010). Good Education in an Age of Measurement : Ethics, Politics, Democracy. Taylor & Francis Group. http://ebookcentral.proquest.com/lib/deakin/detail.action?docID=4186085
Department of Education. (2021). Analysing and Using Data. State Government of Victoria. https://www.education.vic.gov.au/school/teachers/teachingresources/practice/Pages/insight-data.aspx
Department of Education. (2023). Data and research for schooling. Australian Government. Retrieved 12th November from https://www.education.gov.au/data-and-research-schooling#:~:text=School%20data%20is%20collected%20by%20Australian%20Curriculum%20Assessment,base%205%20Deliver%20effective%2C%20evidence-based%20policy%20and%20programs.
Fox, S., & O'Connell, M. (2016, 8th November). Gaps in education data: there are many questions for which we don’t have accurate answers. The Conversation. https://theconversation.com/gaps-in-education-data-there-are-many-questions-for-which-we-dont-have-accurate-answers-65241
Hardy, I., & Lewis, S. (2017). The ‘doublethink’of data: Educational performativity and the field of schooling practices. British Journal of Sociology of Education, 38(5), 671-685.
Lingard, B., & Sellar, S. (2013). 'Catalyst data': perverse systemic effects of audit and accountability in Australian schooling. Journal of Education Policy, 28(5), 634.
Lingard, B., Thompson, G., & Sellar, S. (2015). National Testing in Schools : An Australian Assessment. Taylor & Francis Group. http://ebookcentral.proquest.com/lib/deakin/detail.action?docID=4185863
McDowall, A., Mills, C., Cawte, K., & Miller, J. (2021). Data use as the heart of data literacy: An exploration of pre-service teachers’ data literacy practices in a teaching performance assessment. Asia-Pacific Journal of Teacher Education, 49, 487-502. https://doi.org/10.1080/1359866X.2020.1777529
Microsoft. (2023). Educator's guide to Insights in Microsoft Teams. Retrieved 8th November from https://support.microsoft.com/en-au/topic/educator-s-guide-to-insights-in-microsoft-teams-27b56255-90c0-47aa-bac3-1c9f50157181
O'Halloran, D., & D'Souza, F. (2020, 7th November). Data is the new gold. This is how it can benefit everyone – while harming no one. https://www.weforum.org/agenda/2020/07/new-paradigm-business-data-digital-economy-benefits-privacy-digitalization/
Ryan, A., & Webster, R. S. (2019). Teacher Reflexivity: An Important Dimension of a Teacher's Growth. In W. R. S & J. D. Whelan (Eds.), Rethinking reflection and ethics for teachers.
Once again a spot on post. Your observation that students are 'sick of sitting so many tests' is critical. The ONLY point of data, is if it improves outcomes for students, which means it has to meet their emotional needs as well as learning needs. As soon as it doesn't then it serves no purpose. You don't need it to teach well. If teachers are trusted, administrators don't need it for monitoring purposes. The only data I collected was a 'demonstration of understanding of a concept' made DURING the process of learning. That process of learning for my students was always embedded in an interesting task so was not an extra job they had to do such as a test. The recognition of demonstration of understanding was very low key, resulting in a simple ticking/colouring in/recording made in consultation with the student. Multiple demonstrations meant that a student understood the concept. That was what I used to design their next learning.