I’ve been thinking about my Doctoral thesis quite a bit recently, mainly as a lot of performance conversations I’ve been having are around ‘impact’; an area I focused on somewhat in my thesis. The challenge with impact is how do we measure it? How can we objectively or subjectively define the impact we have had in a project or piece of work we’ve been focussing on? A challenge for all industries, and one certainly faced in the high performance sport industry, where support staff are regularly asked to provide a measure of their ‘impact’ to world class performance programmes. So here are some thoughts on using different methodologies for measuring impact based on my experiences in high performance sport and with my Doctoral thesis…
It’s not all about the numbers: Qualitative vs. quantitative methods when assessing impact
Impact. Hard to objectively and definitively measure. Particularly in providing support to athletes and coaches. How do you measure the ‘impact’ you’ve had? Whether it be through innovating interventions for performance improvement(s), being the subjective ear piece for athletes and coaches to work their way to solutions, or simply providing that performance confirmation marker at key points throughout a season. Creating impact and assessing its effect is a hard concept to master. Even more so when you accept that sports science practitioners ‘operate in a dynamic and volatile environment of accountability, where it is difficult to provide effective measures of success and impact quickly’ (Thompson, 2010). Therefore, whilst physiological assessment and sports performance itself can aid athlete’s and coach’s evaluation of the impact of physiological support, measuring the impact of such support on performance, in relation to the practitioner involved, can be ambiguous.
This conundrum is where I found myself in writing up my Doctorate thesis on ‘evolving the concept of sleep and down time in high performance sport’. I found I needed a measure of ‘impact’ that couldn’t be provided through quantitative means. Whilst sleep is fundamentally a physiological process, the focus of my work wasn’t on the improvement of objectifiable sleep architecture (although this did feature), but more on the impact of behaviour change, raising awareness, education and dissemination of information throughout the high performance sport network. Tricky to measure objectively.
So I braved it and went off-piste. Through the help of a fabulous expert in the field, I employed some qualitative analysis techniques within my doctorate, which supported the more traditional, quantifiable aspect of the thesis. This meant I produced a much more autoethnographical (self-reflective) and interpretivist (empathetic) thesis. Being traditionally more familiar with a positivist form of research i.e. obtaining quantifiable data to interpret and report, this was a particular challenge. However, as I discovered, through adopting a more interpretivist approach, this allowed me to reflect and present a more considered perspective when exploring of the impact of evolving a concept within the high-performance sport industry.
Despite being a little unfamiliar to me, I was confident in employing qualitative techniques as part of my thesis given the methodology has gained increasing popularity across the sport and exercise domain in recent years. This is mainly due to the acceptance of the value that a single or variety of qualitative research methods can achieve to explore phenomena. Certainly, qualitative methodologies provide a more holistic, flexible and naturalistic enquiry which suited certain aspects of my thesis. Furthermore, they offer purposeful perspectives from individual and personal insights, whilst also being process driven.
I also had to consider the type of qualitative analysis tool to employ. Of the many and creative aspects of qualitative research design (e.g. single interviews, focus groups, participant observation, photo-storytelling, write and draw), I used the ‘interview’ technique. This was principally because this format is perhaps the most fundamental and common data collection technique in the qualitative domain. Also, during my ‘reconnaissance phase’, where I gathered intelligence on what could help me provide a measure of impact from my work, the interview format seemed the most fitting for my needs.
Certainly using purposely selected individuals for interview, provided insight, content and context through representations in the form of verbatim quotes and constructed narratives regarding the focus of my thesis. This subsequent data representation, via exact quotes, on evolving the concept of sleep and downtime, added an extra depth for the thesis and crucially, was presented with a comprehensive understanding of what sleep is and current findings in the sport science literature. This qualitative methodology also assisted in encapsulating and understanding the impact of evolving the concept of sleep and downtime to the UK high performance sport industry, much better than my personal observations alone. A more detailed methodology of my qualitative techniques is not for this platform (should you wish to discuss it, feel free to get in touch), and whilst the learning points from the thesis were many, without doubt the process brought home to me how qualitative methods can enhance practitioners’ effectiveness by helping to provide a measure of impact. I’d urge you to climb out of your comfort zone, and explore this paradigm if you’re looking to find ways of displaying impact and adding context, illustration, substance and depth around any piece of work you’re undertaking.
Dr Sarah Gilchrist, December 2019
Reference: Thompson, K. G. (2010). Being an elite sports scientist: a balancing act? International Journal of Sports Physiology and Performance, 5, 12