ITAS in a virtual environment – quality and efficacy in Indigenous tertiary student support
Susan Beetson, Annie Tyhuis, Susan Willsteed, Juliana McLaughlin, Sue Whatman
Background to Research Project
Our enrolment figures and success rates are slightly better than the national average (Oodgeroo Unit, 2005), but our ITAS participation rates are quite low (20%) Given our above discussion that ITAS reporting requirements do not illuminate quality teaching and learning practices, it is crucial that we identify and assess the quality of those aspects of student support that are enabling QUT to perform against the national trends of decreasing Indigenous participation in university education.
The emerging interest in the quality of teaching and learning in ITAS were related to broader developments outside of the Oodgeroo Unit. Firstly, it correlated with a decision concerning the University’s commitments to Reconciliation, and its associated Teaching and Learning expectations (see www.reconciliation.qut.edu.au). Through QUT’s commitment to the Reconciliation Statement, a number of faculties, through the provision of Teaching and Learning Grants, undertook projects which concentrated on Embedding Indigenous Perspectives in the Curriculum (Williamson, 2001, Nash, 2004).
From 2003, the Faculty of Education, in consultation with the Oodgeroo Unit, introduced an Indigenous Education unit as one of its core units. An Oodgeroo Unit academic was seconded to conceptualise, develop and co-ordinate this unit. Consequently, all academic staff took on teaching responsibilities from 2004 to 2006. This involved delivering of keynote lectures, tutorials and accommodating consultation hours for Education Faculty students. An associated development was the publication of the textbook, Introductory Indigenous Studies in Education: The Importance of Knowing, edited by Jean Phillips and Jo Lampert (2005), with chapters contributed by most academic staff, to support teaching and learning.
Other faculties within the university that also commenced embedding Indigenous perspectives into the Curriculum at various levels were QUT Carseldine School of Humanities and Human Services (see Williamson and Towers, 2004; Williamson and Dalal, 2007), School of Justice, Faculty of Law (Carpenter, Field & Barnes, 2002), Faculty of Health (Nash, 2004).
Further, the quality of tutors recruited though the ITAS program was a concern for staff at the Oodgeroo Unit (Oodgeroo Unit, 2003). Similarly, the quality of teaching and learning associated with tutorial support provided through ITAS became consistently discussed at other staff meetings (Oodgeroo Unit Staff Retreat Minutes, 2001). A research project was motivated by the need to get a better profile of the integrity of learning and teaching processes that occur. It was intended that this would provide an opportunity to do some quality control, better deliver ITAS to students already accessing this service and market it to students who are not. Some engagement of students’ attitudes and aspirations in learning would be useful to how ITAS is managed at QUT, to improve the quality of the tutoring experience for students to lead to improved student results and retention rates.
An Oodgeroo Unit ITAS project team was appointed in mid 2006. Conversations commenced immediately, which engaged perspectives of both academic and professional staff on the current management and processing of ITAS. One message which emerged consistently was “cut down the paper work” (Academic and administrative staff). A Web Developer, recent QUT graduate from Information Technology, was appointed to assist in the development of an electronic system to better manage ITAS processing. An ITAS team was appointed which brainstormed the existing ITAS workflow processes and the DEST requirements which were documented in a workflow diagram. A staff retreat followed, which allowed Oodgeroo Unit staff to engage in a focus group / workshop to unpack administrative and academic issues being experienced and the how the electronic processing mechanism would enhance the delivery of the program at our university. The outcome of these workgroups was the development of a more improved workflow for the ITAS processes (as will be demonstrated in Session 2).
Research Aims
- How does the Oodgeroo Unit monitor and assess the quality of teaching through the ITAS program and how does the Unit use this information year to year?
- What are the effects on the quality of teaching and learning through ITAS, and its administrative procedures as they relate to students, ITAS tutors, staff and the ITAS coordinator?
In keeping with Indigenous methodology, a specific political, relevant and useful project outcome for the Indigenous community served by QUT is to provide an online interface for ITAS. The current paper based process is a substantially time- consuming and expensive exercise from all users’ perspectives. A review of the history of ATAS/ITAS in the current environment has shown that a standard, consistent and comprehensive means to manage the ITAS process well is essential to its ongoing success, and for developing the means to assess and improve the quality of teaching and learning through the program. Providing a familiar interface utilising existing QUT’s databases, learning tools and authentication system in an online environment is expected to enable a streamlined process with timely attention and reporting to all users including meeting DEST’s reporting requirements.
Research Approach and Methodology
Decolonising methodology
Unlike other studies conducted into ITAS, this research study into both administrative and educational processes at one university location cannot ignore the struggles that exist within Australian university centres which aim to decolonise knowledge. A commitment to decolonising research processes evolved as a way of redressing colonial processes of knowledge generation and its implications of imperialism and knowledge/power relations.
The researchers went into this study with an understanding of Indigenous knowledge and research ethics (AIATSIS, 2000). Indigenous research protocols including respect and responsibility for participants, the ITAS scheme itself and the Unit which administered and supported the program informed the conceptualisation and development of the study from the beginning.
There was an understanding by the research team that there was existing knowledge gained over years of experience in ITAS administrative tasks by both academic advisors and professional staff within the unit. Researchers utilising a decolonising methodology recognise the responsibility that associates with the conceptualisation, conduct of research and dissemination of its outcomes. Smith (1999, p. 139) cautions insider or Indigenous researchers that the research process may challenge and unsettle their beliefs, values and existing knowledge. This research team was highly cautions of this, and prepared itself for different challenges thrown at the project by fellow colleagues, students and the university administrative personnel.
Indigenous scholars, such as Linda Tuhiwai Smith (1999), Polly Walker (2001) and Karen Martin (2001), argued that decolonising methodologies are defined by interconnectedness and interrelatedness between people, knowledge and the natural world. Indigenous research operates on a paradigm that is a creative, ever changing process rather than that in search of an absolute truth. The research process, then, precedes objective measurement, but honours the primacy of direct experiences, relationships, interconnectedness, holism and cultural values (Walker, 2001). Thus, a sense of responsibility to the “researched” informs the conduct and methods of reporting from the study.
Rigney (1997) classified Indigenous research as research being informed by three fundamental and inter-related principles including resistance, political integrity and privileging Indigenous voices (p.118). While the first two principles inform this research through subtle ways, privileging Indigenous voice is fundamental to this study. This study is focused on past experiences, contemporary practices and aspirations of Indigenous educators and student recipients of ITAS. This research team was highly sensitive to, as Rigney (1997) explained, ideas and practices of colonial hegemony. Therefore, there was a high degree of awareness and respect for all participants associated with ITAS and we are accountable to our colleagues and students, the university and Indigenous community.
Case Study
Given the nature and provision of ITAS in Australian universities, TAFE colleges, schools and other learning institutions, this research project is designed to investigate ITAS provision in one location. The rationale for a concentrated focus into one location will enable staff to constantly and critically reflect on their current practice, and develop innovative ideas for enhancing student support through ITAS. A case study methodology is appropriate for such investigation. For the purpose of this study, the two relevant types of case are intrinsic and instrumental case studies (Stake, 1998). Researchers of intrinsic case study aim at obtaining better understanding of the particular case. The case is the primary interest of this study as it concerns program evaluation (Stake, 1998). However, a case study need not be purely descriptive at a micro-level. Intrinsic case study is highly appropriate and relevant for this study. This research is also an instrumental case study (Stake, 1998). In an instrumental case study, a problem is used to provide insight into an issue for refinement of theory (Bassey, 1999). In other words, understanding ITAS at QUT is of intrinsic interest to us to improve our own processes, yet it is also instrumental in that we hope to influence broader policy impacting upon the delivery of ITAS.
Data Collection Strategies
We wanted to honour the growing academy of Indigenous and non-Indigenous scholars who have argued for respectful research with Indigenous peoples (Brady, 1992; Rigney, 1997, 1999; Hart and Whatman, 1998; Huggins, 1998; Smith, 1999, 2005; Moreton-Robinson, 2000, 2004; Weber-Pillwax, 2001; Martin, 2001; Fredericks, 2007; Houston, 2007; Stewart, 2007). Smith (1999, p.120) described such respectful research as “a reciprocal, shared, constantly interchanging principle which is expressed through all aspects of social conduct”. Rigney (1997) argued that research conducted by Indigenous people in a particular way, is “Indigenist” research, and should achieve the privileging of Indigenous voices, political and lobbying agendas, and ultimately the emancipation of Indigenous peoples. While non-Indigenous people cannot conduct “Indigenist” research in Rigney’s view, Smith (1999) and Martin (2001) contended that non-Indigenous people can work with Indigenous people in support of Indigenist research.
Therefore, in adhering to principles of Indigenous methodology, a pragmatic approach guided the selection of methods. Data collection strategies adopted for this research included workshops as a form of focus groups, document analysis, individual interviews, and qualitative survey.
Workshops / Focus Groups
Stewart proposed that focus group methodology was respectful for Indigenous research as it allowed the creation of ideas and new theory to emerge through the flexibility of shared conversations (2007, p.31). Focus groups are a more appropriate methodological approach that “acknowledge the decolonisation politics of an Indigenous research agenda” (Smith, 1999, p.115).
Focus groups were conducted with Indigenous and non-Indigenous staff, both academic and professional, between July 2006 and June 2007. A number of small groups were convened initially in 2006, leading to a whole staff focus group in November 2006, in which the findings of previous sessions were collated and represented back to the staff for further input, clarification and/or rejection. We essentially collated a “wish list” from every staff member outlining their hopes and expectations of ITAS in a virtual environment.
In June 2007, we held the last focus group, presenting the prototype-in-development of the ITAS virtual environment. Feedback from all staff in this session was captured in field notes (see below) to further inform the research project.
Document Analysis
Documents which may form part of the data in a given study may be described as primary or secondary sources (Cohen and Manion, 1994: 50). In this study, we collected meeting minutes and agendas from 2001 to the present, Oodgeroo Unit Annual ATAS/ITAS reports to DEST, the Oodgeroo Unit Submission to the 2003 Review of Bulk funding Arrangements, DEST policy documentation, QUT policy documents, and various government reports.
Individual Interviews
The interviews conducted in this study focused upon key personnel in the administration of ITAS. The ITAS Administrator, former ITAS Coordinators, academic staff, professional staff, students and tutors participated in unstructured, informal interviews about ITAS between July 2006 and June 2007.
Qualitative Survey
The aim of qualitative survey research is to describe characteristics of individuals, groups and/or organisations, with broader aims of monitoring important trends, testing theoretical understandings of social processes and providing key indicators. We wanted to gather the views of a sample of Indigenous students, staff and ITAS tutors at QUT, with a view to identifying key indicators of the quality of teaching and learning in ITAS.
Since the mid-1980s, research into the cognitive aspects of survey measurement has developed at a rapid pace, resulting in an influential presence in survey literature (Presser et al., 2004; Schwarz, 2007). Cognitive Aspects of Survey Methodology, or CASM (Schwarz, 2007, p. 277) is concerned with the cognitive and communicative processes underlying survey responding by participants, with a view to increasing understanding of basic psychological processes, and thus improving the design of surveys.
Survey researchers such as Strack and Martin (1987) and Tourangeau (1984) assume that when a participant answers a survey, several interrelated tasks are required. Schwarz (2007, p. 278) lists those tasks as;
- to interpret the question to understand what is meant and to determine which information should be provided;
- to retrieve a previously formed attitude judgment from memory or form a judgment on the spot;
- to identify the behaviour in question if the survey questions are behavioural, rather than attitudinal; and
- format their judgment to fit the response alternatives provided as part of the question.
They recognise the need for survey respondents to edit their responses before they communicate them and that the performance of the above stated tasks is extremely context dependent and profoundly shaped by the survey instrument (Schwarz, 2007, p. 278). For example, it is more likely that a survey respondent will form a judgment on the spot, than retrieve existing ones from memory, based on whatever relevant information, or “clues”, that are accessible at the time.
With this view of survey methodology, it is essential to describe for this ITAS research project how the purpose of the survey was conceptualised, by whom, and through what tasks the purpose is expected to be fulfilled.
ITAS Survey Purpose and Design
As described earlier, a central data collection method was the focus group. We utilised the focus group approach with university staff involved in the delivery of ITAS to Indigenous students at our university with a view to eliciting from their perspectives the main issues affecting quality and administration of ITAS. From the data recorded at the focus group, which was returned to the group for “member checking” (Carspecken, 1995), the research team designed 6 key questions for each survey respondent group – the students, the professional staff, the tutors and academic advisers – resulting in four separate survey instruments (see Appendices A – D).
Given that survey respondents tend to look for “clues” as to what information they are supposed to provide, and form attitude judgments on the spot (Schwarz, 2007), the introduction to the survey provided at the start for each one is extremely important.
“The management and staff of the Oodgeroo Unit are committed to enhancing the learning experiences of Aboriginal and Torres Strait Islander students through the Indigenous Tutorial Assistance Scheme (ITAS). To be proactive in this campaign, it is essential that the Oodgeroo Unit staff have an understanding of the needs and experiences of tutors currently providing ITAS tuition. The purpose of this questionnaire is to obtain this useful information and to enable us to develop best practice in student support. We anticipate this will lead to improved student outcomes throughout their studies at QUT.
The information you supply will be collated by the Oodgeroo Unit staff for the sole purpose of reviewing the ITAS program with a view to enhancing its efficacy. This information is confidential and shall not be disclosed to unauthorised people or agencies. While an overall profile of ITAS tutors’ experiences of the scheme will be constructed, your individual identities will remain anonymous.
Please complete the attached questionnaire and return as soon as possible”
(ITAS Tutor Survey – Appendices)
In the above instructions/information for respondents, the research team felt it essential to stress the aspects of the ITAS program to be covered by the survey, the ‘ownership’ of the data, the significance of collecting the data in terms of student outcomes, and assurances of confidentiality, despite the identity of tutors being known. This identity issue needed to be clarified as the survey was administered by email, allowing the research team to know the respondent by the return email address, if they chose to return the survey electronically. Opportunities for anonymously completing the survey were also provided by encouraging tutors and students to print off the survey and return them un-named through internal or external mail, or to a collection box provided on each campus.
The Participants
The participants in the study can be categorised into the following groups:
- Aboriginal and Torres Strait Islander students currently using or have previously used ITAS
- Aboriginal and Torres Strait Islander students who have chosen not to use ITAS, or not using it for other reasons (such as tutor unavailability)
- ITAS tutors – current and past
- Current academic staff who appoint tutors
- Current Professional staff who administer ITAS
All participants were interviewed in a focus group setting or surveyed. Follow up ‘member checks’ (Carspecken, 1996), with a majority of staff, students and tutors were included in the research process after the participants had the opportunity to read and/or respond to our interpretations of their views through presentations in two focus groups (November and June) and email correspondence following their surveys. Some respondents also provided their feedback in writing. We supplemented their feedback with additional field notes, recorded during and immediately after the staff presentations.
The following section gives background information about all of the participants. All names are concealed, with identity discerned by an allocated number.
The Students
Twenty-one students participated in the project. There were ninety-five students undertaking ITAS throughout the project, so the student cohort represented 22% of students actively involved in ITAS, and 5.5% of all students enrolled at QUT (n=380). The students were selected via email requests to complete a survey, which was attached to the original request (See appendix). Academic staff also suggested potential students who were frequent, new or past users of ITAS.
The Tutors
Twenty-one tutors participated in the project. There were sixty-eight tutors employed during the project, so the response rate equated to 30%. The ITAS tutors were selected by a deliberate targeting process, through nomination by academic staff who were responsible for appointing tutors, and through the random return of surveys which were sent to all currently employed tutors over email (see Appendices). By targeting, we specifically mean that once the survey was sent out on email, we approached particular individuals to ask if they had seen the email, and were intending to complete it, as we were very keen to ascertain their point of view.
The Academic Staff
The academic staff who participated in the study volunteered to attend two ITAS Workshops, which formed the bulk of the focus group data collected, and some were also members of the research team. A total of six academic staff participated in the workshops.
The Professional Staff
The Professional staff who participated in the study were targeted by the research team and two were also members of the research team. All Professional staff were asked to complete a survey over email (see Appendices) and some volunteered to attend the ITAS workshop/focus group as well. Two professional staff completed the surveys and three staff participated in the focus group.
Key ‘Informants’
We also involved several key informants, with extensive knowledge of the history and administration of ITAS, including the Unit Manager (Victor Hart), previous ITAS Coordinator (Maureen Ah Sam), and for specific, pre-Bulk Funding Arrangements information (1990-1995), a Queensland DEST Officer (Bev Woodward).
Confidentiality
We were well aware that the study's data, and interpretation of such data, risked exposing the identity of the participants and could have led to censure by their peers and colleagues. Indigenous research ethics (Donovan and Spark, 1997; AIATSIS, 2000; NHMRC, 2003) dictate that participants reserve the right to reveal or conceal their identity - indeed, to be able to control how their identity is constructed in the text (Hart and Whatman, 1998).
For these reasons, a number of strategies were employed, at two levels, to remain sensitive to the needs and rights of the participants. Firstly, when prompting a response to a question posed in focus groups about the views and actions of other participants, we attempted to provide as much contextual information as was required without identifying the source. For example, questions were phrased as ‘some of the students commented...’ or ‘some of the academic staff noted that…’, so that we would avoid becoming an informant of another’s identity.
At another level, individual participants have been protected by the use of pseudonyms and numbers. While readers who are knowledgeable of the case study site and its people may recognise individual identities through deduction or their existing relationships, the use of such pseudonyms provides the confidentiality required to present the data to a wider audience, with permission, without causing harm or distress to the participants, and without distorting the meaningfulness of record. At the time of data collection, a number of participants indicated that they would be happy to have their identities included in the final text if necessary. For example, students and tutors had the option of returning their surveys anonymously, but most chose to return their surveys electronically via email, which meant we had a record of their identity. However, a conscious decision has been made not to reveal their identities for the sake of consistency.