Student Evaluations in Higher Education and Universities

While student evaluations or ‘happy sheets’ become routine in higher education and universities, some question both effectiveness and efficiency in using such instruments to assess quality. Further, what is quality in teaching, learning, assessment, technology, administration and student well-being, then how and when should it be applied?

Student feedback and evaluations in higher education

Student Experience Feedback (Image copyright Pexels)

From the AIM Network Australia:

Mutual Decline: The Failings of Student Evaluation

November 30, 2018 Written by: Dr Binoy Kampmark

That time of the year. Student evaluations are being gathered by the data crunchers. Participation rates are being noted. Attitudes and responses are mapped. The vulnerable, insecure instructor, fearing an execution squad via email, looks apprehensively at comments in the attached folder that will, in all likelihood, devastate rather than reward. “Too much teaching matter”; “Too heavy in content”; “Too many books.” Then come the other comments from those who seem challenged rather than worn down; excited rather than dulled. These are few and far between: the modern student is estranged from instructor and teaching. Not a brave new world, this, but an ignorant, cowardly one.

The student evaluation, ostensibly designed to gather opinions of students about a taught course, is a surprisingly old device. Some specialists in the field of education, rather bravely, identify instances of this in Antioch during the time of Socrates and instances during the medieval period. But it took modern mass education to transform the exercise into a feast of administrative joy.

Student evaluations, the non-teaching bureaucrat’s response to teaching and learning, create a mutually complicit distortion. A false economy of expectations is generated even as they degrade the institution of learning, which should not be confused with the learning institution. (Institutions actually have no interest, as such, in teaching, merely happy customers.) It turns the student into commodity and paying consumer, units of measurement rather than sentient beings interested in learning. The instructor is also given the impression that these matter, adjusting method, approach and content. Decline is assured…

…Education specialists, administrators and those who staff that fairly meaningless body known as Learning and Teaching, cannot leave the instructing process alone. For them, some form of evaluation exercise must exist to placate the gods of funding and quality assurance pen pushers.

What then, to be done? Geoff Schneider, in a study considering the links between student evaluations, grade inflation and teaching, puts it this way, though he does so with a kind of blinkered optimism. “In order to improve the quality of teaching, it is important for universities to develop a system for evaluating teaching that emphasises (and rewards) the degree of challenge and learning that occurs in courses.” Snow balls suffering an unenviable fate in hell comes to mind.

Student feedback or evaluations are an essential part of assessing, maintaining and improving quality in education and training.  However, much research and expertise is required for such instruments to be used optimally for positive outcomes.

For more articles and blogs about higher education teaching, CPD continuing professional development, enrolled student feedback, evaluation, student satisfaction and university teaching skills, click through.

 

Advertisements

Skills of Critical Thinking

Featured

Critical thinking and related literacies are viewed as essential soft, work or life skills to be taught and learnt by school students, apprentices, trainees, university students, employees and broader society, but how?

Following is parts of an article from The Conversation focusing upon argumentation, logic, psychology and the nature of science to help people understand and analyse the world round us in an age of fake news, conspiracy theories, anti-science and anti-education sentiments.

‘How to teach all students to think critically

December 18, 2014 2.27pm AEDT

All first year students at the University of Technology Sydney could soon be required to take a compulsory maths course in an attempt to give them some numerical thinking skills.

The new course would be an elective next year and mandatory in 2016 with the university’s deputy vice-chancellor for education and students Shirley Alexander saying the aim is to give students some maths “critical thinking” skills.

This is a worthwhile goal, but what about critical thinking in general?

Most tertiary institutions have listed among their graduate attributes the ability to think critically. This seems a desirable outcome, but what exactly does it mean to think critically and how do you get students to do it?

So what should any mandatory first year course in critical thinking look like? There is no single answer to that, but let me suggest a structure with four key areas:

 

Argumentation

The most powerful framework for learning to think well in a manner that is transferable across contexts is argumentation.  Arguing, as opposed to simply disagreeing, is the process of intellectual engagement with an issue and an opponent with the intention of developing a position justified by rational analysis and inference.

 

Logic

Logic is fundamental to rationality. It is difficult to see how you could value critical thinking without also embracing logic.  People generally speak of formal logic – basically the logic of deduction – and informal logic – also called induction.  Deduction is most of what goes on in mathematics or Suduko puzzles and induction is usually about generalising or analogising and is integral to the processes of science.

 

Psychology

One of the great insights of psychology over the past few decades is the realisation that thinking is not so much something we do, as something that happens to us. We are not as in control of our decision-making as we think we are.  We are masses of cognitive biases as much as we are rational beings. This does not mean we are flawed, it just means we don’t think in the nice, linear way that educators often like to think we do.

 

The Nature of Science

Learning about what the differences are between hypotheses, theories and laws, for example, can help people understand why science has credibility without having to teach them what a molecule is, or about Newton’s laws of motion.  Understanding some basic statistics also goes a long way to making students feel more empowered to tackle difficult or complex issues. It’s not about mastering the content, but about understanding the process.’

 

This article is from 2014, however it is unclear what Federal and State Education Departments are doing to include the explicit teaching and learning of critical thinking skills to students via curricula and syllabi?

For more articles about university teaching and learning skills click through.

 

 

 

 

How Should Digital Marketing be Taught?

Featured

Marketing has changed significantly in the past ten or fifteen years due to digital channels and more services versus products.  This impacts not just what expertise or qualifications lecturer, teacher or tutor needs, but how marketing is taught in the classroom and coursebooks used e.g. inclusion of relevant digital content and a change in concepts.  Digital communication technology leverages word of mouth and horizontal communication while precluding control of messages.

Teaching, tutoring or lecturing for digital marketing

Digital Marketing Requires Different Teaching and Lecturing Expertise

Following is an introductory summary to teaching marketing round digital concepts in the classroom:

Marketing Digital Offerings Is Different: Strategies for Teaching About Digital Offerings in the Marketing Classroom.

Scott D. Roberts The University of the Incarnate Word, San Antonio, Texas, USA Kathleen S. Micken Roger Williams University, Bristol, Rhode Island, USA

‘Digital offerings represent different challenges for marketers than do traditional goods and services. After reviewing the literature, the authors suggest ways that the marketing of digital goods and services might be better presented to and better understood by students……. The authors also present specific suggestions for assignments and class discussions to foster students’ critical thinking about the marketing implications surrounding digital offerings.

When the U.S. economy shifted away from its manufacturing base, services marketing theory arose to help marketers deal with the unique nature of the increasingly intangible offerings (Berry 1980). More recently, the economy has shifted again, driven by digital technologies. Not only have products been digitized, but information and communication technologies have also made it possible to distance producers from consumers, both in space and time. Marketing practice has responded to this environmental change, but academic marketing thinking has not come as far.

We first became aware of the problem while teaching MBA students concentrating in digital media management. For their marketing management course, we used Kotler and Keller’s (2009) Marketing Management. Kotler’s work has arguably been one of the central repositories of marketing’s received theories and ideas. We quickly realized, however, that the discussion of the digital offerings that these students were so engaged with (film, music, and video games) was lacking……

…..How has the marketing discipline responded? Our purpose here is not to suggest that there has been a dearth of literature about the impact of digital technology but rather that there are significant gaps in the literature about how to address digital offerings conceptually….What is missing, however , are pedagogical proposals for teaching about the challenges of marketing digital offerings.

The need to fill this gap comes not only from marketing practice, but also from accrediting bodies. The 2013 Association to Advance Collegiate Schools of Business (AACSB) standards man date that business programs include learning experiences that help students understand the integration of information technology in business…. Clearly, it is time to equip our students with tools for understanding and embracing all things digital. And it is time to equip faculty with the tools to do so. Faculty are faced with students for whom digital offerings are pervasive, yet who need to learn how to market those offerings strategically…..

IHIP framework of Intangibility, Heterogeneity, Inseparability and Perishability

This IHIP paradigm, however, did not anticipate digital offerings. At its core, a digital offering is made up of data files (recorded ones and zeros) stored on either the drives/media of consumers or on the servers of marketers/facilitators (e.g., in the cloud). These files come together in the form of solutions (bundles of benefits) for consumers.

Many traditional offerings have become available digitally including maps, tax preparation, customer service, reference sources, higher education, and distance medical consulting….. when applying the IHIP framework to digital offerings, some significant differences arise, both in terms of the features of the offerings as well as the attendant marketing challenges…..

Digital technologies have become ubiquitous in marketing. In adjusting pedagogy to acknowledge these changes, marketing faculty have begun to incorporate more technology in the classroom, have begun to address the new options available to marketers for engaging with customers, and in some cases have created not only new courses but also new majors/concentrations.

External forces also propel this movement forward: accrediting agencies and organizations seeking interns and employees who understand the technology as well as how to use it strategically.

The production of unifying marketing frameworks has not always kept pace with the speed of digital business evolution, and thus marketing texts are not providing timely structures for conceptualizing these changes. This paper suggests ways faculty can effectively use the existing services marketing IHIP framework, but also presents the deviations from it necessitated by digital offerings.

Additionally, we offer suggestions for assignments and discussion probes to augment faculty presentations. Faculty may find the suggestions here helpful in organizing their own thinking about these issues, which in turn will help move the discipline forward.’

 

Kotler has recently published a related book ‘Marketing 4.0’ see Digital versus Traditional Marketing.

Click through for details about SEO search engine optimisationdigital marketing and teaching or learning.

ID Instructional Design Models in Education

Cognitivism and Connectivism Learning Theory page as part of an EdX Instructional Design course.

Cognitivism is student centred learning via an existing knowledge base and building upon it according to learner preferences, how they organise memory, how information is linked, learning how to learn, problem-solving and the student learning journey is supported by clear instructions and information (Hanna, 2017).

Further, there is the Three-Stage Information Processing Model including Sensory Register to assess inputs, Short-Term Memory where input can be stored e.g. 20 seconds and then Long-Term Memory and Storage retrievable by linkages that have been developed (Mergel, 1998).

Application of andragogy for adult learning versus pedagogy for school.

Adult Learning Theories in Higher Education (Image copyright Pexels)

Connectivism is like social learning through others or networks, identify patterns, knowledge based round networks and exemplified in complex learning e.g. round information and technology (Ibid.).

Both can be used for the same education and exemplars, by using both theories to support instructional design, student centred activity and learning, building upon knowledge and experience for inexact outcomes; as opposed to behavioural focus.

In the first case, cognitivism using a course e.g. ‘Introduction to Digital or e-Marketing for Small Business’,  focus upon one learning outcome, ‘ability to analyse (digital) marketing and communication’

Rather than present information or content activities which maybe new and/or overwhelming, assess the knowledge level before training, then drive instruction and achievement of learning objectives via learners and learner centred activity (but monitored an assessed closely).

Instructional Design for Adult Learners in ‘Introduction to Digital or e-Marketing for Small Business’:

Preview by using images to elicit key words, channels etc. related to conventional marketing and communication.

Presentation repeat preview to include digital also and elicit the elements.

Practice by learners listing both types of elements in a small business example marketing and communications; report back to class.

Production in pairs for their own business, assist each other, compare notes then present to each other/class.

Wrap-up Class discussion and/or milling activity to compare with other learners’ ‘production’ and feedback on key points, rules or issues.

Connectivism can be applied to the same course area and learning outcome, not just in the direct learning environment but post learning, i.e. back in the workplace and business environment.  Accordingly, if learners are mostly small business people, already responsible for marketing and communications and sharing a desire to improve application of digital in their business practice, they should be motivated for connectivism.

Within the formal learning, connectivism would fit cognitivism approach above, with symmetry in each phase, but especially with increase in learner interactivity with production and wrap up or review.  Connectivism can then also be followed up informally by learners remaining in communication with each other (e.g. WhatsApp or LinkedIn Group), industry sector networking opportunities and/or local chamber of commerce.

Andrew Smith Melbourne LinkedIn Profile

 

Reference List (Harvard):

 

Hanna, M. (2017) Learning Theory Matrix. Available at: https://pdfs.semanticscholar.org/8d28/2833c35fb8b9ea74bf2c930cea22fb1e0fad.pdf (Accessed on: 16 November 2017).

Mergel, B. (1998) Instructional Design & Learning Theory.  Available at: http://etad.usask.ca/802papers/mergel/brenda.htm#Cognitivism(Accessed on: 17 November 2017).

 

TAE40116 Certificate IV Training Assessment Package – ASQA Review Submission

Submission for TAE40116 Training Package Review

 

Written by Andrew Smith; submitted 3 April 2018

Introduction

There has been much discussion amongst training practitioners about the updated TAE Training Package.  One of the main issues has been the perception that it has been designed for quality administration and assessment while neglecting quality of actual training delivery and learning.

ASQA Australian Skills Quality Authority Certificate IV Training and Assessment TAE40116 Review

TAE40116 Certificate IV in Training and Assessment – ASQA Review (Image copyright ASQA)

This has been experienced by the writer currently upgrading BSZ to TAE via a registered training organisation (RTO) by distance learning; PO Box with ‘assessors’ and ‘trainers’ based offshore.  Further, the delivery is based upon basic pedagogy of presentation of content, regurgitation of content according to instructions while seemingly unable to offer explanations or insight for trainees, especially delivery and learning skills based upon andragogy.

 

What is the TAE Training Package?

 

Description

 

This qualification reflects the roles of individuals delivering training and assessment services in the vocational education and training (VET) sector.

This qualification (or the skill sets derived from units of competency within it) is also suitable preparation for those engaged in the delivery of training and assessment of competence in a workplace context, as a component of a structured VET program.

The volume of learning of a Certificate IV in Training and Assessment is typically six months to two years. (Department of Education & Training 2018).

At face value the TAE40116 appears to be a relevant and practical qualification for vocational education and training to deliver accredited training packages, assure quality with a focus upon assessment.  However, there have been many criticisms of the package from practitioners, industry and other stakeholders, why?

Issues with TAE Training Package and Delivery

 

According to the Resources Training Council

 

It has become a qualification for the training industry, not industry that trains. They do not understand workplaces where training and more importantly the outcomes (assessment) must be fit for purpose to achieve what VET is all about.  VET should be about producing safe, proficient (productive) workers and providing an opportunity for learning to be built on as people move along their chosen career path (Munro 2017).

 

From an experienced VET training practitioner

 

To improve assessment practices of RTOs and improve skills and knowledge of trainers and assessors we need to:

  • Update our regulatory framework and move to a real outcome-based regulation, where relevant industry stakeholders have a say in the registration/re-registration process
  • Support the National Regulator in building the required capabilities to assess compliance in a diverse and complex environment
  • Ensure the National Regulator provides an even-playing-field to RTOs
  • Identify the different issues within the assessment system, and consequently identify gaps in current workforce skills (in all AQF levels not only entry level) and update the TAE training package accordingly (Castillo 2016).

 

Research criticisms have included: one size fits all approach whether novice trainer or an already well experienced and/or qualified trainer or teacher, trainer assessor expertise or skills, questions over subsequent assessment outcomes, lack of depth related to training and learning delivery, no clear development pathway and the skill outcomes from the TAE for practitioners to deliver (Clayton 2009).

The focus of criticism has been directed at sub-optimal education and training pedagogy (learning theory for children and young adults or novices) of both the TAE and practitioners for quality delivery, learning and trainee assessment outcomes.  These revolved round, preparedness of trainers to train, opportunity to learn content knowledge, delivery quality, learning the practice of good teaching or training, learning from experts, then more about planning and assessment (Ibid.)

The latter issue of assessment has been raised within sectors whether validation between providers, or simply better understanding of assessment by practitioners (Halliday-Wynes & Misko 2013)

Further, expert input often hints at what is lacking by focusing upon learning theory or ‘pedagogy’ for children and youth, as opposed to ‘andragogy’ for youth and adults.  The latter would be exemplified by self-directed learning or training, responsibility, experience, motivation to learn and preference for real tasks and problem solving (Educators’ Technology 2013); supported by well skilled trainers.

 

Training Delivery and Learning Quality

 

However, delivery of some TAEs has more to do with education and training or ‘pedagogy’ influence from two generations ago manifested in trainer or teacher directed, or top down.  This assumes trainees have no relevant knowledge or practical input to offer, focus upon systems, processes and assessment round any given package, but not delivery i.e. developing quality training and learning skills.  Additionally, very content driven for good reason, however, it is presented or instructed (not elicited) then regurgitated or replicated for satisfying requirements for assessment, then assumed optimal for the workplace?

The significant size of the VET sector requires standard packages, systems, processes and assessment to be compliant and manageable.  However, the risk is that system quality may be based upon indirect top down paper-based systems and processes of (quality) compliance that are reactive when issues emerge, if discovered.  For example, sub-optimal training and learning, versus proactive measures through more intrusive evaluation of actual training delivery quality or bottom up informing.

Quality maybe improved by intrusive quality assessment through mystery shopping on any given TAE course, dynamic (publicised opportunities) for feedback from trainees and clients, evaluation of specific programs and trainers or evidence of dynamic quality evaluation of skills versus merely possessing a TAE qualification or ‘ticket’.

 

References:

 

Castillo, A 2016, Newly endorsed Certificate IV in Training and Assessment – Same Issues, LinkedIn Pulse, viewed 22 March 2018, < https://www.linkedin.com/pulse/newly-endorsed-certificate-iv-training-assessment-amaro-castillo/ >

 

Clayton, B 2009, Practitioner experiences and expectations with the Certificate IV in Training and Assessment (TAA40104): A discussion of the issues, NCVER Melbourne, viewed 22 March 2018, < https://www.ncver.edu.au/__data/assets/file/0023/4658/nr08504r.pdf >

 

Department of Education & Training 2018, MySkills: Certificate IV in Training and Assessment, viewed 22 March 2018, < https://www.myskills.gov.au/courses/details?Code=TAE40116 >

 

Educators’ Technology 2013, AWESOME CHART ON “PEDAGOGY VS ANDRAGOGY”, viewed 1 April 2018, < http://www.educatorstechnology.com/2013/05/awesome-chart-on-pedagogy-vs-andragogy.html >

 

Halliday-Wynes, S & Misko, J 2013, Assessment issues in VET: minimising the level of risk, NCVER, viewed 22 March 2018, < http://www.cmd.act.gov.au/__data/assets/pdf_file/0015/801600/AssessmentIssuesInVET_MinimisingTheLevelOfRisk.pdf >

 

Munro, J 2017, The TAE debacle – a resources sector view, Resources Training Council, viewed 31 January 2018, < http://www.resourcestraining.org.au/news/the-tae-debacle/ >