Competency Based Assessments in Education

Chapter 4

Literature Review on Competence-based Assessment

4.1 Introduction

In this chapter, the researcher discusses the literature on competence-based assessment. The first part examines the purposes, the interpretations of competence-based assessment (CBA), the critical attributes of CBA and the issues related to competence. The second part looks into the implementation of CBA around the world and later focuses on the implementation of CBA in Malaysia.

4.2 Competence-based Assessment: An Overview

The era of the knowledge -economy and globalisation requires not only individuals who possess a sound understanding of specific subject matter but also those who have relevant industry-related skills and interpersonal skills. These attributes and capabilities are necessary for learners to acquire in order to function well in today’s complex and global societies (Baartman et al., 2007). Furthermore, acquisition of complex competences (Baartman et al., 2007) has to be developed in the future human capital through purposeful, effective, learner-centred and competence-based programmes (Baartman et al., 2007) in order to prepare students to meet the needs of tomorrow’s world. The report of the United States Department of Education Secretary’s Commission on Achieving Necessary Skills, the so-called SCANS Report (McNabb, 1997), made clear that students must be ready to function in collaborative settings, interpret complex requirements, and exhibit self-directed, self-assessing behaviour on the job. This means that employers would want more from the graduates than just entry-level job skills which would help develop a nation progressively in accordance to its political and social needs. The relationship between learning and assessment (discussed in Chapters X and Y) means that assessment should take account of political and social purposes (Broadfoot, 1996). Different vocational and educational training programmes from school level to university level have been introduced to prepare and equip individuals to fit into the labour market. One such programme is Competency-Based Education (CBE) with the emphasis on assessment (competency-based assessment) being seen as key to the success of its implementation (Tillema et al, 2000; Frederiksen, 1984; Baartman et al., 2007).

4.2.1 Purposes of Competence-based Assessment

Any forms of assessment – s including CBA – would usually have one or more of three basic purposes to diagnose learning;, to select students for particular provision; to certificate achievements (Carless et. al., 2006; Freeman & Lewis, 1997; Ecclestone, 1996; Rowntree, 1987). CBA has been utilised by schools, training colleges and industriy for two main purposes; to measure competencies (McNerney & Briggins, 1995) and to certificate (International Labour Organisation, 1996).

4.2.1.1 CBA for Measuring Competence

Measuring competence is one of the main purposes of CBA.Generally, the reason for the implementation of CBA is to determine that learners have sufficient knowledge and skills to contribute effectively to the work force(Canning, 2000; Ecclestone, 1997; Kerka, 1998; LPM, 2002; McNabb, 1997). However,according to Hyland (1994), as competence-based education is found to be seriously flawed and ill-equipped to deal with education and training beyond the basic skills., CBA apparently could can be used to measure limited aspects of competence but Hyland (1994). He believes that its influence on training and education for future generations will be actively damaging as it could can only produce individuals who would function without much learning, knowledge and understanding of anything. He attributes this to a This is due to its highly instrumental philosophy that’s combined with a narrow and uncritical behaviourist psychology. (Hyland, 1994). Thus, its qualifications resulting from CBA are viewed as basically reliable as indicators of all the most elementary skills and abilities (Armstrong, 1995). The issues of competence in CBA will be further discussed in section 4.4.

4.2.1.2 CBA for Certification

It is asserted claimed that CBA provides learners with opportunities to achieve qualifications that relate to required performance in the workplace (Erridge & Perry, 1994). Ecclestone (1997) indicates that NVQs, which primarily employ CBA, represent an explicit commitment to creating wider access to accreditation and better levels of achievement. She argues that Tthis could be made possible by severing links between attendance in learning programmes, and the formal assessment and accreditation of outcomes, and by promoting the accreditation of prior learning in which NVQs subsequently serve as serious challenge to traditional assessment approaches (Ecclestone, 1997). For instance, a trainee in a plumbing courseplumber would have the opportunities to acquire the necessary knowledge and skills in plumbing at certain level. He/she would then achieve the qualifications and certification that relate to the required performance of a plumber in the real life workplace once he/she has completed the assessment of at the at particular designated level. Nevertheless, CBA is at the same time, argued to be conceptually confusing, empirically flawed and lacking in meeting the needs of a learning society (Chappell, 1996; Ecclestone, 1997; Hyland, 1994;). This may be the results of the use of confusing language or jargons, the decreasing credibility of the competency standards on how they reflect industry standards (Kerka, 1998) and the indifferent implementation of CBA across the industries due to employers’ ignorance about the its nature and the purpose. of it (Hyland, 1996).

4.2.2 Definitions and Interpretations of CBA

There is a wide range of interpretations and definitions given to CBA. In this discussion, CBA the interpretation of- CBA is looked atinterpreted in terms of three different aspects; the assessors’ roles, the learners’ responsibilities and the learning outcomes that are based on predetermined criteria. Figure ____tries to reflect the interedependence of these elements.The relationships of the interpretations of these three aspects are as shown in Figure ____.

Adapted: Griffin & Nix, 1991; Mcnerney & Briggins, 1995; Hager, 1994; Elliot, 1994; Cotton, 1995; Ecclestone, 1996)

In the assessor’s perspective, CBA consists of the simple process of seeing, collecting, gathering,and obtaining evidence, and the further process a more complex as well as subjective process of judging and interpreting the evidence of competence demonstrated by learners (Rowe, 1995; Ecclestone, 1996; Mcnerney & Briggins, 1995; Hager, 1994; Griffin & Nix, 1991). The assessors have to observe gather and judge the evidence of an individual’s competence against the specified standards. This means that the assessors have to be very careful in their actions of gathering evidence of competence and they have to decide when it is considered sufficient, based on their expert judgment. For example, when a student successfully builds a drywall framework, the assessor has to gather evidence of competence not only from the product which is the framework but also from the process and the preparations before the student begins to work on it such as work schedule, list of materials and equipment to be used, and the like. The assessor then has to use his/her expertise in this area to determine whether or not the evidence of competence gathered is adequate to say that the student has acquired satisfactory competence in building the drywall framework.

McNnerney & Briggins (1995) state that CBA is the process of identifying the competencies which are the underlying characteristics that lead toof successful performance be this by may it be among a group of employees, typically by department, job category or hierarchical level. CThey say that a list of competencies that is tied to one corporate culture is usually used tobecome associatede with exemplary performance (McNerney & Briggins, 1995)ers. They further relate CBA to its training basis where the focus is on who the successful performers are rather than on what people do. This means that it does not just include training in jobs which rely heavily on psychomotor skills, such as manual labour and traditional hourly production work but also involve performing decision-oriented jobs (Mcnerney & Briggins, 1995).

The Scottish Qualifications Authority (SQA) (2001) stresses the process of seeing whether or not an individual has the necessary skill and knowledge they need to be awarded a Scottish Vocational Qualifications as the key factor in CBA; emphsising the need for assessors to be expert . This undoubtedly requires the assessors to have even thorough knowledge and skills in the fields they are assessing in order to make good and fair judgments. This means is due to the fact that the athat assessors have to assess and collectconsider evidence of competence in terms of knowledge, abilities, skills and attitudes (Rowe, 1995; Ecclestone, 1996; Mcnerney & Briggins, 1995) displayed diferentially in authentic contexts by learners in the context of a selected set of real life professional tasks which are of different levels (Hager, 1994). The process of gathering evidence from observable performance is later followed by the more difficult process of making judgment that may be very subjective (Peddie & Wilmut, Macintosh, 1997). Despite the difficulty in making judgment based on evidence gathered, assessors have to determine whether or not the competency has been achieved by learners (Victoria Curriculum and Assesssment Authority (VCAA), 2001). It is the assessors’ responsibility, then, to decide if learners’ performances meet the pre-determined criteria. Thus, assessors have to equip themselves with relevant skills and use appropriate mechanisms in making fair judgments so that the problem of subjectivity among assessors is reduced. Furthermore, assessors have to assess learners’ ability to apply a particular knowledge, skills, attitudes and values in a specific context according to a required performance standards (New Zealand Qualifications Authority (NZQA), 2002). In other words, assessors themselves have to be extremely knowledgeable and skilful in the art of observing and collecting evidence of competence that come in various tangible and intangible forms. The process of gathering evidence from observable performance is later followed by the more difficult process of making judgment that is inevitably very subjective (Peddie & Wilmut, Macintosh, 1997). Despite the difficulty in making judgment based on evidence gathered, assessors still have to determine whether or not competency has been achieved by learners (Victoria Curriculum and Assesssment Authority (VCAA), 2001). It is the assessors’ responsibility then, to decide if learners are considered to be competent in a particular context at a particular level based on their performance whether or not it meets the pre-determined criteria. Thus, assessors have to equip themselves with relevant skills and use appropriate mechanisms in making fair judgments so that the problem of subjectivity among assessors is reduced.

In addition to assessing and making judgment on students’ performance based on evidence gathered, assessors would also have to give constructive and supportive feedback to students on their performance and work (Ecclestone, 1996; Sadler, 2009). The assessors would have to point out the strengths and weaknesses as well as the improvements that could be made in the future (Sadler, 2009). Sadler (2009) further proposes that feedback should be given in a manner that would be able toenables educate students to assess and be able to evaluate their own work and give feedback to themselves as well. He suggests that students shcould be taught to monitor the quality of their productions and make adjustments as necessary while they are actually engaged in doing it.

In From the learners’ point of view on the other handperspective, CBA is the platform for them to demonstrate competencies and learning outcomes (Elliot, 1994). These competencies will range from simple constructed responses to comprehensive collections of work over time in very different contexts.Elliot further asserts that CBA requires learners to demonstrate competencies and learning outcomes in performance assessment which vary in terms of simplicity and complexity such as from simple constructed responses to comprehensive collections of work over time, all of which are then judged . Learners are expected to be deeply involved in the assessment process and they areto be aware of the specified criteria that they have to meet as well as the standards of performance that are expected of them right from the beginning even before the assessment is conducted. Theoretically, tThis allows learners to take charge of and control over their own learning outcomes and success by preparing themselves well in advance. In other words, learners would have the autonomy for in their own learning, as CBA canould promote individuality and personal development (Ecclestone, 1996). The learner’s responsibility includes demostrating the It is also here in CBA that learners have to show their ability to apply theoretical knowledge and procedures, in addition to their understanding ofbeing able to describe the theories or even point to appropriate theoretical knowledge (Cotton, 1995). In demonstrating competence Cotton further elucidates that learners also have to express wise use of common sense in the public by demonstrating good physical, interpersonal and intrapersonal skills with mindful decision- making; that suggests the multiple intelligence described by Gardner (1985). In other words, learners have to demonstrate their abilities in all the three psychological domains of psychomotor, cognitive and affective learning (ANTA, 1998). Similarly, the Australia’s National Training Authority (ANTA) (1998) considers CBA to be a platform for learners to display their skills, knowledge and experience in accomplishing specific tasks as required in the workplace or to obtain a credit towards a qualification in the vocational and education training (VET).

A Both the assessors and learners have one thing in common: to focus on and that is the set of learning outcomes that can be derived from an assessment. Learning outcomes cover diverse range of areas including personal qualities, various forms of knowledge and skills (Ecclestone, 1996). In this case it is the evidence of competence that learners have to demonstrate and which the assessors have to observe for and make judgments on, has to meet specified criteria. Thus, CBA consists of specified set of both the general and specific outcomes that assessors, learners and third parties can make reasonably objective judgments with respect to learners’ achievement or non-achievement of these outcomes (Wolf, 1995). CBA then certifies learners’ progress based on the demonstrated achievement of these outcomes while the assessments may not be necessarily tied to time served in formal educational settings. The emphasis is on the outcomes – specifically, multiple outcomes, each distinctive and separately considered which should be specified clearly and as transparent as possible for assessors, assessees and third parties to understand what is being assessed and what should be achieved (Wolf, 1995) . This definition encapsulates the key-features of CBA as it has been developed and promoted for the vocational, technical and professional education and training in the UK while at the same time it signals the American origins of much of the debate (Wolf, 1995). The demonstrated performance that provides evidence of competence has to be at least of the minimum required quality in the real life workplace environment. These are the predetermined criteria set in CBA which are generally based on endorsed industry benchmark or competency standard (ANTA, 1998). The emphasis on outcomes and transparency is not only peculiar to the competence-context but it is also an essential characteristic of criterion-referenced assessment. The emphasis on what learners can actually do and the beneficial effects of clear criteria on teaching and learning (Glaser, 1963; Popham, 1978) are argued to meet the competence-based literature where in England in the early years of the implementation of CBA, such system was referred to as criterion-referenced approach (Jessup 1991: 167). Jessup (1991) further underlines that what people actually learn from an education and training system and how effectively, as the key factor to measure its success. Thus, CBA is considered a criterion-referenced interpretation of assessment (Nuttall, 1984; Ling, 1999) where individuals are given an award after achieving the pre-determined standards (Cotton, 1995). This critical attribute of CBA will be discussed further in section 4.3.2.

4.3 Critical Attributes of Competence-based Assessment

The following section discusses the two learning theories associated with CBA and the nature of its criterion-referenced assessment.

4.3.1 Learning Theories Associated with Competence-based Assessment

Learning in the psychology and education contexts is a the process of acquiring knowledge, skills, and values, and world views acquisition and enhancement through one’s integrated employment of cognitive, emotional, and experiences (Illeris, 2000; Ormorod, 1995). How this process works is explained variously. Learning as a process focuses on what happens when the learning takes place and the explanations of what happens constitute learning theories. In other words, learning theories are attempts to describe how people and animals learn, and they help uncover the inherently complex process of learning to our understanding. Hill (2002) explains that learning theories have two main values. The first is to provide adequate vocabulary and a conceptual framework in to interpreting examples of observed learning; and the second. Next is to suggest the right directions to look for solutions to practical problem instead of providing the solutions. Learning theories are therefore, the basis for any form of educational assessment (Gipps, 1994) and the theories most commonly associated to with CBA are the behaviourism and, more recently, the constructivism. These two theories will be discussed as CBA essentially involves observable aspects of learning and learning as a process for construction of new knowledge. Although the cognitive theory which looks beyond behaviour to explain brain-based learning is important, the need for it in CBA is not that apparent or crucial. Thus it is not included in the discussion.

4.3.1.1 Behaviourist Learning Theory

Behaviourism is a theory of organism (may it be an animal or human) learning that only focuses on objectively observable behaviours and discounts mental activities (Murphy, 1999; Kerka, 1997; Doolittle & Camp, 1999) with the assumption that a learner starts off as a clean slate (i.e. tabula rasa) and is essentially passive, responding to environmental stimuli (Murphy, 1999; Kerka, 1997) in the acquisition of new behaviour (Chowdhury, 2006). Learning according to the behaviourists takes place as the result of a response that follows on a specific stimulus and that behaviour is shaped through reinforcement (Kerka, 1997). By repeating the Stimulus-Response (S-R) cycle the learner is conditioned into repeating the response whenever the same stimulus is present and thusbehaviour can be modified and learning is measured by observable change in behaviour (Murphy, 1999; Kerka, 1997; Doolittle & Camp, 1999).

Theis emphasis on stimulus-response pairing (Murphy, 1999; Chowdhury, 2006) and the rejections to of structuralism (Kerka, 1997) reflected behaviourism’s positivistic philosophical base, as the analysis of the human condition relies on only verifiable observations of behaviour and not on untenable mentalistic constructs (Kerka, 1997). Furthermore,Accordingly most human behaviour could can be understood as basic reflexive learning mechanisms or laws that operate on one’s experience within the environment (Kerka, 1997). As the approach is seen to be more operational and practical in nature, it has dominated education.n, in which Tthe teacher disseminates selected knowledge, measures learners’ passive reception of facts, and focuses on behaviour control and task completion (Kerka, 1997). These views of the behaviourists and the learning characteristics that can be found in the education setting are summarised in Table___.

Generally, conditioning has been identified in experiments by behaviourists to be a universal learning process. There are two different types of conditioning, each yielding a different behavioural pattern:

1. Classic conditioning occurs when an instinctive reaction responds to a stimulus (Comer, 2004). Essentially, animals and people are biologically “wired” so that a certain stimulus will produce a specific response. As such, learning process takes place when two events that repeatedly occur close together in time are associated in a person’s mind to impulsively produce the same response. The most popular example is Pavlov’s observation that dogs salivate when they eat or even see food where food is unconditioned stimulus and the salivation, the unconditioned reflex (Comer, 2004; Chowdhury, 2006). Pavlov’s theory of classical conditioning is considered a foundation of learning theories to the behaviourists. According to Pavlov’s experiment, when some neutral stimulus, such as the ringing of a bell, is combined with the presentation of food and is repeated for a period of time, the dog salivates with the ringing of the bell, even though food is not given. Hence, the ringing of the bell acts as the conditioned stimulus while salivation is the conditioned response or reflex (Dembo: 1994). The result of this experiment led to the formation of Pavlov’s classical conditioning in which an individual responds to some stimulus that would ordinarily produce such a response.
2. Behavioural or operant conditioning occurs when a response to a stimulus is reinforced. Basically, operant conditioning is a simple feedback system: If a response to a stimulus is rewarded or reinforced, then the response is likely to take place in the future. Similarly, when a particular behaviour is rewarded, that behaviour is repeated as shown in the experiment conducted by B.F. Skinner using reinforcement techniques to teach pigeons to dance and bowl a ball. B.F. Skinner based his theory upon the idea that learning is a function of change in overt behaviour where these changes in behaviour are the result of an individual’s response to events (stimuli) that occur in the environment (Chowdhury, 2006). According to Skinner, a reward or punishment will either strengthen or weaken a voluntary or automatic behaviour (Skinner: 1968).

Ever since its introduction, the reinforced techniques have gone through series of enhancement and have contributed tremendously in training and teaching. The most important aspect of Skinner’s contribution to training is the significance attached to the desired behaviour to be emitted in certain environment. In order for the trainer to ensure the right behaviour is reinforced in the trainees, the trainer should have the clear idea about the terminal behaviour of the trainees, and should closely follow the trainees to appropriately reinforce correct responses. This is the purpose of programmed instructions including competence-based training in its early years of implementation which was based on this theory of reinforcement (Burns, 1995).

.

As the emerging learning theory of the early 1900s, behaviourism provided the final found ation for social efficiency as learning which is seen objectively consists of the formation of links between specific stimuli and responses through the application of rewards ( Wirth, 1972 ). The emphasis on the need of objectivity leads to extensive use of statistical and ma thematical analysis. Despite all the remarkable contribution s the learning theory has to offer , the extreme focus on objectivity has totally ignored the significant role the mind play in shaping one’s behaviour. Men are treated more like robots or machines than human beings as their thoughts and feelings are not taken into consideration. They are expected to demonstrate desired behaviour through the use of reward and punishment neglecting other factors that may have an influence on the change in behaviour. Thus, the behaviourist theory of learning is lacking in utilizing the full potential of the mind in moulding essential behaviour and in constructing new knowledge .

Assessment in Behaviourism

Assessment, according to behaviourism, is a test (the stimulus) for which the answer (the response) is conditioned In accordance to the behaviourist learning theory which focuses on the stimulus-response cycle to attain observable conditioned behaviour, assessment in the behaviourism also applies the same concept. Thus, t he test item is the stimulus, the answer is the response and a learner has to be conditioned to produce the appropriate response to any given stimulus ( (Murphy, 1999; Kerka, 1997; Doolittle & Camp, 1999). NSince the emphasis is on the response that is observable, no attention is paid to any model of the thinking process of the learner which might intervene between stimulus and response. Consequently, the distinctions between rote learning and learning with understanding is not considered as teaching is a matter deliveringof delivering the appropriate stimuli while learning is a matter of repeating the appropriate response, which will be rewarded. is what matters the most in which teaching is by repetition and then rewarding the appropriate responses. As such, a test composed of many short, ‘atomised’, out-of-context questions, and ‘teaching to the test, are both consistent with this approach (Murphy, 1999; Kerka, 1997). Likewise, some forms of CBA – which has always been associated to thewith behaviourist theory – can be seen to assess, atomistically. applied the atomistic but not out of context approach. The assessor who is an observer ticks off a checklist of predetermined criteria whenever a learner has performed a series of discrete observable tasks. The criteria are the stimuli, the accomplished tasks the responses and learner has to be conditioned to demonstrate the ability to meet the criteria successfully.

Although this approach to assessment may developstestify to learner’s ability to perform observable tasksbehaviours, it does not pay much attention to the theoretical knowledge and understanding (Ashworth, 1992) as the role of the mind is considered insignificant in delivering the required behaviour. While assessing competent observable performance is vital, assessing knowledge and understanding is just as important as it is an essential aspect of competence without which an assessment is lacking in credibility or construct validity (Ashworth, 1992). A valid assessment method should be able to measure what it is supposed to measure which in this case (Watson, 1994). Given the extensive discussion in Chapter ? on the idea of competence, both the observable performance behaviour and underpinning knowledge are aspects of competence that should be assessed and measured. People who ‘understand’ are those who have clear mental representation of the situation with which they are confronted and are able to deal with it creatively and imaginatively using the acquired knowledge which acts as an interpretive resource for them (Ashworth, 1992). Thus, it is insufficient to assess one’s competence just by looking at the performance while ignoring the aspect of knowledge and understanding. It is unfortunate then, if such an assessment method should produce people who are like robots in a factory; they couldwho can perform a job or a task efficiently and effectively but they do not have any understanding of what they were are doing. As the approachCBA also emphasises personal competence within competence concentrates on an individual demonstrating competent performance ((Wolf, 1995), traditional notions of CBA have allowed an and emphasises on personal competences, it leads to one being individualistic perspective whilst lacking ignoring the very necessary in the abability of being able to work as a team player to work as a team whereas team work is essential in performing relevant aspect of a job in the actual workplace (Ashworth, 1992). As a result, theis behaviourist view of CBA has eventually shifted to the constructivist belief as discussed in the following section.weakened.

Despite all the remarkable contributions the learning theory has to offer, the extreme focus on objectivity has totally ignored the significant role that the mind plays in influencing one’s behaviour. People are treated more like robots or machines than human beings as their thoughts and feelings are not taken into consideration. They are expected to demonstrate desired behaviour through the use of reward and punishment neglecting other factors that may have an influence on the change in behaviour.

4.3.1.2 Constructivist Learning Theory

Constructivism is a theory of learning that has roots in both philosophy and psychology (Doolitle & Camp, 1999) founded on the premise that learners actively construct their own knowledge, meaning and understanding of the world they live in by reflecting on their experiences (Doolitle & Camp, 1999; Murphy, 1999; Kerka, 1997). Learners learn by doing rather than observing and by bringing prior knowledge into a learning situation (Epstein & Ryan, 2002; Carvin, date?) in which they must critique and re-evaluate their understanding of it until they can demonstrate their comprehension of the subject (Carvin). Furthermore, learners need to analyse and transform new information or problems in their minds based on existing knowledge and understanding where these abstract thoughts evolve from concrete action (Murphy, 1999). Learning, therefore, is simply the process of adjusting their mental models to accommodate new experiences. TBasically, the theory of constructivism rests on the notion that there is an innate human drive to make sense of the world by building cognitive structures which include declarative knowledge (know that – facts, concepts, propositions) and procedural knowledge (know how – techniques, skills, and abilities) (Murphy, 1999). These two components of knowledge have been discussed in depth in Chapter 3. Moreover, learning is a matter of personal and unique interpretation which takes place within the social context and is of useful to the learner as intrinsic motivation emerges from the desire to understand and to construct meaning (Billet, 1996). However, dispositions such as attitudes, values and interests that help learners decide, are often neglected in this theory (Murphy, 1999) making it incomprehensive and insufficient in a way.

Philosophically, the essence of constructivism relies on an epistemology that stresses subjectivism and relativism, where personally unique reality resulted from the concept that reality can be known through experience although it may exist separate from experience (Doolitle & Camp, 1999). Hence came four essential epistemological tenets of constructivism (Von Glasersfeld ,1984; 1998; Doolitle & Camp, 1999);

• Knowledge is the result of active cognizing by the individual ;
• Cognition is an adaptive process that functions to make an individual’s behaviour more viable given a particular environment;
• Cognition organizes and makes sense of one’s experience, and is not a process to render an accurate representation of reality; and
• Knowing has roots both in biological/neurological construction, and in social, cultural, and language-based interactions (Dewey, 1916/1980; Garrison, 1997; Larochelle, Bednarz, & Garrison, 1998; Gergen, 1995).

Thus, constructivism acknowledges the active role learners play in the personal creation of knowledge, the importance of both the individual and social experiences in this knowledge creatio