Literally, the word of evalution means that decide on the valueor
quality of something[1]
Stufflebeam, dkk (1971) define the evaluation is The process of delineating,
obtaining, and providing useful information for judging decision alternatives[2].
While according to Arikunto (2003) define the evaluation is a part of
activities that have purpose to measure how deep the result in course program[3].
Evaluation is intergral part of the teaching process and it has
some pre-course and off-course functions, such as appropriately placing
students or certifying that they have reached certain level of profiency. But
the main role of an evaluation in instructional programs is measure achivement
that is to see how and wheter students have mastered course materials, and to
identify areas where they need additional help.
Huthinson and waters (1987:144) claim that there are two level of
evaluation in ESP, Learners assesment and course evaluation and both have
similiar function in providing feedback on the ESP. Learner assesment is to
asses the learners performance at the beginning and at the end of course. The result
of this kind of evaluation provide valuable information for the institution,
sponsor, teachers, and learner to make decision wheter and how much language
financial support is required. The
course evaluation is the evaluation of ESP course it self. It helps to assess
wheter the course objectives are being met and whether the course is doing what
it was designed to do. The information gathered forms the starting point for
any necessary revison of the course and may help to guide the design other
similiar courses.
Simiarly, Robinson (1991) comment thet with regard to ESP programs,
a course assesment is seeking to establish the effectiveness and afficiency of
teaching programs which can be distinguished in to formative and summative
evaluation. Formative evaluation is carried out during the life of a course or
a project and the results obtained can be used to modify what is being done.
In contrast, summative evaluation is
carried out when the course or project is finished and when it is clearly to
late to do any appropriate changes. Decision will be taken as to whether to
repeat the course or not, whether the money has been well spent or not.
A primary issue in any course evaluatin or test is what to
avaluate, since this will largely determine how students view their learning
objectives. Brinton, Snow and Wesches (1989:185-186) catogorized the “what to
evaluate” as follows:
a.
Knowladge
of elements of the linguistic code : e.g., pronounciation, sentence mechanics,
relevant vocabulary, grammatical forms and patterns, word order.
b.
Knowladge
of discourse ; organizational patterns, specialized vocabulary, and other
conventions appropriate to the type of content baing taught, including
different oral and writing genres.
c.
Interactive
communication skill: ability to communicate affectively in face to face
situation in the clasroom and in the other context of eventual use, including
comprehension and the use of paralinguistic and non linguistic cues.
d.
Academic
language use skill : ability to analyze and manipulate academic content trough
language; critical thinking.
e.
Related
study skill; efficient not taking, ability to use the library, refrencing skill
for research papers, use of non verbal material in reading, and timing
management.
A.
Planning a test
Before making a test, a tester needs to know some steps of test
planning as well as stage and consideration in developing a language test.
1.
Step of testing
a.
To
determine the purpose of the test
In determaine the purpose we have decide the purpose before such as
profiency, achivement, diagnostic or placement. In a result of course there are
four purposes may be there[4] :
-
To
repare the assesment or the process of study (formative)
-
To
decide the succes of student in course (summative)
-
To
identify the students’ problems in a course (diagnostic)
-
To
place the student what level they are suitable.
b.
To
specify the objective of the test : what is to measure
c.
To
define the content area of the test : performance or skill.
d.
To
determine the relative weight of the different parts of the test : oral/writen,
performance/skill, fluency/accuracy and etc.
e.
To
determine what testing method and procedure to use in order to tap the
objectives and content : rol play, simulation and etc.
Table stage and consideration in developing a language test
|
Phase I
Purpose
|
Determining the porpose of the test
The tester decide on why he is testing and what he will use the
result for
|
|
Phase II
Planning
|
Planning the test with the aid of table spesification
The tester consider the objective of the test as they relate to
what will be tasted.
|
|
Phase III
Items/tasks
|
Selecting the test items and task
The tester choose among various testing procedure.
|
|
Phase IV
Administering
|
Administering the test
the tester consider a systematic administration in the
appropriate testing condition and gives the test.
|
|
Phase V
Scoring/Rating
|
Scorring and rating
The tester avaluate the quality of the language sample according
to certain criteria
|
|
Phase VI
Analysis
|
Analyzing the test and its items and task
The tester analyzes the whole thest as well as the spesific items
and task.
|
|
Phaase VII
Reporting
|
Reporting the test result
The tester convert the raw testing scores into meaningful
information for reporting to the specific audience.
|
2.
Determine the purpose of a test
a.
Proficiency
Evaluation of person’s language knowladge in relation to future
language use. It doesn’t necessarily depend on what has been learned before in
given course.
b.
Achivement
To evaluate the test-taker’s language in rlation to a given
curiculum or material which the test-taker had gone through in a given course
c.
Diagnostic
To identify the test-taker’s strength and weakness in thye
language.
d.
Plecement
To evaluate the test-taker’s language knolwladge in relation
curriculum of diferent level of the future language course.
e.
Entarance
To evaluate the test-taker’s language in relation to the language
needed in a future course.
f.
Mastery
To evaluate the test-taker’s language in relation to spesific.
B.
Testing Oral Ability
1.
Problem in testing oral ability
The basic
problem in testing oral ability arise in setting task that objectively
represent oral task which the candidates are expected be able to perfom.
In setting the task,
spesification for oral at the intermediate lavel may involve the following
language functions :
a.
Content
Expressing : thanks,
requirements, opnion, attitude
Narrating : events and
sequnce of event
Eliciting :
information, direction, etc
Directing : ordering,
instructing, advising, etc
Reporting :
description, comment/views, decision
Text type :
dialogues and multi-participant interaction normally of a face to face
nature.
b.
Central
level of performance
Accuracy : clearly intelligible pronounciatio,
although influenced by learners.
Appropriacy : use of language generally approproate to
the function. The overal intention of the speaker is always clear
Range : a fair lavel of language is
avalaible to the candidate. He is able to express himself without overtly
seacrhing words.
Flexibility : is able to take initiative in a
conversation and to adapt to new topics of changes of direction.
Size : most constribution may be
short but some evidence of ability to produce more complex utterance.
c.
Format
Interview : the interview is the most obvious and
the most common format for testing oral interaction.
Interaction
With peers : two or more candidate may be asked to
discuss a topic, make plans, comment on a picture, etc. While this makes it
easier for the tester to assess the candidate’ more objectively, the
performance of one candidate is likely to be affected by that of the others.
Responding
To tape-
recording : uniformity of elicitation can be
achieved through presenting all with the same audio or video recorder stimuli.
2.
Planning and conducting oral tests
a.
Make
the oral test 15-30 minute long to get enough reliable information about the
candidate’ oral communication ability.
b.
include
as wide a sample of specified content as possible in the time available.
c.
Plan
the test carefully. It would be mistake to begin an interview with no more than
a general idea of the course that it might take.
d.
Give
the candidate an opportunity to use more than one format, interact with more
than one tester, and tackle separate.
e.
Select
interviewers carefully and train them with video-recording of interviews.
f.
Ideally,
have to testers present at each interview. This makes the judgment more
reliable.
g.
Set
topicand task that would cause candidates no problem in their own language.
h.
Carry
out the interview in a quaite room with good acoustics.
i.
Put
candidates at ease.
j.
Collect
enough relevant information to help you to make a fair and comperenhensive
assesment of the candidate’s oral ability.
k.
Do
not talk too much ! elicit and let candidates do much of talking.
C.
Elicitation techniques
Elicitation is a technique by which the
teacher gets the learners to give information rather than giving it to them[5].
So elicitation technique gives more chance to students to express theirselves.
The kinds of elicitation techniques :
1.
Question and request for information
Avoid or use only a few ‘yes/no question’ and quikly pass on to
information question to elicit various function.
2.
Picture
Use single picture fo elicitation description and picture series
for elicitation of naration.
3.
Role-play
You can ask candidates to assume a role in particular situation.
Use several short situation as this will help you to elicit a variety of
language situations.
4.
Discussion
Discussion between candidates can be valuable source of
information. They may be asked to discuss a given topic to arrive at a dicision
or point of view.
5.
Tape-recorder stimuli
This is ideal for use in the language lab as a large number of
candidate can be tested at the same time. For this you can use description of
situations, remarks in isolation to respond to stimulated conversation, and
conversation completion with script.
6.
Imitation
Candidates hear a series of sentence each of which they have to
repeat.
D.
Testing Writing
There are many steps you can take to ensure
that your test is more effective and that test writing becomes a learning
experience such as The elements of a good test, Validity of a test, Reliability
of a test, The affect of tests, Other features of a good test, Assessing
difficulty and Conclusion[6].
Writing involve five general components :
a.
Content : the substance of the writing;
the ideas expressed
b.
Form : the organization of the
content
c.
Grammar : the correct use of syntatic
patterns and sturtural words
d.
Style
: the choice of appropriate
struture and lexical items to give a particular flavor or tone to the writing.
e.
Mechanics : the use of the graphic
conversation of the language
1.
Objective test versus composition/essay-task test
while assesment in a problem, composition test are superior to
objective test for the folloeing reason :
a.
Composition
test require students to organize their own ideas expressed in their own words.
b.
Composition
test motivate students to improve their writing with appropriate feedback from
their teacher.
c.
It
take less time and effort to prepare a composition test
d.
Student
may cover up weakness by avoiding problems
2.
Item types
a.
Recognition
item
1.
Dictation : teacher read some sentence and
the student ask to write what teacher says.
2.
Multiple
choise : this item to test
punctuation, spelling etc.
b.
Production
items
The kinds of production items are like taking notes, making list,
filling forms, writing short message, etc
3.
Points to note
a.
Arrange
to take several sample not only one sample
b.
Set
writing task
c.
Make
the writing task clearly and spesific and provide full diretions
d.
Allow
for alternative and the use of variety of forms
e.
Apportion
marks for the various component
f.
Ensure
objectivity in scoring.
4.
An anatical rating scale for evaluation written language
a.
Content
(maximum score 30)
-
30-
27 : excelent to very
good (knowladgeable, subtantive and imaginative)
-
26-22 : good to everage (some
knowladge of sucject, imagination, adequate range.)
-
21-17
: fair to poor (
limited knowladge of subject, little subtance)
-
16-13 : very poor ( doesn’t show
any knowladge of subject/ imagination, not subtantive, etc)
b.
Organization
(maximum score 20)
-
20-18 : excelent to very good (
fluent expession, ideas clearly stated, etc)
-
17-14 : good to average ( somewhat
choppy, loosely organized but main ideas stand out)
-
13-10 : fair to poor ( not fluent,
ideas confused or disconnected)
-
9 -7 : very poor ( doesn’t
communicate, no organization)
c.
Vocabulary
(maximum score 20)
-
20-
18 : excelent to very
good (sophisticated range, effective word choise and usage, etc)
-
17-14 : good to everage (eduquate
range, occusional errors of word, etc)
-
13-10
: fair to poor (
limited range, frequent errors of word/idiom form, etc)
-
9 -
7 : very poor (
essentially translation, little knowladge of english vocabulary)
d.
Language
use (maximum score 25)
-
25-
22 : excelent to very
good (efective complex construction)
-
21-19 :
good to everage (effective but simplem constructions)
-
18-11
: fair to poor ( major
problems in simple/complex contructions)
-
10 -
5 : very poor ( virtually
no mastery of sentence constructional rules)
e.
Mechanics
(maximum score 5 )
-
5 : excelent to very good
(demonstrates mastery of conventions)
-
4 : good to everage (occasional
errors of spelling punctation)
-
3 : fair to poor ( frequent errorss
of spelling, punctuation)
-
2 : very poor ( no mastery
conventions dominated by errors of spelling, punctuation capitalization
pragraphing, etc)
5.
A
holistic rating for evaluating written language
|
SCORE
|
HOLISTIC ASSESMENT
|
|
0 – 9
|
Just an attempt
|
|
10 – 29
|
Some language barely comprehensible
Not fully communicative
|
|
30 – 39
|
More than barely comprhensible; only simplest most basic language
produced
|
|
40 – 59
|
Comprehensible
Has some gramatical, lexical or syntactic problems
Communicative but erorrs, etc
|
|
60 – 69
|
Communicative
Awarness of sociolinguistic aspect
Good syntax
Idiomatic language, etc
|
|
70 – 79
|
Near native; sporadic errors
|
|
80 – 100
|
Native; like production
|
E.
Testing Reading Comprehension
Some of the skills and sub-skills
that should be tested in Reading Comprehension are the students’ abilitiy:
a.
To
guess meanings of word/expressions from context
b.
To predict
what the text or messages are about, baset on initial input
c.
Identify
the main idea (s)
d.
To
identify specific details
e.
To
make inferences
f.
To
ideantify sequence of events, ideas, etc.
g.
To
differentiete between relevant and irrelevant information
h.
To
differentiate fact and opinion
i.
To
differentiate between reality and fantasy
j.
To
recognise/transfer the information
k.
To
perform a task using the information in the text.
1.
Points to consider
While testing students for reading comperhension, the following
points should be born in mind.
a.
Select
different forms and styles of writing (stories,letters,
dialogues,diaries,articles,repots, etc.
b.
While
selecting/constructing items,include some authentic tsks the test-taker will
have to do in real life.
c.
Do
not attempt to look for many questions for each text; for most short to medium
text only a few relevant questions are possible.
d.
Try
to use many short texts in a variety of forms and on a variety offorms and on a
variety of topics with a few questions on each, rather than one long passage.
e.
Make
sure that the language of the question and the instructions is clear and
appropriate to the level.
f.
Do
not fokus on irrelevant information.
g.
Include
questions on pronominal refrence as well.
2.
Item types
a.
Recognition
Restatment
, completion, question-answer
True
–false
Matching
sentence parts.
b.
Production
Completion
Question-answer
Cloze
with wordist/word alternatives
Cloze
with no clues
True
–false questions with justifaction
Form
/table-completion
Task
performance
3.
Points to remember
a.
In
constructing multiple-choice items, ensure that the stem presents the problem.
b.
Avoid
questions that students can answer on the basis of
1)
Visual
discrimination
2)
Word/phrase
matching
3)
Outside
knowledge
c.
Adept
passages if necessary to ensure that there aren’t too many unfamiliar word in
passage.
d.
Test
both lower order and higher order skills.
e.
Use
authentic materials and set appropriate tasks wherever possible.
F.
Testing Listening Comprehension
in
listening comprehension tests we primarily test the following aspects:
a.
Sound
discrimination
b.
Sesitivity
to sress and intonation
c.
Global
comprehension
d.
Discrete-point
comprehension
e.
Other
aural comprehension-related subskills
f.
Taks
performance
1.
Important hints in testing aural/oral skills
a.
Focus
on the massage
b.
Do
not test memory alone
c.
Stress
authentic contexts,materials and use, as far as possible
d.
Provide
sufficient context-clues
e.
Give
simple, clear and unambiguous directions
f.
Try
to create a non-threatening environment
g.
Avoid
using unfamiliar accent
2.
Item types
a.
Recognition
-
Ticking
the right word
-
Picture/word
matching
-
Word
in context
Example
: I’II.....the needle for you
a.
Thread b. Dread
c. Threat d. Tread
Matching the spoken word with written meaning
Example : cota. Baby’s bed
b. Stopped and held c. Pulled by horses d. Small animal
b.
Comprehension
Restatement
Student hear ‘i wish i had done what you
told me to’ and choose a sentence corresponding in meaning or complete a
sentence keeping the meaning intact.
Example
: ‘you told me to do it but i.....’.A. don’t B. Didn’t C.
Won’t D. Shouldn’t
Appropriate
response
(you can use either ‘wh’ questions or
‘yes-no’ questions)
Example : students hear ‘why are you going
to jakarta in december ?’
And choose from A,B,C, and D below
A.
At
six B. To get married C. By bus
D. Yes, Iam
c.
Picture
–spoken word correlation
d.
Identifying
location /direction in diagrams
e.
Following
directions/instructions
f.
Information
transfer
Form
filling, object placement on a picture/diagram (story-based), etc.
g.
Identfying
summary/outcome of short dialogues
h.
Lecture-based
note-taking
3.
Sub-skills in testing comprehension
a.
Literal
recognition (reading) or recall (listening)
1)
Recognition
or recall of details
2)
Recognition
or recall of main ideas
3)
Recognition
or recall of sequence
4)
Recognition
or recall of comparison
5)
Recognition
or recall of cause and effect relationship
6)
Recognition
or recall of character traits
b.
Inference
1)
Inferring
suporting details
2)
Inferring
the main ideas
3)
Inferring
squence
4)
Inferring
comparisons
5)
Inferring
cause and effect relationship
6)
Inferring
character traits
7)
Predicting
outcomes
8)
Inferring
about figurative language
c.
Evaluation
1)
Judgments
of reality or fantasy
2)
Judgments
of fact or opinion
3)
Judgments
of adequency or validity
4)
Judgments
of approprateness
5)
Judgments
of worth, desirability or acceptability
d.
Appreciation
1)
Emotional
response to plot or theme
2)
Identification
with characters and incidents
3)
Reactions
to the speakers’ use of language
4)
Imagery
e.
Task
performance
1)
Following
directions
2)
Following
instructions
G.
Testing Grammar
1.
in testing grammar there are
some points to remember
a.
avoid
testing only discrete items unless it is for diagnostic purpose.
b.
Use
natural running contexts wherever possible.
c.
Ensure
that the test includes a broad range of relevant grammatical problems in proprotions
reflecting their relative importance.
d.
Avoid
infreguent or invloved constructions found only in very formal writing.
e.
Do
not use non-existing form.
f.
Do
not have too many items of one type or on one area.
g.
Try
to test grammar in context ising items like modified cloze or a
dialogue-completion exercise. Either multiple choice or open ended, depending
on the level of the class. These items also help you to relate the teaching and
testing diagnostic or remedial purpose, use a large number of discrete items as
well.
2.
item types
a.
Multiple
choice
1)
Completion
Example
: tim ought not to____me the story, but he did
a.
Tell b.
Have told c. Be telling
d. Have been told
2)
Rearrangement
Example
: complete the following by putting the letters (a,b,c,d)
For
the words in the right order.
“won’t i need a coat?”
“well you know how___”
A.
It B. Today
C. Warm D. Is
3)
Recognition
of right form
Which
of the following is grammatical correct ?
A.
I
saw him yesterday
B.
I
had seen seeing him yesterday
C.
I
have been seeing him yesterday
D.
I
see him yesterday.
4)
Error
recognition
Example : choose one word or phrase, from A,B,C,and D which would
not be acceptable in standard english.
Here
is a picture , a good friend of hisand I which wastaken
at
A.
The B. Beach
C. Last D. Summer
5)
Short
response
Example
: I wish Mr.jacob would speak more clearly.
A.
“so
do i “ C. “so wish I”
B.
“i
wish so “ D. “I would I”
b.
transformation
Example
:
i have’t written to her for a long time.
It’s
a long time______
c.
Writing
appropriate word forms
Example
: researchers.............(convince) that a cure for AIDs is nearly insight.
d.
Matching
parts
Match
part from A and B correctly by drawing a line
|
A
|
B
|
|
Are you going to see a film to night?
|
So can I
|
|
I’m fond of chocolate
|
no, i don’t.
|
|
How was the movie, ‘gandhi’?
|
are you ? i don’t care for them much my self
|
|
Do you like Mr. rajan’s class ?
|
yes, i probably will
|
|
|
I thought it was very interesting.
|
e.
extension
I’ve done the
work. (already)
I’ve already
done the work.
f.
Modified
cloze –multiple choice or open ended
Testing
articles : candidates are required to write the, a or NA-No article
In the united states children go
to ......school from Monday to Friday........school that hasyim goes to is very
small. He walks there each morning with........friend. one morning they saw
............man throwing ............stones and ........pieces of food
at..............dog..........dog was afraid of.............man.
H.
Testing Vocabulary
1.
points to consider
a.
A
word acquires meaning in context and can have several meanings.
b.
A
word can have several synonyms and antonims and false synonyms as well.
c.
Words
are used diffrently in formal context.
d.
Learners
have a larger reportoire of recognition vocabulary.
e.
Words
are/ should be thought on the basis of frequency and range, and appropriateness
to level.
f.
Word
are often used incombination (as phrasal words, collocations, associate
pairs,etc.)
2.
item types
a.
Recognition
–multiple choice
Definition
Completion
Paraphrase
Picture/word
matching
Parts
of a whole
Elimination
of non-homogenous word
b.
Production
Sentence
with word list/word alternatives
Cloze
with word list/word alternatives
Replacing
word in context with suitable synonyms
Writing
approprite forms of word, given the context.
Scrambled
words.
Producing
apprpriate contexts for a given word.
Using
words/phrase approprite to context
Explaining
meaning of word /phrase in context
Modified
(certain classes)of standard cloze (every 3 or five words are deleted)
use appropriate
context/language ensure that :
a.
The
correct option and the distracters are at the same level of difficulty and the
same lenght, as far as possible.
b.
Instructions
are simple and clear
c.
There
are no synonyms among the distracters.
d.
All
choice relate to the same area or activity and part of speech.
e.
You
avoid tricks and traps.
Refrences
Oxford Learner’s
Pocket Dictionary, Fourth Edition (Oxford University Press, 2008)
NAVEL’S BLOG,
014 Pengertian Evaluasi, Pengukuran, Dan Penilaian Dalam
Dunia Pendidikan retrieved from : http://navelmangelep.wordpress.com
Eka
hardiyanti, dkk,
2013 Tahap Evaluasi dan
Teknik Ealuasi hasil Belajar retrieved from : http://dekha-sajalah.blogspot.com
British
Council College (BBC), 2013 Elicitation Technique retrieved from : www.teachingenglish.org
M Basri Wello
and Syarifuddin Dollah, Fundamental
Aspect of English Spesific purpose. Makassar: Badan penerbit UNM, 2008
Abdul
hakim,
014. English for Spesific Porpose task: english
education student TBI 4, semester VII, English for Spesific Porpose. STAIN
Watampone, Bone
[1]
Oxford
learner’s pocket dictionary (forth edition ) p. 151
[2]
NAVEL’S BLOG, Pengertian
Evaluasi, Pengukuran, Dan Penilaian Dalam Dunia Pendidikan at
http://navelmangelep.wordpress.com/2012/02/14/pengertian-evaluasi-pengukuran-dan-penilaian-dalam-dunia-pendidikan/
accesed on 7th december 2014
[3]
Ibid.,
[4] Eka
hardiyanti, dkk, Tahap Evaluasi dan Teknik Ealuasi hasil Belajar at http://dekha-sajalah.blogspot.com/2013/03/tahap-evaluasi-dan-teknik-evaluasi_2181.html.
accesed on 7th December 2014
[5]
British Council
College (BBC), Elicitation Technique at www.teachingenglish.org.uk/knowledge-database/elicitation
accesed on 7th December 2014
[6]
British Council
College (BBC), Writing Test at www.teachingenglish.org.uk/knowledge-database/elicitation
accesed on 7th December 2014
Tidak ada komentar:
Posting Komentar