The Forum on Assessment Issues VI (FOAI)
Review by İdil Ertugan
The Forum on Assessment Issues VI (FOAI) took place on 15th April at Bahçeşehir University, Istanbul. The theme of the forum was ‘Placement and Proficiency exams’ and representatives from a variety of state, foundation and private universities participated in the event.
In the first part of the session, the representatives of the institutions gave a brief presentation outlining their practices on placement and proficiency exams. This was followed by in-depth focus group discussions. Finally, the focus groups gave presentations based on their discussions which were followed by a whole group discussion.
The mini presentations enabled the assessors to get a brief idea on how placement and proficiency exams are prepared, administered, and the components they include in other institutions.
During the focus group discussions, there were some questions that the participants had to dwell upon in their mini groups. The questions and the ideas that came up during the discussions are as follows:
1.Is it feasible or desirable to link different institutions’ Placement and/or Proficiency exams? Or is it feasible or desirable to establish a mutual recognition agreement by which different institutions agree to recognize one another's Placement and/or Proficiency exam?
All the focus groups decided that it is not feasible to link different institutions’ Placement and Proficiency exams. The rationale behind this is that, the requirements, pass grades, needs and objectives of each and every institution are different. Even the student backgrounds are different.
2.How are cut scores established in both exams in your institution? What descriptors are used?
Since this is a question directed to each and every institution that took part in FOAI, one direct answer is not possible. However, generally the cut scores are CEFR aligned. Expert judgement is used by looking at the item analysis or other statistical devices. Also, the cut scores differ from one faculty to the other even in the same institution.
3.What is done to ensure the reliability of marking for the open-ended sections (e.g. writing, gap-fill etc.)
Almost all the universities use multiple choice questions for the Placement exams. Double marking is done in open ended questions to ensure reliability. In terms of writing and speaking parts of the exams when there is a discrepancy the papers are taken to third marking or the moderator comes into the picture. Standardization sessions are done for writing parts. Also, having clear criteria for both writing and speaking components of the exam makes those parts more reliable.
4.Can an initial placement decision be altered? If yes, how and when?
Some universities allow misplaced students to change levels after some procedures are followed. Diagnostic writing is the most popular procedure. Mostly, the first week diagnostic writing tasks and/or tests are given to students and the teachers decide whether those students should be placed in a higher or lower level. In some universities students are not offered a chance to move from one level to another.
5.Should Writing and/or Speaking be a component of the Placement exam? Why/Why not?
Almost all university representatives believe that there should not be a writing and/or speaking component in placement exams. Since these exams are given frequently, lack of resources and time constraints would make it difficult to incorporate these components within the exams. Representatives believe that only the students who have the necessary language level can be tested for their writing and speaking skills in proficiency exams.
6.How do the Placement / Proficiency exam results inform the instructional program in your institution?
Assessors agreed that these tests provide a positive feedback loop whereby analysis of the results informs teachers about the considered effectiveness of the program content.
7.How important is it to align the Placement / Proficiency exam to an internationally recognised framework such as CEFR?
Although it takes a lot of time and energy, it is considered good practice to align the objectives and the curriculum with the CEFR framework. It is a lengthy process and different perspectives from different experts/ institutions are needed. Financial burden is another point to consider. However, some representatives believe that it is impossible to survive without this framework. Another point that the representatives discussed was the difficulty of interpreting the descriptors of CEFR since they are not clearly defined. Therefore, it would be useful if an international exam bank existed and when exams were to be prepared, this bank would set examples.
In brief, it was a great opportunity for the representatives of some universities to come together and share their experiences in this one-day event. The next event will take place in November 2016 at Pamukkale University in Denizli on the theme “Validation and Reliability”.
