English proficiency in the Saudi Air Academy: validating a new test battery

Abstract

When the validity of an operational syllabus or a test is questioned, in relation to a particular use, then either should facilitate modifications or changes. One of the most common complaints of instructors and administrators alike of the language programme (teaching and testing materials) at the Royal Saudi Air Force (R. S. A. F) institutions in general, and the King Faisal Air Academy (K. F. A. A) in particular, is that it prevents them from doing any useful work with cadets. Concern about the validity of the test and the teaching materials led to the setting up of this project by the K. F. A. A in Saudi Arabia.

Its purposes were firstly, the assessment (investigation into the validity) of the test and the teaching materials produced by the Defence Language Institute in the U. S. A and used by the R. S. A. F, in relation to their use in this context, but most importantly the production of new test material to assess the English proficiency of trainee pilots who are going to pursue a course in aeroscience and flying conducted in the medium of English. The ultimate aim was to provide a prognostic test which it is hoped will improve the teaching of English by discouraging the teaching of "synonyms and encouraging the teaching of English" with emphasis on listening, speaking and reading.

Relevant work in the U. K and the U. S. A is described and discussed. The theoretical principles underlying the construction of the American Language Course (A. L. C) are described and discussed. The shortcomings of the A. L. C and the E. C. L are presented and discussed. Alternative specifications for a new syllabus are described and discussed.

The principles underlying the construction of the new test material, the Test of English for Air Cadets (T. E. A. C) are described and discussed and a number of hypotheses are set up. These hypotheses relate to the:

a. English Comprehension Level predictive and content validity;

b. Test of English for Air Cadets concurrent and predictive validity;

c. Construct of language proficiency;

d. T. E. A. C's discriminant power between different levels of performance.

Preliminary, pre-pilot and pilot test work is described and the final version of the T. E. A. C. is considered in detail. Test analyses of the data, both internal and external are described and tabulated. The analyses considered are item analysis, reliability of the sub-tests and the battery as a whole, test intercorrelations, factor analysis and Analysis of Variance. Concurrent and predictive validation studies are considered in some detail. Expectancy tables are used to establish cut-off points. The hypotheses are considered and on the basis of evidence they are all rejected in their null form.

As a result of these analyses it is argued that the T. E. A. C. shows satisfactory reliability and validity. One optimum cut-off is established for all academic subjects for the two levels tested. The implications of the results are discussed and a follow-up study is recommended.