Language test for airline pilots

The AEROSTA Framework Airline Pilot Language Test was developed by Aviation English Asia Ltd in co-operation with the Aviation English Organisation.  The test is designed to identify the degree that language affects communication between pilots, air traffic controllers and ground personnel.  The test has been placed in the hands of partner organisations to be administered worldwide as a recognised, universal, valid and standardised language test.  The test is available to corporate customers. An automated computer-based version of the test is expected to be available from late 2018.

The AEROSTA Framework Airline Pilot Language Test measures the specific reading, listening, and oral proficiency skills required to successfully perform flight operations in English.   It is suitable for candidates with 250 or more flying hours.  It is not suitable for cadet and ab initio candidates.

PLACEMENT AND PROFICIENCY TESTS 

The AEROSTA Framework Airline Pilot Language Test is offered to airlines and aviation authorities for both licensing and screening purposes.  The purposes of the two tests are different. 

1. The Placement Test enables airlines, and stakeholders to identify employees who would benefit from English language training. The Placement Test identifies starting level, and specific difficulties with pronunciation, vocabulary, grammar and listening comprehension.  The Placement Test can be used to group candidates with similar levels of proficiency.  If an employee lacks sufficient proficiency, then remedial training can be provided.  The Placement test is also suitable for screening purposes.
   
   
2. The Proficiency Test identifies if an employee can consistently perform workplace-related language tasks effectively in English. The purpose of the test is to verify that employees have sufficient English language proficiency in order to perform their duties.  In countries where AEROSTA is recognised by local Civil Aviation Authorities it is possible to use the test for compliance with ICAO language proficiency requirements.  The Proficiency Test can also be used as a check and balance to make sure that training provided has been effective, or to check the validity of other ICAO language test results.

As a matter of best practice, the Placement Test is recommended before starting a language development course, and the Proficiency Test is recommended after completing a course.

AEROSTA Framework (APLT) Test format

Candidates are fully briefed about the test format before they take the test.  The brochure AEROSTA Framework - Information for Stakeholders contains test procedures.

The test has four parts, paper 1, paper 2, paper 3 and paper 4. Parts can be omitted depending on budget and time available to conduct each assessment.

Paper 1 is the Interactions test. It typically takes 15 minutes. Candidates are asked to read an appropriate response to a prompt. This test reveals difficulties in pronunciation, structure, listening comprehension, fluency and interaction. 

Paper 2 is an oral interview which the assessor uses to elicit samples of speech related to the candidates flying experience.  In addition to the interview, a range of authentic aviation matter can be used as prompts, eg charts, photos, manuals, METARs, schematics etc.

Paper 3 is a series of short listening comprehension activities which takes about 10 minutes.  The candidate will hear a series of communications from an air traffic controller. The candidate is expected to read back the clearances/instructions from the controller.  The recordings will be authentic standard phraseology and include natural speech patterns (not voice actors reading from a script) but the recordings will be clear and not affected by interference.  This section of the test primarily assesses a candidate’s ability to distinguish functions and numbers as used in an aviation context, though some basic structures may also be included which drastically alter the nature of the communication.  Therefore, proficiency in AEROSTA basic structures is essential to prepare for this part of the test.

Paper 4 is a longer listening comprehension test which takes about 15 minutes.  The candidate must listen to a 3-5-minute monologue or dialogue about a flight operations-related incident.  The recording will be authentic and include natural speech patterns (not voice actors reading from a script).  Typical examples could be a dialogue with an air traffic controller, or two colleagues discussing a matter.  The language used will be correctly spoken English, but may be an accented native speaker, or a non-native speaker.  The candidate is allowed to take notes during the audio on a specified area of the answer sheet. 

The candidate must then deliver an oral report about the incident with as much detail and information as possible.  The candidate should use full, complete sentences in intelligible speech.  The candidate has 2 minutes to deliver the report.

TEST RELIABILITY AND VALIDITY

Reliability is a measure of candidates of equal ability receiving the same scores, or of receiving the same scores if retested. Validity is the extent to which a test measures what it is intended to measure. It relates to the uses of test scores and the ways in which test scores are interpreted. During the test design phase consideration was given to ensure that:

• Users of the test believe that the test is appropriate.

• Performance in the test can be interpreted as a measure of language proficiency.

• The test content is relevant to the purpose of the test.

• Performance is predictive of performance in non-test environment.

Language in the AEROSTA Framework APLT

Both the prompts and responses contain flight operations-related vocabulary graded according to their frequency of use and technicality. High frequency words may include times, dates and locations eg instruments, ramp. Low frequency words may include words such as components, ADIRU, the underside of the starboard wing, country names, chipped. The prompts and responses contain a selection of basic, and complex structures. All vocabulary and structures are directly related within range expected of airline pilots.

Pronunciation of high frequency workplace-related vocabulary and grammatical structures is tested with reference to the ease of understanding on the part of the listener, regardless of accent.

TEST RESULTS

A Certificate of Proficiency is generated for each candidate. The Candidate Test Report will indicate the following:

• If language is a serious obstacle to flight operations. Further language training is essential.

• If language is likely to be an obstacle to flight operations or engaging in flight training courses. Training is beneficial.

• If language is unlikely to be an obstacle to flight operations. Training is not urgently required but may be undertaken in order to facilitate more effective communication

Scores are aligned to the Common European Framework of Reference, and a series of CAN DO statements are generated for each candidate. A Progress Report is generated for the entire group with two paragraphs of feedback per candidate.

CEFR A1 (80-120)	CEFR A2 (120-140)	CEFR B1 (140-160)	CEFR B2 (160-180)	CEFR C1 (180-210	CEFR C2 (210-230)
BULATS: 0-19 IELTS: X TOEFL: X TOEIC: 120-224	BULATS: 20-39 IELTS: 3.0 TOEFL: 337-459 TOEIC: 225-549	BULATS: 40-59 points IELTS: 4.0 - 4.5 points TOEFL: 460-542 points TOEIC: 550- 784 points	BULATS: 60-74 points IELTS: 5.0 – 6.0 points TOEFL: 543- 626 points TOEIC: 785-944 points	BULATS: 75-89 points IELTS: 6.0 - 7.0 points TOEFL: 627 points TOEIC: 945 points	BULATS: 90-100 IELTS: 7-9

 

Meaningful scores - Many candidates at ICAO level 4 vary in proficiency from marginal to borderline extended proficiency, and as they have equal scores in paper may not be motivated to improve further. The AEROSTA test was designed so that it can encourage improvement in specific areas of difficulty eg pronunciation of vowels, or consonant clusters, subject verb agreement, so scores are given as a CEFR level A1-C2, and as a range in ICAO level eg pronunciation 4.2-4.5 rather than giving a flat score like level 3, 4, 5 for each ICAO descriptor.

Trained assessors - Assessment is conducted by qualified and experienced language professionals who have a thorough understanding of aviation communications and the type of language tasks which would be required during a commercial flight.  To avoid conflict of interest and negative washback, AEROSTA assessors must not be involved in training candidates or have contact with candidates prior to the screening test.

For more information about AEROSTA visit aviationenglish.org