Certification Exam
Development
CIC leaders are committed to ensuring the quality, relevance, validity, and fairness of each of our certification exams programs. Together with our psychometrics vendor, Kryterion Global Testing Solutions and experienced Subject Matter Experts serving under the leadership of the CIC Governing Council, great care is given to assessing, evaluating, and improving these exams on an ongoing cycle. To follow is a description of the process.
Job Analysis
In June of 2021, the Crane Institute Certification (CIC) entered into a contract with Kryterion Global Testing Solutions to conduct formal job analyses on a series of content-valid certification examinations in five areas:
-
Telescoping Boom, Under 21 Tons (General Knowledge Exam + Supplemental Exam + Practical Exam)
-
Telescoping Boom, 21 – 75 Tons (General Knowledge Exam + Supplemental Exam + Practical Exam)
-
Telescoping Boom, Over 75 Tons (General Knowledge Exam + Supplemental Exam + Practical Exam)
-
Lattice Boom Crane Carrier/Crawler (General Knowledge Exam + Supplemental Exam + Practical Exam)
-
Articulating Boom Crane (General Knowledge Exam + Supplemental Exam + Practical Exam)
Each certification examination consists of a Written General Knowledge examination, a specific supplemental examination, and a Practical Examination. CIC maintains two versions of the General Knowledge Exam, two versions of each of the written supplemental examinations, and one version of the practical exam.
A new job analysis is conducted every seven years to ensure currency with current practice in the industry. The Governing Council approved this timeframe during the 2018 Annual Meeting to reflect the limited pace of change within the industry.
The job analysis process includes the following:
-
A review of existing mobile crane operator and rigger/signalperson job descriptions, job analyses, and industry standards documents (ASME, OSHA) is used to create a preliminary outline of key knowledge, skills, and abilities necessary for effective operation of mobile cranes.
-
A panel of subject matter experts (SMEs) from the crane industry is facilitated to finalize the knowledge, skills, and abilities (KSAs) necessary for safe mobile crane operation and safe rigging. The panel of SMEs then complete a linkage questionnaire identifying which KSAs link to which tasks.
-
A copy of the Tasks and KSAs in the form of a job analysis questionnaire is distributed to industry SMEs who rate the importance and frequency of Tasks and KSAs recommended by the initial expert panel for safe mobile crane operation.
-
SMEs also rate task importance relative to scoring.
Exam blueprints/content specifications and an examination development plan are generated as a result of the preceding steps.
Item Writing
To support writing of new items, Exam Development Training is conducted to provide guidelines for item development from a psychometric standpoint. Based on the exam blueprints, SMEs from the crane industry draft questions for the written assessments, tasks for the practical exams and instructions / processes for administration of the written and practical exams.
All exam items are linked to KSAs identified in the job analysis, and a reference is provided where the answer can be verified. A review of the questions, tasks, instructions, and processes is conducted by SMEs to ensure appropriate question content and responses. Assessment Consultants conduct a review in order to ensure appropriate measurement/ psychometric properties of the items.
Standard Setting and Equating
SME Panel for cutoff/weighting (Angoff Committee) - An SME Panel provides formal training on how to assign the Angoff ratings that define the cutoff score for new items and exams.
New assessment items (written) are tested as unscored items embedded in the existing certification exams, and statistical analysis of item response characteristics is conducted when sufficient data is gathered (i.e., 100-500 responses). New practical assessments are field-tested with small groups of five or ten crane operators to ensure the instructions and process are fair and valid. Equating alternative forms of the assessments – a 3-parameter IRT model is used as recommended by the CIC psychometric consultant.
Examination Versions
All exam forms are assembled according to the approved exam content outline.
The percentage of items in each of the content categories should be regarded as estimates. Future item edits to the written examinations could result in slight variations in these percentages. The General Knowledge Exam has three active versions, and each supplemental exam has two active versions.
