Original Article

Psychometrics of Questionnaires Designed for the Evaluation of Safety Levels and Safety Culture Case Study: Power Plant Generator Manufacturing Company

Abstract

The psychometrics of instruments in safety performance evaluation is essential for the accreditation of an organization’s safety evaluation and has been emphasized in many studies. Psychometrics pertains to the validity and reliability of an evaluation instrument and describes its precision and consistency. The aim of the present study was to evaluate the psychometrics of safety level and safety culture questionnaires to provide a reliable and valid instrument for safety performance evaluation in industries. This descriptive cross sectional study was conducted with the intention of psychometric instruments used to evaluate the safety level and the safety climate at the Mapna Pars power plant generator manufacturing and engineering company. The face validity of the questionnaires was analyzed quantitatively and qualitatively. In the quantitative method, the importance of the influence score was applied and the content validity was calculated using the Lawshe method. Content Validity Ratio (CVR) and Content Validity Index (CVI) were also used. For this purpose, elicitation was obtained from experts within the professional health and safety community. The reliability of the instruments was determined via the Cronbach's alpha test and the Interclass Correlation Coefficient (ICC) test. Finally, the standardized questionnaires were used to evaluate the safety level and safety climate of the industry as a case study. The obtained data were analyzed using SPSS software solution v20.  Based on the quantitative face validity results obtained at the Mapna Pars Company, one question was removed from each of the final instruments. The content reliability analysis revealed that the safety level questionnaire had a CRI of 0.99, CVR of 0.91, and was accepted. The safety climate questionnaire had an acceptable CVI of 0.95 and an acceptable CVR of 0.82. Regarding the reliability analysis, a Cronbach's alpha of 0.72 and 0.89 was obtained for the safety level questionnaire and the safety climate questionnaire, respectively. It can be concluded that both questionnaires had an acceptable level of internal consistency. The re-application of the questionnaires after two weeks revealed a relatively consistent safety level (ICC=0.90) and safety climate (ICC=0.74).  Analyzing the data obtained in the present study showed that the safety level questionnaire with 66 questions and the safety climate questionnaire with 93 questions had acceptable validity and reliability. Thus, it may provide a useful approach for safety evaluations in similar industries.

1. Pfeiffer Y, Manser T. Development of the German version of the Hospital Survey on Patient Safety Culture: Dimensionality and psychometric properties. Saf Sci. 2010;48(10): 1452-1462.
2. Lyu S, Hon CK, Chan AP, Wong FK, Javed AA. Relationships among safety climate, safety behavior, and safety outcomes for ethnic minority construction workers. Int J Environ Res Publ Health. 2018 Mar;15(3):484-89.
3. Griffin MA, Neal A. Perceptions of safety at work: a framework for linking safety climate to safety performance, knowledge, and motivation. J Occup Health Psychol. 2000;5(3): 347-356.
4. Neal A, Griffin MA, Hart PM. The impact of organizational climate on safety climate and individual behavior. Saf Sci. 2000;34(1):99-109.
5. Heydarei MH, Farshad A, Arghamei Sh. The survey of relation between safety calamite and safety behavior in metal industry workers in Arak. Iran Occup Health. 2007;4(3,4): 1-8.[ In Persian].
6. Zohar D. Safety climate in industrial organizations: theoretical and applied implications. J Appl Psychol. 1980;65(1): 96-102.
7. Zohar D, Luria G. A multilevel model of safety climate: cross-level relationships between organization and group-level climates. J Appl Psychol. 2005;90(4):616-628.
8. Zohar D. Thirty years of safety climate research: Reflections and future directions. Accid Anal Prev. 2010;42(5): 1517-1522.
9. Sarsagni V. Evaluation of safety climate and its structual factor in some of iran uranium mines. MSc thesis, Shahid Beheshti Medical University, Tehran, Iran, 2011. [In Persian].
10. Jafari MJ, Sadighzadeh A, Sarsangi V, Zaeri F, Zarei E. Development and Psychometrics of" Safety Climate Assessment Questionnaire". J Saf Promot Inj Prev. 2013; 1(3): 123-133. [In Persian].
11. Barkhordari A, Dehghani A, Kianfar A, Mahmoudi S, Aminifard F. Safety performance evaluation using proactive


indicators in a selected industry. J Occup Hyg Eng. 2015;1(4):49-59. [ In Persian].
12. DeArmond S, Chen PY. Occupational safety: The role of workplace sleepiness. Accid Anal Prev. 2009;41(5): 976-984.
13. Eskandari D. Developing of an instrument to determine the safety performance considering organizational, individual and environmental factors and its verification in a pertrochemical plant. Tehran, Iran. PhD thesis, Shahid Beheshti


University of Medical Sciences 2016. [In Persian].
14. Foxcroft C, Paterson H, Le Roux N, Herbst D. Psychological assessment in South Africa: A needs analysis: the test use patterns and needs of psychological assessment practitioners. 2004; Available from: www.hsrc.ac.za/en/research-data/ktree-doc/1186.
15. Fillenbaum GG, Smyer MA. The development, validity, and reliability of the OARS multidimensional functional assessment questionnaire. J. Gerontol. 1981;36(4): 428-434.
16. Brennan RL. An essay on the history and future of reliability from the perspective of replications. J Educ Meas. 2001;38(4): 295-317.
17. Standards for educational and psychological testing: Amer Educational Research Assn. 1999. https://www.aera.net.
18. De Wet C, Spence W, Mash R, Johnson P, Bowie P. The development and psychometric evaluation of a safety climate measure for primary care. Qual Saf Health Care. 2010;19(6): 578-584.
19. Kaya S, Barsbay S, Karabulut E. The Turkish version of the safety attitudes questionnaire: psychometric properties and baseline data. BMJ Qual Saf. 2010;19(6): 572-527.
20. Hoffmann B, Domanska OM, Albay Z, Mueller V, Guethlin C, Thomas EJ, Gerlach FM. The Frankfurt patient safety climate questionnaire for general practices (FraSiK): analysis of psychometric properties. BMJ Qual Saf. 2011;20(9): 797-805.
21. Jafari MJ, Eskandari D, Valipour F, Mehrabi Y, Charkhand H, Mirghotbi M. Development and validation of a new safety climate scale for petrochemical industries. Work. 2017;58(3): 309-317.
22. Guldenmund FW. The use of questionnaires in safety culture research–an evaluation. Saf Sci. 2007;45(6):723-743.
23. Guldenmund FW. The nature of safety culture: a review of theory and research. Saf Sci. 2000;34(1):215-57.
24. Mohammadbeigi A, Mohammadsalehi N, Aligol M. Validity and reliability of the instruments and types of measurments in health applied researches. JRUMS. 2015;13(12):1153-70. [In Persian].
25. Juniper EF, Guyatt GH, Streiner DL, King DR. Clinical impact versus factor analysis for quality of life questionnaire construction. J Clin Epidemiol. 1997;50(3):233-8.
26. Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489-97.
27. Delshad MH, Hidarnia A, Niknam S. Psychometric measure continuous variables preventive behaviors of hepatitis B virus infection in health care workers. JMUMS. 2014;23(109):71-82. [In Persian].
28. Sigari N, Ghafori B. Reliability of Persian Version of COPD Assessment Test and its correlation with disease severity. Sci J Kurd Univ Med Sci. 2013; 18 (4) :59-65. [In Persian].
29. Mohammadfam I, Saraji GN, Kianfar A, Mahmoudi S. Developing the health, safety and environment excellence instrument. Iran J Environ Health Sci Eng. 2013;10(1):7.
30. Asgharizadeh E, Ghasemi AR, Behrooz MS. Accident assessment based on controlled indicators of safety performance by EVAMIX. J Emerg Manag. 2014;1(3):57-63.
31. Janicak CA. Safety metrics: Tools and techniques for measuring safety performance: Government Institutes; 2009.
32. Hosseini M, Yaghmaei F, Jabbari M. Correlation between population characteristics and safety attitude to work of women workers in food factories. Journal of Health Promotion Management. 2012;1(3):64-72. [In Persian].
33. Hassanzadeh Rangi N, Allahyari T, Khosravi Y, Zaeri F, Saremi M. Development of an Occupational Cognitive Failure Questionnaire (OCFQ): evaluation validity and reliability. Iran Occupational Health. 2012;9(1):29-40. [In Persian].
34. Yaghmaei F. Measuring Behavior in Research by Valid and Reliable Instruments. Tehran: Shahid Beheshti University of Medical Sciences and Health Services; 2005 [In Persian].
35. Lu C-S, Tsai C-L. The effects of safety climate on vessel accidents in the container shipping context. Accid Anal Prev. 2008;40(2):594-601.
36. Osterlind S, Tabachnick B, Fidell L. SPSS for window workbook to acompany: using multivariate statistics.--4th.ed. Needham Heights, Mass: Allyn and Bacon; 2001.
Files
IssueVol 13 No 4 (2021) QRcode
SectionOriginal Article(s)
Published2021-12-30
DOI https://doi.org/10.18502/ijoh.v13i4.8428
Keywords
Safety level Safety climate Safety performance Power plant generator manufacturing
Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
1.
Andishe S, Jafari M, Fasih Ramandi F, Khodakarim S, Salehi A. Psychometrics of Questionnaires Designed for the Evaluation of Safety Levels and Safety Culture Case Study: Power Plant Generator Manufacturing Company. Int J Occup Hyg. 2021;13(4):337-346.