Volume 6, Issue 1 (1-2022)
EBHPME 2022, 6(1): 5-15 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Ahmady S, Salari H, Esfandiari A, Nemati R, Habibi A. Design, Implementation, and Evaluation of Physicians Empowerment Course for Evidence-Based MRI Prescription at Bushehr University of Medical Sciences in 2021. EBHPME 2022; 6 (1) :5-15
URL: http://jebhpme.ssu.ac.ir/article-1-367-en.html
URL: http://jebhpme.ssu.ac.ir/article-1-367-en.html
Department of Medical Education, Virtual School of Medical Education and Managment, Shahid Behshti University of Medical Sciences, Tehran, Iran , Amin28@gmail.com
Full-Text [PDF 1317 kb]
(344 Downloads)
| Abstract (HTML) (954 Views)
Full-Text: (164 Views)
Background: Evidence-based medicine is considered as a new and reliable approach which has new discussions on judgment to find the best evidence or qualitative evidence. This study was designed in Bushehr province for designing, implementing, and evaluating physician empowerment courses in evidence-based MRI prescriptions in the training hospitals of Bushehr University of Medical Sciences in 2021.
Methods: The present study was an educational scholarship which was conducted in Bushehr province. This study was performed in two phases in accordance with Glassic criteria. A number of five workshops were designed and conducted by researchers using the ADDIE Model in systemic educational design. The course was designed through the review of texts and interviews with experts and in-depth group discussions (FGD) with the presence of seven experts. After that, this training course was evaluated and the end-of-course test was performed using the Kirkpatrick model. In order to evaluate the reaction of participants during the training course, a questionnaire was designed and its face validity and content validity were approved by experts. In addition, a four-choice test was designed based on the presented scientific content to evaluate the learning level of participants. A number of 40 questions were designed for five workshops. Descriptive statistics (frequency, mean) were used in SPSS22 for data analysis.
Results: In terms of gender, 53.7 % were male and 46.3 % were female. In terms of age, 24.4 % were in the age group of 31-35 years. In terms of improving awareness, 34.1 % of participants were at a very good level after the empowerment course. Learning findings indicated that 85.7 % of participants in the final test received a passing score while 14.3 % failed.
Conclusion: In general, the findings of the study indicated that holding an empowerment course for physicians in Bushehr improved their awareness and knowledge. In fact, such improvement in awareness and knowledge can reduce unnecessary prescriptions and treatment costs but improve the quality of services.
Key words: Empowerment, Evidence-based medicine, Design, MRI, Training course
The increasing diversity of the medical equipment market, as well as demand and expectations of physicians and patients for the selection and use of new medical technologies has resulted inan increase in health care costs, especially in the medical sector throughout the world. In particular, the competition between countries and within countries to provide more capital medical equipment and attract more patients has resulted in unnecessary allocation of equipment in some areas and hospitals, as well as the excessive and unnecessary use of these devices (4).
Nowadays, health care costs are increasing rapidly. In the US, health expenditures increased from 7 % of GDP in 1970 to 15 % of GDP in 2003 (5) and increased to 17.8 % in 2016 (6). Meanwhile, imaging is in the middle of this huge flood of diagnostic procedures and is regarded to be the fastest growing cost among medical services (7). The use of radiology services increased by 14.6 % during 1993-1999, including for almost 10 % of total medical services costs (8). However, a study conducted during 1997-2006 showed that the increase was 24 %-35 % in terms of number and was from 54 %-70.2 %in terms of cost (9). Inappropriate exploitation is a big problem in health care, especially in capital-oriented fields such as imaging (8).
With the introduction of new generations of medical diagnostic and therapeutic equipment in Iran during the recent years, we have witnessed the arrival of such devices to the country and follow-up is being performed for more devices to enter Iran (10). The first MRI machine arrived in Iran for the first time in 1991. Since then, the distribution of this technology has risen rapidly in the country so that the number of this technology increased 20 times per one million of the country's population during 1993- 2003 (11).
The use of clinical guidelines to improve the knowledge and skills of prescribing diagnostic and treatment procedures based on the indications seems necessary. This type of skill is related to evidence-based medicine and plays akey role in improving the professional ethics of physicians. Thus, the inclusion of evidence-based medical education, clinical guidelines, and prescriptions in line with valid indications can be highly helpful and necessary (12, 13).
Due to the significance of the issue and the need to increase the capabilities of physicians, it is expected to lead to better diagnoses and effective resource management. This process was designed in Bushehr province for evaluating physicians' empowerment courses in evidence-based MRI prescriptions in the training hospitals of Bushehr University of Medical Sciences in 2021.It should be noted that patients face long queues to receive such diagnostic services due to the limited number of MRI machines in Bushehr province. With the process of empowering physicians, the percentage of unnecessary prescriptions is expected to reduce, resulting in shorter waiting times, faster diagnoses, and better resource management.
Materials and Methods
The present study was an educational scholarship conducted in Bushehr province and aimed to evaluate the empowerment courses of physicians in evidence-based MRI prescriptions in 2021. This study was performed in two phases in accordance with classic criteria including clear goals, appropriate preparation, appropriate methods, considerable results, effective presentation, and thoughtful review.
Executive steps of the study
Design and implementation of physician empowerment course:
At this step, the necessary needs assessments were conducted using text review, expert opinion and Focus Group Discussions (FGD). In addition, scientific materials, indications and educational content were designed and implemented to empower physicians of the University of Medical Sciences in the form of workshops using the approach ADDIE (Analysis, Design, Development, Implementation And Evaluation).In the design part, the Kern curriculum planning model was used in four steps including problem identification and needs assessment, identification of general and specific objectives, determination of training methods, and determination of comprehensive assessment methods. This course, which included a total of five workshops was held for six months
In order to hold the workshops, the required coordination was performed with the education department of the university and the workshops were registered in the national system of continuous education for physicians. After that, the licenses were allocated for courses and workshops. Localized guidelines were used for educational content in previous studies (14, 15).
Description of participants
The studied population in the training course included the physicians of Bushehr province who were MRI prescribers, as well as the final year students and assistants. Forty physician participated in the study. Sampling method was census used. In fact, all participants were selected. The learners included general practitioners, internists, neurologists, neurosurgeons, emergency medicine specialists, and pediatricians, as well as internship students and assistants. After obtaining the permits for workshops, extensive information was performed in form of administrative letters, as well as other informal follow-ups.
Evaluation of participants
The evaluation process was performed using the Kirkpatrick model. In the evaluation stage of the first and second levels of the training course, the above-mentioned population was studied as a sample and the required data were collected using a questionnaire.
Data collection tools
The Kirkpatrick method is an appropriate method for the evaluation and measurement of the training effectiveness.
The level of reaction and learning was measured based on the above-mentioned model and the tools which are designed and described below. In order to evaluate the reaction of participants in the training course, an appropriate questionnaire was designed and its face validity and content validity were approved by experts. A four-choice test was designed according to the presented scientific content to evaluate the learning level of participants. A total of 40 questions were designed for five workshops. Face validity and content validity were confirmed by experts and the questionnaire reliability was confirmed by Cronbach's alpha of 0.8. The tests were carried out immediately after each workshop by the national system of continuous education. The participants were reported as passed and failed. For each workshop, eight scientific questions were designed based on the educational content of the teacher. All of the participants in the workshops participated in the test. The participants who answered at least five out of eight questions received a passing score.
The Kirkpatrick method is an appropriate method for the evaluation and measurement of the training effectiveness. This method is appropriate for evaluating and measuring the effectiveness of training. The Kirkpatrick model evaluates the effectiveness of programs at four levels: Level 1 evaluation: Reaction, Level 2 evaluation: Learning, Level 3 evaluation: Behavior and Level 4 evaluation: Result (16). Based on the above- mentioned model and the tools which are designed and described below, two levels of reaction and learning were measured.
Data analysis method
In order to analyze the data, descriptive statistics (absolute and relative frequency, mean, and standard deviation) and inferential statistics (independent t-test, ANOVA and Pearson correlation coefficient) were used in SPSS22. In addition, the significance level of the tests was considered as 0.05.
In order to satisfy the participants, the objectives of the research were explained to them and they were asked to participate.
This study was a part of two dissertations with ethics ID IR.VUMS.REC.1400.012 and IR.VUMS.REC.1400.013 approved in Tehran Virtual University of Medical Sciences.
Results
In this section, the considerable results, effective presentation, and thoughtful review of the current study are explained. Table 1 indicates the titles of the designed and implemented workshops related to the empowerment course for physicians at Bushehr University of Medical Sciences in 2021 which was conducted during six months. Tables 2 and 3 present the results of the demographic variables of age and sex of the participants and the answers to the checklist questions. Based on the findings, most of the participants were over 35 years old. In addition, the lowest percentage belonged to the group of 20-25 years. Furthermore, most of the answers were related to the very good category. About 34 % of participants reported that their awareness had improved significantly. The announced percentage related to other categories is given in Table 3.
Regarding the level of the course difficulty, the highest percentage of answers was related to the good category with a frequency of 53.7 %.
Regarding the training method, the highest percentage of answers in terms of training method was related to a very good category with a frequency of 41.5 %.
Based on the findings, 36.6 % of the participants in the course evaluated the completeness of its objectives at a good level.
In this study, the rate of satisfying the participants' expectations from the training course was evaluated at a good level (34.1 %) and very good (26.8 %).
The quality of the discussions raised in the training course was evaluated by the participants. Out of 41 participants, 14 participants (34.1 %) considered it good and 11 participants (26.8 %) considered it very good.
The highest answer of the participants in the training course in terms of the quality of group activities was related to the average option with 29.3 %. The details of these results are shown in Tables 1, 2 and 3.
Findings of participants' learning level
In order to evaluate the knowledge and learning of the participants in the training course, eight questions were designed for each workshop and the test was conducted through continuous training. The details of these results are shown in Table 4.
Effective presentation and thoughtful review
After the realization of the research, the results were presented as an educational poster and provided to all hospitals in the province and all physicians, medical students, and assistants. In the internal meetings of the hospital, thoughtful review was considered from the beginning of the process to the end. At each stage, the reviews were conducted in different stages of design and implementation with suggestions provided by experts.
For instance, due to the low participation of physicians at the beginning of the process, the heads of hospital were asked to consider motivational factors for participants, which had a highly positive effect on the presence and active participation of physicians in hospitals in the province.
Effective presentation: The present educational scholarship process was published in the following ways:
*N: Number **P:Percent
Discussion
This study aimed to design, implement and evaluate physician empowerment courses in evidence-based MRI prescriptions at Bushehr University of Medical Sciences teaching hospitals in 2021.This course, which included five workshops, was designed and implemented during six months. In addition, an educational poster was designed and provided to physicians through formal and informal ways. The Kirkpatrick learning model was used to evaluate and measure the effectiveness of the training course. In evaluating the first level of items to improve participants' awareness, difficulty of the course, teaching method of the course, completeness of the course objectives, meeting expectations, quality of discussions, quality of group activities, relationship between the course and job activity, compatibility of teaching aids, overall quality of teaching aids, ability of teacher, and its overall effectiveness were evaluated. The results of the first level indicated that most of the answers were related to very good and good options for most of the questions. Average and excellent options were in the next categories and the lowest answers were related to the weak option. The evaluation of the second level using the test indicated that the majority of participants passed the course while only about one-sixth of them failed. A study entitled “Effect of evidence-based medical education on the ability of reviewing articles by medical students of Tehran University of Medical Sciences: Was conducted by Rafiei et al. (17). This study was conducted on medical students of the internship at Tehran University of Medical Sciences in Winter 2007. The participants who were selected by convenience sampling method, participated in the evidence-based medicine workshop, which was held at the Student Research Center of Tehran University of Medical Sciences and received a standard questionnaire before and after participating in the workshop. Answering the evidence-based medical application questions in the critical assessment of articles after participating in the workshop had significantly increased, which is consistent with the results of the present study. It seems that the evidence-based medical skills in physicians and students increase by holding empowerment courses and training courses. Thus, education managers and health policy makers should pay special attention to this issue. For instance, a study entitled “Evidence-based medical education and its effect on knowledge, attitude and ability to use clinical students of Bushehr University of Medical Sciences in 2013”was conducted by Mirzaei et al.(18). This study aimed to determine the effect of short-term evidence-based medical education on the knowledge and ability of employing clinical medical students at Bushehr University of Medical Sciences. The above-mentioned study was a quasi-experimental control study before and after training which was conducted on two groups of medical students of clinical courses (intern students studying in the academic year 2012-2013) of Bushehr University of Medical Sciences. The students were selected by census method. At the beginning of the course, the students were trained in five standard stages of evidence-based medical practice in the intervention group after providing a description of the executive objectives of the process. Such states included questioning, searching, evaluating evidence critically, using and assessing in four sessions of four hours and then performed under the guidance of the teacher. Then, the level of knowledge and ability of students in intervention groups (N= 26) and control group (N= 23) in relation to the use of evidence-based medicine using valid and reliable questionnaires, pre-test and post-test (in the intervention group three months after the training course) was evaluated. The results of the study indicated an increase in students' familiarity with practical terms and resources in evidence-based medicine. In this study, the students' awareness of the process of performing evidence-based medicine and their ability to implement and use evidence-based medicine after the intervention were shown, which is consistent with the results of the present study. A study entitled “Analysis of the methods of teaching evidence-based medicine at universities inside and outside Iran in 2015” was conducted by Karimian et al. (19). In the study, different teaching methods and levels of effectiveness of the courses were studied with an analytical look at the related articles. This study was conducted with a theoretical analytical approach based on the library method and search for articles in scientific databases. The results revealed that teaching evidence-based medical courses begins during the early years of medicine in most studies. Although the lecture method was used in most of the trainings to present the concepts, the integrated approach and the use of active practical methods and group discussion was asignificant part of the students' training. In this review study, the following results were achieved: The studies which were conducted on the effectiveness of courses often measured satisfaction and knowledge while there is less evidence in the use of courses. Thus, it is necessary to pay more attention to the use of interactive and integrated methods with an interdisciplinary approach in the education of medical students. It is better to teach evidence-based medicine courses in a step-by-step way in accordance with the educational needs of the student in the basic sciences course. It is necessary to conduct further studies on the effectiveness of training courses in practice and behavior change. In addition, the present study conducted a training course to empower physicians and assessed satisfaction and awareness.
In some studies, such as the present study, a workshop training program was used and the results of the study indicated that holding a workshop has a positive result in the knowledge, attitude, and practice of learners. In a study by Taheri et al. (20) in 2006 that aimed to determine the effect of an evidence-based medical workshop curriculum on the skill of its use by medical students, 24 medical students in the fourth and fifth years who had passed the internal or surgery courses and participated in a workshop in three 5-hour sessions on evidence-based medicine. The difference between students' attitudes toward holding the workshop regarding the pre-test and post-test was significant, which is consistent with the results of the present study.
Some studies indicated that despite positive attitudes, evidence-based medical skills are weak. In a study by Amini et al. (21) entitled “The level of knowledge and use of evidence-based medicine among the specialized assistants of Shiraz University of Medical Sciences as one of the most significant groups of educators and learners”, the results indicated that 93.9% of the residents believed that using the results of research could improve their daily decisions, while 80.3 % of them failed to receive written training in evidence-based medicine. In addition, 80 % of respondents had access to the Internet, more than 70 %were unaware of evidence-based medical sources, and 60.9 % said that they had used Medline or other search engines less than 10 times for their patients in the past year. The results of this study revealed that specialized assistants, despite a positive attitude towards evidence-based medicine and access to the Internet for clinical decision-making, practically fail to use evidence-based medicine and are unaware of specialized evidence-based medicine websites. Perhaps the reason is that they have not received any formal training in this field. Due to the significance of this group in clinical decisions, as well as lower education, there is an urgent need for comprehensive education regarding the use of evidence-based medicine in this group.
In the present study, a set of empowerment workshops was designed and implemented for physicians and assistants due to the significance of this issue and the evaluation of the workshop indicated positive results.
Some studies revealed that even assistants have no sufficient knowledge about evidence-based medicine.
In a study by Kalvani et al. (22) in 2019 entitled “Assessing the level of knowledge and use of concepts and evidence-based medical databases among the specialized assistants of Shahid Beheshti University of Medical Sciences” the results revealed that the residents received a score of 2.9 for the level of knowledge and 2.7 for the use of evidence-based medical databases from the mean score of 5, indicating that the residents' knowledge and use of databases from evidence-based medical information is at the average level. Based on the results of their study, most of the residents have no sufficient knowledge about the concepts and databases of evidence-based medicine, so that most of the resources used to meet their information needs are the sources other than evidence-based information sources, thus, planning for evidence-based medical education and its bases to residents is necessary as highly regarded in the present study. A study by Yoon et al. (23) evaluated the continuing education empowerment program for physicians and assistants in hospitals using the Kirkpatrick model. In this study, the participants' reaction level was evaluated using a questionnaire. Behavior change assessment was conducted through the files written by physicians. The results indicated that the participants were satisfied with the training course so that the average was 4.4 out of 5. The performance score at the end of the period was 3.8 out of 5 while it was 2.3 at the beginning of the period.
The score for writing medical records rose from 2.9 to 3.3. Their study indicated that holding a continuous education course to empower physicians, which was designed and implemented with the full participation and responsibility of participants, was efficient and effective, which is in line with the results of the present study.
Solivetti et al. (24) in a study entitled "Appropriate rate of knee MRI Prescriptions: Clinical, economic and technical issues" conducted in Italy found that 21 % of all knee MRI prescriptions were completely inappropriate and 18.8 % of the total prescriptions were unreliable.
In fact, the most considerable result is that about 40 % of prescriptions were inappropriate or unreliable and general practitioners prescribe more than one-third of knee MRIs. Inappropriate MRI examinations and prescriptions have caused a significant financial burden in the national health system of that country and are in line with studies conducted in Bushehr province and show the need for educational empowerment courses. Sheehan et al. (25) in a study entitled "Decrease of unnecessary shoulder MRI prescriptions in a health care system: The potential role of shoulder ultrasound" realized that 106 cases(45 %) out of 237 MRIs prescribed for the shoulder area were inappropriately prescribed which were mostly due to the lack of previous radiography.
The physicians with non-orthopedic specialties had a higher percentage of inappropriate prescriptions than orthopedic specialists, i.e. 44 % and 17 %, respectively. In that study, ultrasound could show all the relevant pathologies in 157 cases, i.e. 66 % of cases in 237 cases of shoulder MRI. Such studies show the necessity and significance of designing and implementing training empowerment courses for physicians more than ever.
Andrade et al. (26) in a study entitled "Case study of demand-based supply: case study of imaging scans (CT Scan and Magnetic Resonance Imaging) in Manaus " concluded that the provision of imaging services in itself increases the demand for these services increases centers for diagnostic services, CT scans and MRIs in Manaus.
This issue indicates that if the supply of radiology services in Bushehr province increases, the possibility of induced demand and inappropriate prescriptions will increase. Therefore, policymakers and planners should consider the necessary measures and seriously, design and implement workshops on empowerment training.
Previous studies in the field of evidence-based medicine in the country often focused on medical students since it is easier to coordinate and monitor their work. In addition, such studies more focused on measuring knowledge, attitude or performance based on self-declaration. Nevertheless, the present study was conducted as an educational scholarship on physicians.
One of the limitations of the study was that not all physicians in the target group participated in the workshops despite follow-up, reminders, formal correspondence, extensive information, incentives, and retraining scores. In order to eliminate this limitation, the researcher printed a poster of the nine indications for MRI prescription for patients with headache and distributed it in all hospitals and educational centers.
Conclusion
This study aimed to design and implement an empowerment course for physicians in evidence-based prescriptions of brain MRI at Bushehr University of Medical Sciences in 2021.According to studies conducted in Iran and Bushehr province, the evidence showed that physicians and assistants require training for evidence-based prescriptions and evidence of the effectiveness of empowerment courses and workshops on knowledge, attitude, and performance of physicians.
Based on the available statistics and evidence, the rate of inappropriate prescriptions for brain MRI in the province was about 30 %. In this study, educational planning was conducted and the workshop was designed and implemented. The workshop evaluation indicated positive results in the knowledge and attitude of the participants.
Thus, it is necessary for educational managers and policy makers of the health system to highly consider the role of education, especially continuous and in-service training to expect evidence-based prescriptions to improve the quality of services and reduce health system costs and also reduce out of pocket payments for people. In this regard, the diagnoses will be made in time with the shortening of the waiting line.
Acknowledgments
The research team would like to appreciate all the participants in the study. In addition, we would like to thank the Virtual University of Medical Sciences for approving and supporting this project.
Conflict of interests
The authors declared no conflict of interests.
Authors' contributions
Ahmadi S, Salari H, Esfandiari A, Habibi A, and Nemati R designed research; Ahmadi S, Salari H, Esfandiari A, Habibi A, and Nemati R conducted research; Salari H and Esfandiari A analyzed data; and Salari H and Esfandiari A wrote the paper. Salari H and Esfandiari A had primary responsibility for final content. All authors read and approved the final manuscript.
Funding
Non applicable.
Methods: The present study was an educational scholarship which was conducted in Bushehr province. This study was performed in two phases in accordance with Glassic criteria. A number of five workshops were designed and conducted by researchers using the ADDIE Model in systemic educational design. The course was designed through the review of texts and interviews with experts and in-depth group discussions (FGD) with the presence of seven experts. After that, this training course was evaluated and the end-of-course test was performed using the Kirkpatrick model. In order to evaluate the reaction of participants during the training course, a questionnaire was designed and its face validity and content validity were approved by experts. In addition, a four-choice test was designed based on the presented scientific content to evaluate the learning level of participants. A number of 40 questions were designed for five workshops. Descriptive statistics (frequency, mean) were used in SPSS22 for data analysis.
Results: In terms of gender, 53.7 % were male and 46.3 % were female. In terms of age, 24.4 % were in the age group of 31-35 years. In terms of improving awareness, 34.1 % of participants were at a very good level after the empowerment course. Learning findings indicated that 85.7 % of participants in the final test received a passing score while 14.3 % failed.
Conclusion: In general, the findings of the study indicated that holding an empowerment course for physicians in Bushehr improved their awareness and knowledge. In fact, such improvement in awareness and knowledge can reduce unnecessary prescriptions and treatment costs but improve the quality of services.
Key words: Empowerment, Evidence-based medicine, Design, MRI, Training course
Introduction
Evidence-based medicine refers to the ability and skill of the physician to use and integrate the best evidence obtained from repetitive and non-biased clinical trials along with the patient's preferences and circumstances. Currently, evidence-based medicine is considered as a new and reliable approach which has new discussions on judgment to find the best or qualitative evidence (1, 2). In evidence-based medicine, physicians make their decisions according to the medical information search process, critical evaluation of the retrieved information quality, matching of the current information with the desired disease, and finally the judgment and use of the best available evidence (2, 3). Unfortunately, there is often a gap between modern medical education and medical practice. Physicians can play akey role in solving this problem as one of the significant groups of learners and teachers. If the significance of evidence-based medicine is clarified for physicians and they can use it appropriately, it will be highly effective in providing appropriate clinical services. (2,3)The increasing diversity of the medical equipment market, as well as demand and expectations of physicians and patients for the selection and use of new medical technologies has resulted inan increase in health care costs, especially in the medical sector throughout the world. In particular, the competition between countries and within countries to provide more capital medical equipment and attract more patients has resulted in unnecessary allocation of equipment in some areas and hospitals, as well as the excessive and unnecessary use of these devices (4).
Nowadays, health care costs are increasing rapidly. In the US, health expenditures increased from 7 % of GDP in 1970 to 15 % of GDP in 2003 (5) and increased to 17.8 % in 2016 (6). Meanwhile, imaging is in the middle of this huge flood of diagnostic procedures and is regarded to be the fastest growing cost among medical services (7). The use of radiology services increased by 14.6 % during 1993-1999, including for almost 10 % of total medical services costs (8). However, a study conducted during 1997-2006 showed that the increase was 24 %-35 % in terms of number and was from 54 %-70.2 %in terms of cost (9). Inappropriate exploitation is a big problem in health care, especially in capital-oriented fields such as imaging (8).
With the introduction of new generations of medical diagnostic and therapeutic equipment in Iran during the recent years, we have witnessed the arrival of such devices to the country and follow-up is being performed for more devices to enter Iran (10). The first MRI machine arrived in Iran for the first time in 1991. Since then, the distribution of this technology has risen rapidly in the country so that the number of this technology increased 20 times per one million of the country's population during 1993- 2003 (11).
The use of clinical guidelines to improve the knowledge and skills of prescribing diagnostic and treatment procedures based on the indications seems necessary. This type of skill is related to evidence-based medicine and plays akey role in improving the professional ethics of physicians. Thus, the inclusion of evidence-based medical education, clinical guidelines, and prescriptions in line with valid indications can be highly helpful and necessary (12, 13).
Due to the significance of the issue and the need to increase the capabilities of physicians, it is expected to lead to better diagnoses and effective resource management. This process was designed in Bushehr province for evaluating physicians' empowerment courses in evidence-based MRI prescriptions in the training hospitals of Bushehr University of Medical Sciences in 2021.It should be noted that patients face long queues to receive such diagnostic services due to the limited number of MRI machines in Bushehr province. With the process of empowering physicians, the percentage of unnecessary prescriptions is expected to reduce, resulting in shorter waiting times, faster diagnoses, and better resource management.
Materials and Methods
The present study was an educational scholarship conducted in Bushehr province and aimed to evaluate the empowerment courses of physicians in evidence-based MRI prescriptions in 2021. This study was performed in two phases in accordance with classic criteria including clear goals, appropriate preparation, appropriate methods, considerable results, effective presentation, and thoughtful review.
Executive steps of the study
Design and implementation of physician empowerment course:
At this step, the necessary needs assessments were conducted using text review, expert opinion and Focus Group Discussions (FGD). In addition, scientific materials, indications and educational content were designed and implemented to empower physicians of the University of Medical Sciences in the form of workshops using the approach ADDIE (Analysis, Design, Development, Implementation And Evaluation).In the design part, the Kern curriculum planning model was used in four steps including problem identification and needs assessment, identification of general and specific objectives, determination of training methods, and determination of comprehensive assessment methods. This course, which included a total of five workshops was held for six months
In order to hold the workshops, the required coordination was performed with the education department of the university and the workshops were registered in the national system of continuous education for physicians. After that, the licenses were allocated for courses and workshops. Localized guidelines were used for educational content in previous studies (14, 15).
Description of participants
The studied population in the training course included the physicians of Bushehr province who were MRI prescribers, as well as the final year students and assistants. Forty physician participated in the study. Sampling method was census used. In fact, all participants were selected. The learners included general practitioners, internists, neurologists, neurosurgeons, emergency medicine specialists, and pediatricians, as well as internship students and assistants. After obtaining the permits for workshops, extensive information was performed in form of administrative letters, as well as other informal follow-ups.
Evaluation of participants
The evaluation process was performed using the Kirkpatrick model. In the evaluation stage of the first and second levels of the training course, the above-mentioned population was studied as a sample and the required data were collected using a questionnaire.
Data collection tools
The Kirkpatrick method is an appropriate method for the evaluation and measurement of the training effectiveness.
The level of reaction and learning was measured based on the above-mentioned model and the tools which are designed and described below. In order to evaluate the reaction of participants in the training course, an appropriate questionnaire was designed and its face validity and content validity were approved by experts. A four-choice test was designed according to the presented scientific content to evaluate the learning level of participants. A total of 40 questions were designed for five workshops. Face validity and content validity were confirmed by experts and the questionnaire reliability was confirmed by Cronbach's alpha of 0.8. The tests were carried out immediately after each workshop by the national system of continuous education. The participants were reported as passed and failed. For each workshop, eight scientific questions were designed based on the educational content of the teacher. All of the participants in the workshops participated in the test. The participants who answered at least five out of eight questions received a passing score.
The Kirkpatrick method is an appropriate method for the evaluation and measurement of the training effectiveness. This method is appropriate for evaluating and measuring the effectiveness of training. The Kirkpatrick model evaluates the effectiveness of programs at four levels: Level 1 evaluation: Reaction, Level 2 evaluation: Learning, Level 3 evaluation: Behavior and Level 4 evaluation: Result (16). Based on the above- mentioned model and the tools which are designed and described below, two levels of reaction and learning were measured.
Data analysis method
In order to analyze the data, descriptive statistics (absolute and relative frequency, mean, and standard deviation) and inferential statistics (independent t-test, ANOVA and Pearson correlation coefficient) were used in SPSS22. In addition, the significance level of the tests was considered as 0.05.
In order to satisfy the participants, the objectives of the research were explained to them and they were asked to participate.
This study was a part of two dissertations with ethics ID IR.VUMS.REC.1400.012 and IR.VUMS.REC.1400.013 approved in Tehran Virtual University of Medical Sciences.
Results
In this section, the considerable results, effective presentation, and thoughtful review of the current study are explained. Table 1 indicates the titles of the designed and implemented workshops related to the empowerment course for physicians at Bushehr University of Medical Sciences in 2021 which was conducted during six months. Tables 2 and 3 present the results of the demographic variables of age and sex of the participants and the answers to the checklist questions. Based on the findings, most of the participants were over 35 years old. In addition, the lowest percentage belonged to the group of 20-25 years. Furthermore, most of the answers were related to the very good category. About 34 % of participants reported that their awareness had improved significantly. The announced percentage related to other categories is given in Table 3.
Regarding the level of the course difficulty, the highest percentage of answers was related to the good category with a frequency of 53.7 %.
Regarding the training method, the highest percentage of answers in terms of training method was related to a very good category with a frequency of 41.5 %.
Based on the findings, 36.6 % of the participants in the course evaluated the completeness of its objectives at a good level.
In this study, the rate of satisfying the participants' expectations from the training course was evaluated at a good level (34.1 %) and very good (26.8 %).
The quality of the discussions raised in the training course was evaluated by the participants. Out of 41 participants, 14 participants (34.1 %) considered it good and 11 participants (26.8 %) considered it very good.
The highest answer of the participants in the training course in terms of the quality of group activities was related to the average option with 29.3 %. The details of these results are shown in Tables 1, 2 and 3.
Findings of participants' learning level
In order to evaluate the knowledge and learning of the participants in the training course, eight questions were designed for each workshop and the test was conducted through continuous training. The details of these results are shown in Table 4.
Effective presentation and thoughtful review
After the realization of the research, the results were presented as an educational poster and provided to all hospitals in the province and all physicians, medical students, and assistants. In the internal meetings of the hospital, thoughtful review was considered from the beginning of the process to the end. At each stage, the reviews were conducted in different stages of design and implementation with suggestions provided by experts.
For instance, due to the low participation of physicians at the beginning of the process, the heads of hospital were asked to consider motivational factors for participants, which had a highly positive effect on the presence and active participation of physicians in hospitals in the province.
Effective presentation: The present educational scholarship process was published in the following ways:
- Preparing posters
- Preparing pamphlets
- Presenting the educational scholarship results in the working group
- Submitting the article in the journal
Table1. The workshops of development course for physicians of Bushehr University of Medical Sciences in 2021
| Number | Topic | Length | Presenter |
| 1 | Evidence-based medicine | 2 hours | Dr. Hedayat Salari |
| 2 | Application of clinical guidelines and evidence- based medicine | 2 hours | Dr. Atefeh Esfandiari |
| 3 | Principles of development and adaptation of clinical guidelines | 2 hours | Dr. Hedayat Salari |
| 4 | Search methodology for clinical guides and reference databases | 1 hour | Dr. Hedayat Salari |
| 5 | The importance of studies types and level of the evidence | 1 hour | Dr. Hedayat Salari |
| 6 | Subject selection and prioritization of clinical guidance topics | 1 hour | Dr. Hedayat Salari |
| 7 | PICO formulation and determining Guideline Scope | 1 hour | Dr. Hedayat Salari |
| 8 | Teamwork includes: A- selecting the subject and determining the priority in each specialized field B- Searching at least 3 clinical guidelines C- Extracting questions from clinical guides D- Formulation of clinical questions for native guide |
||
| 9 | Appraisal tools and critical appraisal of clinical guidelines based on AGREE | 1 hour | Dr. Hedayat Salari |
| 10 | Introduction of Brain MRI reference guidelines | 1 hour | Dr. Reza Nemati Dr. Milad Sohrabi |
| 11 | Introduction of brain MRI prescription indications-section 1 | 1 hour | Dr. Reza Nemati Dr. Milad Sohrabi |
| 12 | Introduction of brain MRI prescription indications-section 2 | 1 hour | Dr. Reza Nemati Dr. Milad Sohrabi |
Table 2. Distribution of participants in the development course according to demographic variables
| Variable | Range | Numbers | Percent |
| Sex | Male | 22 | 53.7 |
| Female | 19 | 46.3 | |
| Age Groups | 20-25 years | 7 | 17.1 |
| 26-30 years | 8 | 19.5 | |
| 31-35 years | 10 | 24.4 | |
| Above 35 year | 16 | 39.0 | |
| Total | 41 | 100.0 |
Table 3. Results of evaluating the level of reaction of participants to the development course in Bushehr in 2021
| Question | Weak | Medium | Good | Very Good | Excellent | Total | |
| N* (p**) | N (p) | N (p) | N (p) | N (p) | N (p) | ||
| 1 | Improving awareness as a result of participating in a training course | 2 (14.9) | 5 (12.5) | 11 (26.8) | 14 (34.1) | 9 (22.0) | 41 (100) |
| 2 | Level of difficulty of the training course | 3 (7.3) | 11 (26.8) | 22 (53.7) | 2 (4.9) | 3 (7.3) | 41 (100) |
| 3 | Teaching method in the training course | 2 (4.9) | 2 (4.9) | 11 (26.8) | 17 (41.5) | 9 (22.0) | 41 (100) |
| 4 | Completeness of course objectives | 2 (4.9) | 2 (4.9) | 15 (36.6) | 14 (34.1) | 8 (19.1) | 41 (100) |
| 5 | Meet expectations | 2 (4.9) | 5 (12.5) | 14 (34.1) | 11 (2608) | 9 (22.0) | 41 (100) |
| 6 | Quality of discussions | 2 (4.9) | 5 (12.5) | 14 (34.1) | 11 (26.8) | 9 (22) | 41 (100) |
| 7 | Quality of group activities | 7 (17.1) | 12 (29.3) | 11 (26.8) | 5 (12.5) | 6 (14.6) | 41 (100) |
| 8 | The relationship between the training course and job activity | 1 (2.4) | 1 (2.4) | 14 (34.1) | 15 (36.6) | 10 (24.4) | 41 (100) |
| 9 | Adaptation of educational and media aids to the objectives of the course. | 2 (4.9) | 9 (22.0) | 13 (31.7) | 10 (24.4) | 7 (17.1) | 41 (100) |
| 10 | Overall quality of teaching aids | 4 (9.8) | 8 (19.5) | 11 (26.8) | 11 (26.8) | 7 (17.1) | 41 (100) |
| 11 | The ability of the instructor to control the training course | 2 (4.9) | 1 (2.4) | 11 (26.8) | 13 (31.7) | 14 (34.1) | 41 (100) |
| 12 | The quality of the feedback | 2 (4.9) | 4 (9.8( | 14 (34.1) | 14 (34.1) | 7 (17.1) | 41 (100) |
| 13 | Encourage learners to participate in discussions | 4 (9.8) | 8 (19.5) | 15 (36.6) | 9 (22.0) | 5 (12.5) | 41 (100) |
| 14 | clarity of the teacher's explanations | 2 (4.9) | 4 (9.8) | 9 (22.0) | 15 (36.6) | 11 (26.8) | 41 (100) |
| 15 | The overall effectiveness of the instructor | 2 (4.9) | 4 (9.8) | 11 (26.8) | 15 (36.6) | 9 (22.0) | 41 (100) |
Table 4. Results of competency test of physicians in evidence-based medicine in brain MRI prescription
| Results | Number | Percent |
| Accept | 132 | 85.7 |
| Reject | 22 | 14.3 |
| Total | 154 | 100.0 |
Discussion
This study aimed to design, implement and evaluate physician empowerment courses in evidence-based MRI prescriptions at Bushehr University of Medical Sciences teaching hospitals in 2021.This course, which included five workshops, was designed and implemented during six months. In addition, an educational poster was designed and provided to physicians through formal and informal ways. The Kirkpatrick learning model was used to evaluate and measure the effectiveness of the training course. In evaluating the first level of items to improve participants' awareness, difficulty of the course, teaching method of the course, completeness of the course objectives, meeting expectations, quality of discussions, quality of group activities, relationship between the course and job activity, compatibility of teaching aids, overall quality of teaching aids, ability of teacher, and its overall effectiveness were evaluated. The results of the first level indicated that most of the answers were related to very good and good options for most of the questions. Average and excellent options were in the next categories and the lowest answers were related to the weak option. The evaluation of the second level using the test indicated that the majority of participants passed the course while only about one-sixth of them failed. A study entitled “Effect of evidence-based medical education on the ability of reviewing articles by medical students of Tehran University of Medical Sciences: Was conducted by Rafiei et al. (17). This study was conducted on medical students of the internship at Tehran University of Medical Sciences in Winter 2007. The participants who were selected by convenience sampling method, participated in the evidence-based medicine workshop, which was held at the Student Research Center of Tehran University of Medical Sciences and received a standard questionnaire before and after participating in the workshop. Answering the evidence-based medical application questions in the critical assessment of articles after participating in the workshop had significantly increased, which is consistent with the results of the present study. It seems that the evidence-based medical skills in physicians and students increase by holding empowerment courses and training courses. Thus, education managers and health policy makers should pay special attention to this issue. For instance, a study entitled “Evidence-based medical education and its effect on knowledge, attitude and ability to use clinical students of Bushehr University of Medical Sciences in 2013”was conducted by Mirzaei et al.(18). This study aimed to determine the effect of short-term evidence-based medical education on the knowledge and ability of employing clinical medical students at Bushehr University of Medical Sciences. The above-mentioned study was a quasi-experimental control study before and after training which was conducted on two groups of medical students of clinical courses (intern students studying in the academic year 2012-2013) of Bushehr University of Medical Sciences. The students were selected by census method. At the beginning of the course, the students were trained in five standard stages of evidence-based medical practice in the intervention group after providing a description of the executive objectives of the process. Such states included questioning, searching, evaluating evidence critically, using and assessing in four sessions of four hours and then performed under the guidance of the teacher. Then, the level of knowledge and ability of students in intervention groups (N= 26) and control group (N= 23) in relation to the use of evidence-based medicine using valid and reliable questionnaires, pre-test and post-test (in the intervention group three months after the training course) was evaluated. The results of the study indicated an increase in students' familiarity with practical terms and resources in evidence-based medicine. In this study, the students' awareness of the process of performing evidence-based medicine and their ability to implement and use evidence-based medicine after the intervention were shown, which is consistent with the results of the present study. A study entitled “Analysis of the methods of teaching evidence-based medicine at universities inside and outside Iran in 2015” was conducted by Karimian et al. (19). In the study, different teaching methods and levels of effectiveness of the courses were studied with an analytical look at the related articles. This study was conducted with a theoretical analytical approach based on the library method and search for articles in scientific databases. The results revealed that teaching evidence-based medical courses begins during the early years of medicine in most studies. Although the lecture method was used in most of the trainings to present the concepts, the integrated approach and the use of active practical methods and group discussion was asignificant part of the students' training. In this review study, the following results were achieved: The studies which were conducted on the effectiveness of courses often measured satisfaction and knowledge while there is less evidence in the use of courses. Thus, it is necessary to pay more attention to the use of interactive and integrated methods with an interdisciplinary approach in the education of medical students. It is better to teach evidence-based medicine courses in a step-by-step way in accordance with the educational needs of the student in the basic sciences course. It is necessary to conduct further studies on the effectiveness of training courses in practice and behavior change. In addition, the present study conducted a training course to empower physicians and assessed satisfaction and awareness.
In some studies, such as the present study, a workshop training program was used and the results of the study indicated that holding a workshop has a positive result in the knowledge, attitude, and practice of learners. In a study by Taheri et al. (20) in 2006 that aimed to determine the effect of an evidence-based medical workshop curriculum on the skill of its use by medical students, 24 medical students in the fourth and fifth years who had passed the internal or surgery courses and participated in a workshop in three 5-hour sessions on evidence-based medicine. The difference between students' attitudes toward holding the workshop regarding the pre-test and post-test was significant, which is consistent with the results of the present study.
Some studies indicated that despite positive attitudes, evidence-based medical skills are weak. In a study by Amini et al. (21) entitled “The level of knowledge and use of evidence-based medicine among the specialized assistants of Shiraz University of Medical Sciences as one of the most significant groups of educators and learners”, the results indicated that 93.9% of the residents believed that using the results of research could improve their daily decisions, while 80.3 % of them failed to receive written training in evidence-based medicine. In addition, 80 % of respondents had access to the Internet, more than 70 %were unaware of evidence-based medical sources, and 60.9 % said that they had used Medline or other search engines less than 10 times for their patients in the past year. The results of this study revealed that specialized assistants, despite a positive attitude towards evidence-based medicine and access to the Internet for clinical decision-making, practically fail to use evidence-based medicine and are unaware of specialized evidence-based medicine websites. Perhaps the reason is that they have not received any formal training in this field. Due to the significance of this group in clinical decisions, as well as lower education, there is an urgent need for comprehensive education regarding the use of evidence-based medicine in this group.
In the present study, a set of empowerment workshops was designed and implemented for physicians and assistants due to the significance of this issue and the evaluation of the workshop indicated positive results.
Some studies revealed that even assistants have no sufficient knowledge about evidence-based medicine.
In a study by Kalvani et al. (22) in 2019 entitled “Assessing the level of knowledge and use of concepts and evidence-based medical databases among the specialized assistants of Shahid Beheshti University of Medical Sciences” the results revealed that the residents received a score of 2.9 for the level of knowledge and 2.7 for the use of evidence-based medical databases from the mean score of 5, indicating that the residents' knowledge and use of databases from evidence-based medical information is at the average level. Based on the results of their study, most of the residents have no sufficient knowledge about the concepts and databases of evidence-based medicine, so that most of the resources used to meet their information needs are the sources other than evidence-based information sources, thus, planning for evidence-based medical education and its bases to residents is necessary as highly regarded in the present study. A study by Yoon et al. (23) evaluated the continuing education empowerment program for physicians and assistants in hospitals using the Kirkpatrick model. In this study, the participants' reaction level was evaluated using a questionnaire. Behavior change assessment was conducted through the files written by physicians. The results indicated that the participants were satisfied with the training course so that the average was 4.4 out of 5. The performance score at the end of the period was 3.8 out of 5 while it was 2.3 at the beginning of the period.
The score for writing medical records rose from 2.9 to 3.3. Their study indicated that holding a continuous education course to empower physicians, which was designed and implemented with the full participation and responsibility of participants, was efficient and effective, which is in line with the results of the present study.
Solivetti et al. (24) in a study entitled "Appropriate rate of knee MRI Prescriptions: Clinical, economic and technical issues" conducted in Italy found that 21 % of all knee MRI prescriptions were completely inappropriate and 18.8 % of the total prescriptions were unreliable.
In fact, the most considerable result is that about 40 % of prescriptions were inappropriate or unreliable and general practitioners prescribe more than one-third of knee MRIs. Inappropriate MRI examinations and prescriptions have caused a significant financial burden in the national health system of that country and are in line with studies conducted in Bushehr province and show the need for educational empowerment courses. Sheehan et al. (25) in a study entitled "Decrease of unnecessary shoulder MRI prescriptions in a health care system: The potential role of shoulder ultrasound" realized that 106 cases(45 %) out of 237 MRIs prescribed for the shoulder area were inappropriately prescribed which were mostly due to the lack of previous radiography.
The physicians with non-orthopedic specialties had a higher percentage of inappropriate prescriptions than orthopedic specialists, i.e. 44 % and 17 %, respectively. In that study, ultrasound could show all the relevant pathologies in 157 cases, i.e. 66 % of cases in 237 cases of shoulder MRI. Such studies show the necessity and significance of designing and implementing training empowerment courses for physicians more than ever.
Andrade et al. (26) in a study entitled "Case study of demand-based supply: case study of imaging scans (CT Scan and Magnetic Resonance Imaging) in Manaus " concluded that the provision of imaging services in itself increases the demand for these services increases centers for diagnostic services, CT scans and MRIs in Manaus.
This issue indicates that if the supply of radiology services in Bushehr province increases, the possibility of induced demand and inappropriate prescriptions will increase. Therefore, policymakers and planners should consider the necessary measures and seriously, design and implement workshops on empowerment training.
Previous studies in the field of evidence-based medicine in the country often focused on medical students since it is easier to coordinate and monitor their work. In addition, such studies more focused on measuring knowledge, attitude or performance based on self-declaration. Nevertheless, the present study was conducted as an educational scholarship on physicians.
One of the limitations of the study was that not all physicians in the target group participated in the workshops despite follow-up, reminders, formal correspondence, extensive information, incentives, and retraining scores. In order to eliminate this limitation, the researcher printed a poster of the nine indications for MRI prescription for patients with headache and distributed it in all hospitals and educational centers.
Conclusion
This study aimed to design and implement an empowerment course for physicians in evidence-based prescriptions of brain MRI at Bushehr University of Medical Sciences in 2021.According to studies conducted in Iran and Bushehr province, the evidence showed that physicians and assistants require training for evidence-based prescriptions and evidence of the effectiveness of empowerment courses and workshops on knowledge, attitude, and performance of physicians.
Based on the available statistics and evidence, the rate of inappropriate prescriptions for brain MRI in the province was about 30 %. In this study, educational planning was conducted and the workshop was designed and implemented. The workshop evaluation indicated positive results in the knowledge and attitude of the participants.
Thus, it is necessary for educational managers and policy makers of the health system to highly consider the role of education, especially continuous and in-service training to expect evidence-based prescriptions to improve the quality of services and reduce health system costs and also reduce out of pocket payments for people. In this regard, the diagnoses will be made in time with the shortening of the waiting line.
Acknowledgments
The research team would like to appreciate all the participants in the study. In addition, we would like to thank the Virtual University of Medical Sciences for approving and supporting this project.
Conflict of interests
The authors declared no conflict of interests.
Authors' contributions
Ahmadi S, Salari H, Esfandiari A, Habibi A, and Nemati R designed research; Ahmadi S, Salari H, Esfandiari A, Habibi A, and Nemati R conducted research; Salari H and Esfandiari A analyzed data; and Salari H and Esfandiari A wrote the paper. Salari H and Esfandiari A had primary responsibility for final content. All authors read and approved the final manuscript.
Funding
Non applicable.
Type of Study: Original article |
Subject:
Health Policy
Received: 2021/11/17 | Accepted: 2022/01/30 | Published: 2022/03/29
Received: 2021/11/17 | Accepted: 2022/01/30 | Published: 2022/03/29
References
1. 1. Huddle TS, Heudebert GR. Taking apart the art: The risk of anatomizing clinical competence. Academic Medicine. 2007; 82(6): 536-41. [DOI:10.1097/ACM.0b013e3180555935]
2. Sackett DL, Rosenberg WM, Gray JM, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it isn't. British Medical Journal Publishing Group. 1996; 312(7023): 71-2. [DOI:10.1136/bmj.312.7023.71]
3. Heneghan C, Badenoch D. Evidence-based medicine toolkit. 2nd edn. Oxford: John Wiley & Sons; 2013.
4. Matsumoto M, Koike S, Kashima S, Awai K. Geographic distribution of CT, MRI and PET devices in Japan: A longitudinal analysis based on national census data. PLoS One. 2015; 10(5): e0126036. [DOI:10.1371/journal.pone.0126036]
5. Blackmore CC, Mecklenburg RS, Kaplan GS. Effectiveness of clinical decision support in controlling inappropriate imaging. Journal of the American College of Radiology. 2011; 8(1): 19-25. [DOI:10.1016/j.jacr.2010.07.009]
6. Papanicolas I, Woskie LR, Jha AK. Health care spending in the United States and other high-income countries. Jama. 2018; 319(10): 1024-39. [DOI:10.1001/jama.2018.1150]
7. Bresnahan BW. Economic evaluation in radiology: Reviewing the literature and examples in oncology. Academic Radiology. 2010; 17(9): 1090-5. [DOI:10.1016/j.acra.2010.05.020]
8. Lehnert BE, Bree RL. Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: How critical is the need for improved decision support?. Journal of the American College of Radiology. 2010; 7(3): 192-7. [DOI:10.1016/j.jacr.2009.11.010]
9. Smith-Bindman R, Miglioretti DL, Larson EB. Rising use of diagnostic medical imaging in a large integrated health system. Health Affairs. 2008; 27(6): 1491-502. [DOI:10.1377/hlthaff.27.6.1491]
10. Haji Aghajani M, Hashemi H, Haghdoost A, Noori Hekmat S, Janbabaee Gh, Maher A, et al. Distribution of major medical equipment in Iran in 2016 and the estimation of needs in 2026. Iranian Journal of Epidemiology. 2018; 13(5): 37-47. [In Persian]
11. Palesh M, Fredrikson S, Jamshidi HR, Tomson G, Petzold M. How is magnetic resonance imaging used in Iran?. International Journal of Technology Assessment in Health Care. 2008; 24(4): 452-8. [DOI:10.1017/S0266462308080598]
12. Kotur PF. Introduction of evidence-based medicine in undergraduate medical curriculum for development of professional competencies in medical students. Current Opinion in Anesthesiology. 2012; 25(6): 719-23. [DOI:10.1097/ACO.0b013e32835a1112]
13. Tamim HM, Ferwana M, Banyan EA, Alwan IA, Hajeer AH. Integration of evidence based medicine into a medical curriculum. Medical Education Online. 2009; 14: 15. [DOI:10.3885/meo.2009.F0000225]
14. Hagh Nedgahdar A, Salari H, Esfandiari A, Khammarnia M. Determination of prescription indications of brain MRI using appropriateness RAND in Shiraz University of Medical Sciences. Sadra Medical Journal. 2016; 4(1): 69-76. [In Persian]
15. Salari H, Ravanbod MR, Akbari Sari A, Farzanegan Gh, Esfandiari A. Developing appropriate indications for prescriptions of brain MRI using RAND appropriateness method. Evidence Based Health Policy, Management and Economics. 2017; 1(4): 205-10.
16. Kirkpatrick D, Kirpatrick JD. The kirkpatrick four levels. USA: Kirkpatrick Partners; 2011.
17. Rafiei S, Abdollahzadeh S, Ghajarzadeh M, Habibollahi P, Fayazbakhsh A. The effect of introducing evidence based medicine on critical appraisal skills of medical students. Iranian Journal of Medical Education. 2008; 8(1): 149-53. [In Persian]
18. Mirzaei K, Zahmatkesh S, Amini M. Effect of evidence-based medical education on knowledge and ability to use and apply it among clinical students of Bushehr University of Medical Sciences: A controlled trial. Iranian South Medical Journal. 2016; 19(3): 398-410. [DOI:10.18869/acadpub.ismj.19.3.398]
19. Karimian Z,Kojouri J,Sagheb MM. A review of evidenced-based medical education at universities in and outside Iran. Interdisciplinary Journal of Virtual Learning in Medical Sciences. 2015; 6(1): 64-75. [In Persian]
20. Taheri H, Mirrmohammad Sadeghi M, Adibi I, Ashourioun V, Sadeghizadeh A, Adibi P. The effect of an evidence based medicine workshop on undergraduate medical students' skills in applying Ebm. Iranian Journal of Medical Education. 2006; 6(2): 71-8. [In Persian]
21. Amini M, Sagheb MM ,Moghadami M, Shayegh S. The rate of knowledge and practice of medical residents of Shiraz Medical School in regard to evidence-based medicine. Strides in Development of Medical Education. 2007; 4(1): 30-5. [In Persian]
22. Kalavani A, Kazerani M, Shekofteh M. Evaluation of sbmu residents' knowledge and Use of evidence-based medical databases and concepts. PayavardSalamat. 2018; 12 (1): 34-43. [In Persian]
23. Yoon HB, Shin JS, Bouphavanh K, Kang YM. Evaluation of a continuing professional development training program for physicians and physician assistants in hospitals in Laos based on the Kirkpatrick model. Journal of Educational Evaluation for Health Professions. 2016; 13: 21. [DOI:10.3352/jeehp.2016.13.21]
24. Solivetti FM, Guerrisi A, Salducca N, Desiderio F, Graceffa D, Capodieci G, et al. Appropriateness of knee MRI prescriptions: Clinical, economic and technical issues. La Radiologia Medica. 2016; 121(4): 315-22. [DOI:10.1007/s11547-015-0606-1]
25. Sheehan SE, Coburn JA, Singh H, Vanness DJ, Sittig DF, Moberg DP, et al. Reducing unnecessary shoulder MRI examinations within a capitated health care system: A potential role for shoulder ultrasound. Journal of the American College of Radiology. 2016; 13(7): 780-7. [DOI:10.1016/j.jacr.2016.03.015]
26. Andrade EdO, Andrade ENd, Gallo JH. Case study of supply induced demand: The case of provision of imaging scans (computed tomography and magnetic resonance) at Unimed-Manaus. Revista Da Associacao Medica Brasileira. 2011; 57(2): 138-43. [DOI:10.1016/S0104-4230(11)70034-3]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
