Home About us Articles Multimedia Search Instructions Login 
IF 2017: 1.596 (® Clarivate Analytics)
Total Cites: 7606
Q2 in Medicine, General & Internal
Follow Us
Follow Us
  • Users Online: 1704
  • Home
  • Print this page
  • Email this page

 Table of Contents  
Year : 2017  |  Volume : 130  |  Issue : 23  |  Page : 2872-2877

Establishment of a Comprehensive Evaluation System on Medical Quality Based on Cross-examination of Departments within a Hospital

Medical Administration Department, Beijing Hospital, National Center of Gerontology, Beijing 100730, China

Date of Submission10-Jul-2017
Date of Web Publication24-Nov-2017

Correspondence Address:
Tong Chen
Medical Administration Department, Beijing Hospital,National Center of Gerontology, Beijing 100730
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/0366-6999.219163

Rights and Permissions

Keywords: Medical Management; Medical Quality; Medical Safety

How to cite this article:
Wu SW, Chen T, Pan Q, Wei LY, Xuan Y, Li C, Wang Q, Song JC. Establishment of a Comprehensive Evaluation System on Medical Quality Based on Cross-examination of Departments within a Hospital. Chin Med J 2017;130:2872-7

How to cite this URL:
Wu SW, Chen T, Pan Q, Wei LY, Xuan Y, Li C, Wang Q, Song JC. Establishment of a Comprehensive Evaluation System on Medical Quality Based on Cross-examination of Departments within a Hospital. Chin Med J [serial online] 2017 [cited 2018 Dec 16];130:2872-7. Available from: http://www.cmj.org/text.asp?2017/130/23/2872/219163

  Introduction Top

The management of medical quality is the core element of hospital management, and it is reflected in every part and aspect of medical behavior. Strengthening the management on medical quality is crucial in ensuring medical safety, promoting the ability of clinical diagnoses and treatments as well as the increasing the efficiency of medical service.[1] On this basis, the establishment and operation of a comprehensive evaluation system on medical quality is not only the foundation in ensuring the orderly operation of the medical behavior and medical safety but also an effective means in continuously improving the quality of medical management.[2] This paper aims to discuss the establishment of an operable and objective evaluation system as well as the corresponding assessment indicators on internal medical quality within the hospital based on cross-examination of departments through extensive research into various national and regional medical quality evaluation systems, and practice was carried out to achieve the purpose of the continuous improvement in medical quality as well as ensuring medical safety.

  Several Major Medical Quality Evaluation Systems Worldwide Top

Performance assessment tool for quality improvement in hospital of WHO

In 2003, the European WHO office proposed the performance assessment tool for quality improvement in the hospital (PATH) to help medical institutions evaluating their performance on medical service, and the results of the evaluation can be adopted in the continuous improvement of medical quality.[3] The evaluation framework of PATH consists of 6 dimensions within 4 areas (clinical effectiveness, efficiency, the adaptability of medical workers, and response management), as well as 2 transverse aspects (medical safety and the consciousness of patient-centered). According to the applicability as well as the cost in data collection, PATH divides the evaluation indicators into two categories as follows: core indicators and optional indicators.[4] Core indicators include the use of antibiotic drugs, the mortality of major diseases and surgeries, the re-admission rate of major diseases and surgeries, the monitoring indicators of day surgeries and so on. Optional indicators include the mortality of major diseases and surgeries through risk adjustment, the re-admission rate of major diseases and surgeries through risk adjustment, the occurrence of pressure ulcer within stroke and fracture patients, as well as hospital-acquired infection rate and so on.[5]

Joint commission on accreditation of healthcare organizations

The concept of the joint commission on accreditation of healthcare organizations (JCI) was first proposed by the joint commission on accreditation of international healthcare institutions, and all the evaluation standards of JCI are designed through the perspectives of patients.[6] JCI evaluation altogether covers 368 indicators (including 200 core indicators and 168 peripheral indicators). Each indicator contains several assessment items that sum up 1033 in total, all of which target to the most important sectors and aspects of medical treatment and nursery care in hospital management.[7] Notably, the objective of JCI management is to provide patients with the medical services that meet their various needs and increase the utilization of medical resources to promote life quality. Moreover, the core value of the evaluation is the continuous improvement of medical quality based on the patient-centered concept.[8]

International quality indicator project

International quality indicator project was formulated by the American center for performance research specialized in evaluating the clinical quality and safety of medical institutions.[9] It is the most widely used hospital medical quality evaluation system that specifically focusing on the results of medical service worldwide. There are altogether 30 categories of indicators focusing on the medical quality of general hospitals among the system, covering all aspects of medical care for outpatients and inpatients. For example, the inpatient mortality indicators include inpatient mortality, neonatal mortality, perioperative mortality and so on. Unplanned return indicators include the rate of unplanned readmission, the rate of unplanned readmission to Intensive Care Unit (ICU), the rate of unplanned readmission to the surgical operation, and so on. Hospital infectious indicators include ICU infection rate, surgery-related infection rate, methicillin-resistant Staphylococcus aureus infection rate, multiple drug-resistant bacteria infection rate and so on. Patients' safety indicators include the complications of falls, sedation and pain relieving, the incidence of pressure ulcers, the incidence of postoperative deep vein thrombosis and pulmonary embolisms, the incidence of outpatient falls, etc. Besides, the management of the ICU medical equipment usage, the use of antimicrobial agents in surgical procedures, the management of parturition process, the management of postoperative thrombosis preventions is also included in the evaluation system.[10]

Clinical Indicator Program

Clinical Indicator Program (CIP) was the unified national clinical service quality index system that was first developed and adopted in 1989 by the Australian Council on healthcare standards.[11] CIP was used in the evaluation of inpatients, outpatients and community health-care services, with a regular update of the system to ensure that the indicators are suitable and operable in clinical evaluation, and that are conducive in reflecting the problems of clinical procedures as well as improving the quality of medical service.[12] The classification of CIP system is based on the attribution of clinical departments, and hence that clinical indicators as well as indicators of medical auxiliary departments (such as radiology and pathology department) are all included in the system. Currently, 22 indicators in 353 clinical areas are included in CIP system.[13] Among them, procedure-related indicators include records-related indicators, time-related indicators as well as preventive indicators. While results-related indicators included readmission indicators, mortality indicators, hospital infectious indicators, patients' safety indicators, efficiency-related indicators, and so on.

Hospital rating system of the United Kingdom

Hospital rating system of the United Kingdom was first introduced by UK department of health in 2001, and it had been playing an important role in promoting the quality of medical service in the UK ever since.[14] The rating system mainly assesses two sectors of hospital services, which are the management of medical quality and service efficiency in spite of the scale sizes and technical levels of the hospitals. The rating system mainly includes four categories of indexes: medical services, clinical errors, patients' satisfaction, and staff performances. In specific, all the 21 evaluation indicators are divided into four categories: key indicators of medical service, indicators in related to patients, indicators in related to clinical procedures as well as indicators in related to clinical capacities and capabilities.[15] Among them, 9 evaluation indicators are defined as key indicators. When a hospital reaches all the standards of key indicators, it will be rated as three-star hospital; two-star will be rated if the hospital fails 1 or 2 standards. When a hospital fails 3 or more key indicators, it will be rated as one-star hospital or unqualified hospital.[16]

Hospital evaluation system of Japan

In 1995, Japan established the third party evaluation organization of medical institutions, and officially started the evaluation project in 1997. Differ from other assessment systems, the first procedure of the evaluation is document review. In this process, a fundamental understanding of the basic situation of the hospital is acquired through 5 types of the questionnaire by the evaluation organization, including questionnaire of the hospital, questionnaire of the headquarter departments, questionnaire of financial management, questionnaire of diagnostic ability, and questionnaire of discharged patients. Then, the hospital surveyed should propose the existing problems and potential solutions for the organization to assess the cognition of the existing problems. The final course of the evaluation process is the interview of the surveyed hospital. The interview is conducted by a group of expertise from the evaluation organization based on the review criteria, raw data and self-assessment results. Moreover, the contents of the interview include the tenets and organizational structure of the hospital; the satisfaction degree of local residents toward the hospital; the medical quality of diagnoses and treatments; the suitability and effectiveness of nursery services as well as the rationality of hospital management. A review meeting will be held by the evaluation group after the interview to discuss the results of the evaluation, and an official report will be formed and fedback to the hospital for continuous improvement.

  The Characteristics and Limitations of Medical Quality Evaluation System in China Top

The current evaluation system and key indicators of medical quality in China

The evaluation system that most widely applied in assessing the medical quality in China is the Chinese medical quality indicator system (CHQIS), and the evaluation indicators were generated according to the standards of tertiary general hospitals released by ministry of health in 2011, which emphasizes the public welfare of state-owned hospitals in the contents and indicators of the evaluation. CHQIS has generated the experiences of hospital evaluation of China in the past in combine with the evaluation standards of other countries and regions, such as JCI, the hospital evaluation systems of United States, Europe, and Japan. Meanwhile, the concept of plan-do-check-act was widely applied in the formulation of standards in fields of clinical management, nursery management, medical technology management, hospital infectious management, and so on. Moreover, the key monitoring indicators include: basic hospital operation indicators, medical quality and safety indicators of inpatients, management indicators of single diagnostic diseases, medical quality management indicators of ICU, indicators of rational drug application, indicators of hospital infection, etc.

Limitations of the medical quality evaluation system in China

The current medical quality evaluation system of China covers various aspects of medical management of public hospitals, such as medical services, patients' safety, the continuous improvement of medical quality and safety, nursery management, routine statistics, and so on. However, compared with other countries, we found that there are still limitations in the evaluation system. First, according to the approach of the evaluation, the indicators are divided into process-oriented indicators and results-oriented indicators. Most indicators included in the evaluation system are results-oriented indicators, and process-oriented indicators are relatively rare. However, extensive experiences showed that process-oriented indicators are more related to the quality of clinical services and that are able to reflect the outcome of clinical services more explicitly. Thus, it is imperative for medical evaluation system to adopt more process-oriented indicators in the assessment. In regards to the content of the current evaluation system, indicators concerning to work efficiency and quantity are far more than indicators concerning to quality and safety in medical management compare with evaluation system of other countries and areas. Besides, the evaluation cycle of the system is relatively long, thus it is not operable for the assessment and monitoring of daily medical quality within the hospital. Therefore, it is imperative for medical administration department of medical institutions to develop an evaluation system with specific assessment methods and indicators that are operable and effective for hospitals to carry out the internal evaluation.

  The Establishment of a Comprehensive Evaluation System of Medical Quality Based on Cross-examination of Departments within the Hospital Top

Therefore, to carry out medical quality assessment more comprehensively and practically within medical institutions, in the light of the existing evaluation system and key indicators, in combination with the characteristics of the hospital, an internal medical quality evaluation system based on cross-examination department was developed and had already been put into implementation in a grade-A tertiary hospital in Beijing for a year. According to the result of the assessments, significant improvement in the medical quality of the hospital has been going on through the process.

The framework and indicators of the evaluation system

The cross-examination evaluation system integrates the key indicators and procedures of medical quality as well as medical safety. Furthermore, it is suitable for management departments of the hospital to carry out statistics, analysis, and inspections with strong operability. The evaluation framework altogether consists of three parts: routine indicator evaluation, on-site evaluation, and survey on patients' satisfaction. The full score of the evaluation system is 100, the proportion for the three parts are 40%, 40%, and 20%. The results of evaluation are directly linked to the bonus salary of department staffs in each quarter.

Routine indicator evaluation

Most of the routine indicators are result-oriented, the evaluation is conducted mainly by medical statistics and management departments at the end of each quarter of the year and the scores are marked according to the performance of clinical departments. The indicators of routine evaluation mainly include indicators of in-patient and out-patient medical quality, indicators of nursery management as well as hospital infectious management, and the specifics are shown in [Table 1].
Table 1: Routine evaluation indicators of clinical departments

Click here to view

On-site evaluation

Differ from routine evaluation, most on-site evaluation indicators are process-oriented. The hospital established the panel of evaluation that consisted of experts within different major areas, and the experts within the panel are distributed into different department for cross-evaluation. The major contents of on-site evaluation include the management of medical records quality, the management of surgical operation safety, the implementation of key regulations on medical quality, and so on. The evaluation of medical auxiliary departments such as pharmacy, laboratory, and pathology departments are performed by experts from the quality control centers of Beijing in specialized major assessments to ensure the objectivity and impartiality of the evaluation. The design of specific indicators of on-site evaluation is shown in [Table 2].
Table 2: On-site evaluation indicators of clinical departments

Click here to view

Survey on patients' satisfaction

The survey on patients' satisfaction was regularly conducted by the follow-up center of the hospital toward discharged patients. The contents of the survey include the patients' satisfaction degree and suggestions toward the management of medical service, nursery service, wards environment as well as the management on auxiliary examination, and grades are scored according to the results of the survey.

  The Practice of Medical Quality Evaluation Based on the Cross-examination System of the Hospital Top

The general situation and results of medical quality evaluation of 2016 of the hospital

To examine the effect of the cross-evaluation system toward the improvement in medical quality, a practice was operated in a Grade A tertiary hospital of Beijing during 2016. The evaluation was carried out among 41 clinical departments within the hospital by quarters of the year, and the results of evaluation were fed back to the department for continuous improvement of medical quality and safety. The results of evaluation show that the overall average scores among the hospital are improving through the repeatedly evaluation, as it is shown in [Table 3]. While the general fractional distribution indicated the average scores of auxiliary department are the highest, followed by nonsurgical departments, and the average score of surgical departments are the lowest among the three departmental categories. As we analyze into the possible reasons to the variance within different departments, there are multiple factors that might be linking to the results. First, the evaluation indicators of medical auxiliary departments are relatively specialized compared with indicators of clinical departments and that the practitioners are more acquainted with so that they are inclined to perform better in the assessments. Second, the workloads of medical auxiliary departments are relatively less than clinical departments, so that the staffs of these departments are able to focus more on the management of medical quality. Third, since the evaluation experts of medical auxiliary department are chosen from quality control centers of Beijing instead of the panel within the hospital, the understanding of evaluation criteria may varied greatly among different individual and resulted to the deviation in grading. These facts may all lead to the general situation of the general fractional distribution mentioned above.
Table 3: Overall evaluation results of a Grade A tertiary hospital in 2016

Click here to view

The achievement obtained through the cross-examination evaluation

Through a year of comprehensive cross-examination evaluation between different departments, the medical quality of the hospital was greatly improved within the sampled hospital. As statistics showed that the number of discharged patients increased by 14.4% compared with same period last year; the average length of hospital stay decreased by 4.7% compared within last year; the rate of bed utilization increased by 2.6%; and the amount of the total surgical operation increased by 18.8%; the rate of Grade A medical records increased by 0.5%; the intensity and rate of antibiotic use decreased by 1.8% and 4.5%, respectively. On the contrary, the incidence of adverse medical events and medical tangles decreased significantly. All the key indicators of medical quality and safety showed a significant improving trend, as it is shown in [Table 4].
Table 4: General statistics of medical quality before and after the evaluation within the hospital

Click here to view

The characteristics, shortcomings of the system as well as the improvement plan in the next stage of evaluation

Compared with the existing evaluation system of medical quality that officially formulated, the cross-examination evaluation of medical quality system shares the advantages of aptness in operation, objectiveness in assessment, as well as the focus on both the process and results of every sector in medical management within the hospital. Medical administrative departments are able to identify the problem existed in clinical departments through the evaluation and continuously improvement can be made afterwards. In addition, since the results of the evaluation are directly linked with the quarterly salary bonus, the incentive effects are obvious for medical staffs. However, problems and shortcomings of the evaluation system were also exposed after a year of practice. As we mentioned above, some of the indicators failed to reflect the characteristics and specialties of different departments, and deviation were seen in indicator weight setting and the process of scoring due to the lack of homogenization in evaluation criteria of different experts. Therefore, constant revision of standards is also a crucial part for the continuous improvement management. Thus, in the next stage further refinement of the evaluation standards with specialty characteristics, as well as the increase in the index weight and contents of specialty evaluation (such as endoscopy center and hemodialysis room) to better reflect the medical quality of specialty characteristics will be add to the evaluation system for continuous improvement.

Limitations and the generalization of the research

According to the research, significant improvement in medical quality was going on in the hospital through the process, which could be provided as valuable experiences for other medical institutions to follow. However, since the theoretical research and practical operation was based on a single centered tertiary hospital, the generalization should be more carefully in other subordinate and specialized medical institutions in the setting of evaluation index and weight of scores based on local situation.

In conclusion, through the construction of the comprehensive evaluation system on medical quality based on cross-examination of departments within the hospital along with a year of practice, the medical quality of the hospital was greatly improved. However, problems and shortcoming still exist in the current evaluation system and indicators which call for the continuous improvement in the next Plan-Do-Check-Act (PDCA) round.

Financial support and sponsorship


Conflicts of interest

There are no conflicts of interest.

  References Top

Indira C. Medical equipment industry in India: Production, procurement and utilization. Chin Med J (Engl) 2013;57:203-7. doi: 10.4103/0019-557X.123242.  Back to cited text no. 1
Wu S, Ge Y, Xu X, Xuan Y, Chen T. Research of admission management standards and working patterns of first class medical technologies. Chin Med J (Engl) 2014;127:2860-2. doi: 10.3760/cma.j.issn.0366-6999.20140948.  Back to cited text no. 2
Kuchenbecker R, Polanczyk CA. Institutionalizing health technology assessment in Brazil: Challenge ahead. Value Health Reg Issues 2012;1:257-61. doi: 10.1016/j.vhri.2012.09.009.  Back to cited text no. 3
Wu SW, Chen T, Wu M, Pan Q, Xuan Y, Wei LY, et al. Management of medical technology under the new medical policy background in China. Chin Med J (Engl) 2016;129:2745-8. doi: 10.4103/0366-6999.193453.  Back to cited text no. 4
Rosen AK, Chen Q, Borzecki AM, Shin M, Itani KM, Shwartz M, et al. Using estimated true safety event rates versus flagged safety event rates: Does it change hospital profiling and payment? Health Serv Res 2014;49:1426-45. doi: 10.1111/1475-6773.12180.  Back to cited text no. 5
Zrelak PA, Utter GH, Sadeghi B, Cuny J, Baron R, Romano PS, et al. Using the agency for healthcare research and quality patient safety indicators for targeting nursing quality improvement. J Nurs Care Qual 2012;27:99-108. doi: 10.1097/NCQ.0b013e318237e0e3.  Back to cited text no. 6
Martin C, Odell K, Cappelleri JC, Bancroft T, Halpern R, Sadosky A, et al. Impact of a novel cost-saving pharmacy program on pregabalin use and health care costs. J Manag Care Spec Pharm 2016;22:132-44. doi: 10.18553/jmcp.2016.15180.  Back to cited text no. 7
Karlsberg Schaffer S, Sussex J, Hughes D, Devlin N. Opportunity costs and local health service spending decisions: A qualitative study from wales. BMC Health Serv Res 2016;16:103. doi: 10.1186/s12913-016-1354-1.  Back to cited text no. 8
McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003-2009. Med Educ 2010;44:50-63. doi: 10.1111/j.1365-2923.2009.03547.x.  Back to cited text no. 9
Thurgar E, Barton S, Karner C, Edwards SJ. Clinical effectiveness and cost-effectiveness of interventions for the treatment of anogenital warts: Systematic review and economic evaluation. Health Technol Assess 2016;20:v-vi, 1-486. doi: 10.3310/hta20240.  Back to cited text no. 10
Schaumberg A. The matter of 'fidelity': Keep it simple or complex? Best Pract Res Clin Anaesthesiol 2015;29:21-5. doi: 10.1016/j.bpa.2015.01.005.  Back to cited text no. 11
Naik VN, Brien SE. Review article: Simulation: A means to address and improve patient safety. Can J Anaesth 2013;60:192-200. doi: 10.1007/s12630-012-9860-z.  Back to cited text no. 12
Black N, Murphy M, Lamping D, McKee M, Sanderson C, Askham J, et al. Consensus development methods: A review of best practice in creating clinical guidelines. J Health Serv Res Policy 1999;4:236-48. doi: 10.1177/135581969900400410.  Back to cited text no. 13
Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: Methods and implications of rapid reviews. Implement Sci 2010;5:56. doi: 10.1186/1748-5908-5-56.  Back to cited text no. 14
Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007;7:10. doi: 10.1186/1471-2288-7-10.  Back to cited text no. 15
Tricco AC, Zarin W, Antony J, Hutton B, Moher D, Sherifali D, et al. An international survey and modified Delphi approach revealed numerous rapid review methods. J Clin Epidemiol 2016;70:61-7. doi: 10.1016/j.jclinepi.2015.08.012.  Back to cited text no. 16


  [Table 1], [Table 2], [Table 3], [Table 4]


Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

  In this article
Several Major Me...
The Characterist...
The Establishmen...
The Practice of ...
Article Tables

 Article Access Statistics
    PDF Downloaded63    
    Comments [Add]    

Recommend this journal