Purpose: This study aims to classify research impact indicators based on their characteristics and scope. A concept of evidence-based nomenclature of research impact(RI) indicator has been introduced for generalizatio...Purpose: This study aims to classify research impact indicators based on their characteristics and scope. A concept of evidence-based nomenclature of research impact(RI) indicator has been introduced for generalization and transformation of scope. Design/methodology/approch: Literature was collected related to the research impact assessment. It was categorized in conceptual and applied case studies. One hundred and nineteen indicators were selected to prepare classification and nomenclature. The nomenclature was developed based on the principle—"every indicator is a contextual-function to explain the impact". Every indicator was disintegrated into three parts, i.e. Function, Domain, and Target Areas.Findings: The main functions of research impact indicators express improvement(63%), recognition(23%), and creation/development(14%). The focus of research impact indicators in literature is more towards the academic domain(59%) whereas the environment/sustainability domain is least considered(4%). As a result, research impact related to the research aspects is felt the most(29%). Other target areas include system and services, methods and procedures, networking, planning, policy development, economic aspects and commercialisation, etc. Research limitations: This research applied to 119 research impact indicators. However, the inclusion of additional indicators may change the result. Practical implications: The plausible effect of nomenclature is a better organization of indicators with appropriate tags of functions, domains, and target areas. This approach also provides a framework of indicator generalization and transformation. Therefore, similar indicators can be applied in other fields and target areas with modifications. Originality/value: The development of nomenclature for research impact indicators is a novel approach in scientometrics. It is developed on the same line as presented in other scientific disciplines, where fundamental objects need to classify on common standards such as biology and chemistry.展开更多
Academic evaluations such as tenure/promotion applications and society fellowship nominations rely heavily on bibliometric measures of each candidate’s research impact, including their research citations. This articl...Academic evaluations such as tenure/promotion applications and society fellowship nominations rely heavily on bibliometric measures of each candidate’s research impact, including their research citations. This article first reviews existing evaluation criteria such as the h-index and<em> q</em>-most-citations, and then proposes a weighted w-index which minimizes shortcomings in existing single-number measures. The w-index consists of three factors<span style="white-space:nowrap;">—</span>3 most cited first-author publications, 3 most cited publications as the corresponding/last author, and 3 additional most cited publications as a co-author, but does not allow double counting of these publications.展开更多
Traditionally, the success of a researcher is assessed by the number of publications he or she publishes in peer-reviewed, indexed, high impact journals. This essential yardstick, often referred to as the impact of a ...Traditionally, the success of a researcher is assessed by the number of publications he or she publishes in peer-reviewed, indexed, high impact journals. This essential yardstick, often referred to as the impact of a specific researcher, is assessed through the use of various metrics. While researchers may be acquainted with such matrices, many do not know how to use them to enhance their careers. In addition to these metrics, a number of other factors should be taken into consideration to objectively evaluate a scientist＇s profile as a researcher and academician. Moreover, each metric has its own limitations that need to be considered when selecting an appropriate metric for evaluation. This paper provides a broad overview of the wide array of metrics currently in use in academia and research. Popular metrics are discussed and defined, including traditional metrics and article-level metrics, some of which are applied to researchers for a greater understanding of a particular concept, including varicocete that is the thematic area of this Special Issue of Asian Journal of Andrology. We recommend the combined use of quantitative and qualitative evaluation using judiciously selected metrics for a more objective assessment of scholarly output and research impact.展开更多
Although citation analysis is broadly used to design and develop bibliometric indicators or methods measuring the research impact,some limitations of citation analysis regarding citing motivation and behavior,database...Although citation analysis is broadly used to design and develop bibliometric indicators or methods measuring the research impact,some limitations of citation analysis regarding citing motivation and behavior,database coverage and bias,as well as the analytical methods may threaten the validity and reliability of the measurement.This paper reviews the literature on the limitations of citation analysis in the measurement of research impact,which is not addressed by many bibliometric studies introducing new indicators or methods for research evaluation,to remind researchers of the possible inappropriate use of citation-based indicators or methods on research evaluation.展开更多
This paper proposes a holistic framework for the development of models for the assessment of research activities and their impacts. It distinguishes three dimensions, including in an original way, data as a main dimen...This paper proposes a holistic framework for the development of models for the assessment of research activities and their impacts. It distinguishes three dimensions, including in an original way, data as a main dimension, together with theory and methodology. Each dimension of the framework is further characterized by three main building blocks： education, research, and innovation （theory）; efficiency, effectiveness, and impact （methodology）; and availability, interoperability, and ＂unit-free＂ property （data）. The different dimensions and their nine constituent building blocks are attributes of an overarching concept, denoted as ＂quality.＂ Three additional quality attributes are identified as implementation factors （tailorability, transparency, and openness） and three ＂enabling＂ conditions （convergence, mixed methods, and knowledge infrastructures） complete the fi-amework. A framework is required to develop models of metrics. Models of metrics are necessary to assess the meaning, validity, and robustness of metrics. The proposed framework can be a useful reference for the development of the ethics of research evaluation. It can act as a common denominator for different analytical levels and relevant aspects and is able to embrace many different and heterogeneous streams of literature. Directions for future research are provided.展开更多
Background: International research and innovation efforts for neglected tropical diseases have increased in recent decades due to disparities in overall health research funding in relation to global burden of disease....Background: International research and innovation efforts for neglected tropical diseases have increased in recent decades due to disparities in overall health research funding in relation to global burden of disease. However, within the field of neglected tropical diseases some seem far more neglected than others. In this research the aim is to investigate the distribution of resources and efforts, as well as the mechanisms that underpin funding allocation for neglected tropical diseases. Methodology: A systematic literature review was conducted to establish a comprehensive overview of known indicators for innovation efforts related to a wide range of neglected tropical diseases. Articles were selected based on a subjective evaluation of their relevance, the presence of original data, and the breadth of their scope. This was followed by thirteen in-depth open-ended interviews with representatives of private, public and philanthropic funding organizations, concerning evaluation criteria for funding research proposals. Results: The findings reveal a large difference in the extent to which the individual diseases are neglected with notable differences between absolute and relative efforts. Criteria used in the evaluation of research proposals relate to potential impact, the probability of success and strategic fit. Private organizations prioritize strategic fit and economic impact;philanthropic organizations prioritize short-term societal impact;and public generally prioritize the probability of success by accounting for follow-up funding and involvement of industry. Funding decisions of different types of organizations are highly interrelated. Conclusions: This study shows that the evaluation of funding proposals introduces and retains unequal funding distribution, reinforcing the relative neglect of diseases. Societal impact is the primary rationale for funding but application of it as a funding criterion is associated with significant challenges. Furthermore, current application of evaluation criteria leads to a primary focus on short-term impact. Through current practice, the relatively most neglected diseases will remain so, and a long-term strategy is needed to resolve this.展开更多
高校科研产出是体现高校教学科研能力的重要指标,是衡量高校学科发展水平内涵式发展的核心要素。文章利用ESI和InCites数据库,从Web of Science论文数、被引频次、引文影响力、H指数、Q1论文、国际国内合作论文、潜力值(被引频次/阈值)...高校科研产出是体现高校教学科研能力的重要指标,是衡量高校学科发展水平内涵式发展的核心要素。文章利用ESI和InCites数据库,从Web of Science论文数、被引频次、引文影响力、H指数、Q1论文、国际国内合作论文、潜力值(被引频次/阈值)等指标体系,对比分析安徽建筑大学的科研论文产出及影响力,预测优势及潜力学科,提出优化学科发展布局、拓展深层次的国际合作与交流等建议,为新一轮高水平建筑类大学学科建设提供基础支撑和要素保障。展开更多
文摘Purpose: This study aims to classify research impact indicators based on their characteristics and scope. A concept of evidence-based nomenclature of research impact(RI) indicator has been introduced for generalization and transformation of scope. Design/methodology/approch: Literature was collected related to the research impact assessment. It was categorized in conceptual and applied case studies. One hundred and nineteen indicators were selected to prepare classification and nomenclature. The nomenclature was developed based on the principle—"every indicator is a contextual-function to explain the impact". Every indicator was disintegrated into three parts, i.e. Function, Domain, and Target Areas.Findings: The main functions of research impact indicators express improvement(63%), recognition(23%), and creation/development(14%). The focus of research impact indicators in literature is more towards the academic domain(59%) whereas the environment/sustainability domain is least considered(4%). As a result, research impact related to the research aspects is felt the most(29%). Other target areas include system and services, methods and procedures, networking, planning, policy development, economic aspects and commercialisation, etc. Research limitations: This research applied to 119 research impact indicators. However, the inclusion of additional indicators may change the result. Practical implications: The plausible effect of nomenclature is a better organization of indicators with appropriate tags of functions, domains, and target areas. This approach also provides a framework of indicator generalization and transformation. Therefore, similar indicators can be applied in other fields and target areas with modifications. Originality/value: The development of nomenclature for research impact indicators is a novel approach in scientometrics. It is developed on the same line as presented in other scientific disciplines, where fundamental objects need to classify on common standards such as biology and chemistry.
文摘Academic evaluations such as tenure/promotion applications and society fellowship nominations rely heavily on bibliometric measures of each candidate’s research impact, including their research citations. This article first reviews existing evaluation criteria such as the h-index and<em> q</em>-most-citations, and then proposes a weighted w-index which minimizes shortcomings in existing single-number measures. The w-index consists of three factors<span style="white-space:nowrap;">—</span>3 most cited first-author publications, 3 most cited publications as the corresponding/last author, and 3 additional most cited publications as a co-author, but does not allow double counting of these publications.
文摘Traditionally, the success of a researcher is assessed by the number of publications he or she publishes in peer-reviewed, indexed, high impact journals. This essential yardstick, often referred to as the impact of a specific researcher, is assessed through the use of various metrics. While researchers may be acquainted with such matrices, many do not know how to use them to enhance their careers. In addition to these metrics, a number of other factors should be taken into consideration to objectively evaluate a scientist＇s profile as a researcher and academician. Moreover, each metric has its own limitations that need to be considered when selecting an appropriate metric for evaluation. This paper provides a broad overview of the wide array of metrics currently in use in academia and research. Popular metrics are discussed and defined, including traditional metrics and article-level metrics, some of which are applied to researchers for a greater understanding of a particular concept, including varicocete that is the thematic area of this Special Issue of Asian Journal of Andrology. We recommend the combined use of quantitative and qualitative evaluation using judiciously selected metrics for a more objective assessment of scholarly output and research impact.
文摘Although citation analysis is broadly used to design and develop bibliometric indicators or methods measuring the research impact,some limitations of citation analysis regarding citing motivation and behavior,database coverage and bias,as well as the analytical methods may threaten the validity and reliability of the measurement.This paper reviews the literature on the limitations of citation analysis in the measurement of research impact,which is not addressed by many bibliometric studies introducing new indicators or methods for research evaluation,to remind researchers of the possible inappropriate use of citation-based indicators or methods on research evaluation.
基金financial support of the Italian Ministry of Education and Research(through the PRIN Project N.2015RJARX7)Sapienza University of Rome(through the Sapienza Awards no.6H15XNFS)the Lazio Region(through the Project FILAS-RU-2014-1186)
文摘This paper proposes a holistic framework for the development of models for the assessment of research activities and their impacts. It distinguishes three dimensions, including in an original way, data as a main dimension, together with theory and methodology. Each dimension of the framework is further characterized by three main building blocks： education, research, and innovation （theory）; efficiency, effectiveness, and impact （methodology）; and availability, interoperability, and ＂unit-free＂ property （data）. The different dimensions and their nine constituent building blocks are attributes of an overarching concept, denoted as ＂quality.＂ Three additional quality attributes are identified as implementation factors （tailorability, transparency, and openness） and three ＂enabling＂ conditions （convergence, mixed methods, and knowledge infrastructures） complete the fi-amework. A framework is required to develop models of metrics. Models of metrics are necessary to assess the meaning, validity, and robustness of metrics. The proposed framework can be a useful reference for the development of the ethics of research evaluation. It can act as a common denominator for different analytical levels and relevant aspects and is able to embrace many different and heterogeneous streams of literature. Directions for future research are provided.
文摘Background: International research and innovation efforts for neglected tropical diseases have increased in recent decades due to disparities in overall health research funding in relation to global burden of disease. However, within the field of neglected tropical diseases some seem far more neglected than others. In this research the aim is to investigate the distribution of resources and efforts, as well as the mechanisms that underpin funding allocation for neglected tropical diseases. Methodology: A systematic literature review was conducted to establish a comprehensive overview of known indicators for innovation efforts related to a wide range of neglected tropical diseases. Articles were selected based on a subjective evaluation of their relevance, the presence of original data, and the breadth of their scope. This was followed by thirteen in-depth open-ended interviews with representatives of private, public and philanthropic funding organizations, concerning evaluation criteria for funding research proposals. Results: The findings reveal a large difference in the extent to which the individual diseases are neglected with notable differences between absolute and relative efforts. Criteria used in the evaluation of research proposals relate to potential impact, the probability of success and strategic fit. Private organizations prioritize strategic fit and economic impact;philanthropic organizations prioritize short-term societal impact;and public generally prioritize the probability of success by accounting for follow-up funding and involvement of industry. Funding decisions of different types of organizations are highly interrelated. Conclusions: This study shows that the evaluation of funding proposals introduces and retains unequal funding distribution, reinforcing the relative neglect of diseases. Societal impact is the primary rationale for funding but application of it as a funding criterion is associated with significant challenges. Furthermore, current application of evaluation criteria leads to a primary focus on short-term impact. Through current practice, the relatively most neglected diseases will remain so, and a long-term strategy is needed to resolve this.
文摘高校科研产出是体现高校教学科研能力的重要指标,是衡量高校学科发展水平内涵式发展的核心要素。文章利用ESI和InCites数据库,从Web of Science论文数、被引频次、引文影响力、H指数、Q1论文、国际国内合作论文、潜力值(被引频次/阈值)等指标体系,对比分析安徽建筑大学的科研论文产出及影响力,预测优势及潜力学科,提出优化学科发展布局、拓展深层次的国际合作与交流等建议,为新一轮高水平建筑类大学学科建设提供基础支撑和要素保障。
