Publications impact indexes

Frecuently asked questions

 

MEASURING SCIENTIFIC ACTIVITY

There are several different metrics which are used to measure and evaluate the quality of scientific publications.

Scientific production is measured and evaluated based on a series of criteria relating to the quantity of works published and the number of times that these works have been cited. Among these criteria it is helpful to highlight:

  • the number of works published
  • the total number of citations received
  • the average number of citations per work published
  • the number of significant works published
  • the number of citations received by the most frequently-cited works

 

What are publications with “impact”?

Publications with “impact” are those which are indexed in information portals that measure the quality and visibility of published material.

Metrics used for journals: impact indices or impact indicators

Impact indicators measure the impact that a journal has had on scientific literature. This impact is measured by analysing the citations received by the articles published within that journal. Impact indicators measure the importance of a publication within a specific subject area.

They make it possible to create comparisons between, and rankings of, journals in addition to measuring the relevance of each title in the subject area that it is associated with.

 

Tools for understanding impact indexes

There are several tools which enable users to look up and automatically calculate the impact indicators of the most important scientific journals:

  • Journal Citation Reports (JCR): It includes publications reviewed by the most widely-cited experts in the world and covers approximately 200 different disciplines. JCR can be accessed online via the Web Of Science platform (WOS) and can be used to run online searches and look up the Impact Factor of a given journal or a group of journals and make comparisons between these. The impact factor is calculated annually by the Institute for Scientific Information (ISI).
    • This is an essential tool for investigating the level of influence and impact that a journal has had on the international research community.
    • It does not measure the quality of an article but rather that of the journal in which the article was published.
    • Not every journal has a JCR impact factor and journals that do have one do not have this permanently.
    • The impact factor of a journal is updated every year and may vary from one year to another.
    • One single journal can be associated with several subject areas and it is likely that the journal will have a different level of impact within each of these subject areas.
    • The impact index of a journal title in any given year is a fixed index in JCR.
    • Each subject category of journals is divided into four quartiles: Q1, Q2, Q3, Q4. Q1 is occupied by the top 25% of journals in the list; Q2 is occupied by journals in the 25 to 50% group; Q3 is occupied by journals in the 50 to 75% group and Q4 is occupied by journals in the 75 to 100% group. The most prestigious journals within a subject area are those which occupy the first quartile, Q1.
    • There are two different versions: one for Science and the other for Social Sciences.
    • Coverage is provided from 1997 onwards. JCR is a paid-for tool which is funded by the FECYT (Spanish Foundation for Science and Technology) https://www.recursoscientificos.fecyt.es/
  • SCIMAGO Journal and Country Rank (SJR): This ranking emerged as a free-of-charge alternative to JCR. It analyses publications indexed in the Scopus database which is provided by the publisher Elsevier, dating from 1997 to the present. SJR enables users to run online searches using the Scopus platform, which is a paid-for tool, or using the SCIMAGO Journal and Country Rank-SJR.
    • SJR does not measure the quality of an article but rather the quality of the journal in which the article is published.
    • Not every journal has an SJR impact factor and journals that do have one do not have this permanently.
    • The impact factor of a journal is updated every year and may vary from one year to another.
    • The tool enables users to look up the impact factor of a given journal or group of journals and to make comparisons between these.
    • One single journal can be associated with several subject areas and it is very likely that the journal will have a different level of impact within each of these subject areas.
    • The impact index of a journal title in any given year is not a fixed index in SJR and may vary.
    •  SJR includes a greater number of journals than the JCR, making it less selective.
    • SJR is commonly used in science and social sciences. There are no separate versions for each subject area.
    • Each subject group of magazines is divided into four quartiles: Q1, Q2, Q3, Q4. Q1 is occupied by the top 25% of journals in the list; Q2 is occupied by journals in the 25 to 50% group; Q3 is occupied by journals in the 50 to 75% group and Q4 is occupied by journals in the 75 to 100% group. The most prestigious journals within a subject area are those occupying the first quartile, Q1.

 

How the impact factor is calculated in the Journal Citation Report (JCR)?

One way of determining the impact factor of a journal is by looking at the average number of times, in one year, that articles published in the previous two years have been cited.

A simple formula is used to calculate the Impact Factor (IF):

The total number of citations received in the previous two years

divided by

the total number of articles published in those previous two years

How the impact index is calculated by SJR?

The SJR impact index was developed based on the algorithm that was conceived by Google to organise its search results (Google PageRank). This means that not all citations carry the same value: rather, their value depends on the position of the magazine cited. This means, for example, that a citation from a journal with a high SJR index will have a greater value than a citation from a journal with a lower SJR index.

SJR details the number of links that a journal receives based on the weighted citation of its documents relative to the number of documents published in that year by each publication. The weighting of the citations is based on those received by the citing publication.

The citation period is three years - one year longer than JCR - and it can be calculated on a yearly basis from 1999, although data from Scopus publications have been compiled since 1996. In addition to this, the calculation disregards citations to documents published within the journal itself.

What are quartiles?

In addition to the Impact Factor or Impact Index, rankings of journals in each subject category are divided into quartiles by both JCR and SJR.

These quartiles rank the journals from highest to lowest based on their impact factor or impact index. There are four quartiles: Q1, Q2, Q3 and Q4.

Q1 is occupied by the top 25% of journals in the list; Q2 is occupied by journals in the 25 to 50% group; Q3 is occupied by journals in the 50 to 75% group and Q4 is occupied by journals in the 75 to 100% group.

  • Q1 is occupied by the top 25% of journals in the list
  • Q2 is occupied by journals in the 25 to 50% group
  • Q3 is occupied by journals in the 50 to 75% group
  • Q4 is occupied by journals in the 75 to 100% group.

The most prestigious journals within a subject area are those occupying the first quartile, Q1. The importance of the other journals declines as we move down through the quartiles.

Which are the differences between JCR and SJR?

One of the most important differences between SJR and JCR is the access that they provide:

  • JCR is a paid-for tool which is accessed via the Web Of Science platform. The database of citations on which it is supported – the Web Of Science Core Collection – is also a paid-for service.
  • SJR is free of charge. However, the database of citations on which it is supported – Scopus – is a paid-for service.

Another important difference between JCR and SJR relates to the variability of indicators:

  • the impact factor values provided by JCR are fixed and non-variable
  • the values provided by SJR and the other indicators on the platform are all variable

In terms of the citation data collected for calculating indicators:

  • in JCR the citation period covers two years and every citation has the same weighting and the same value
  • in SJR the citation period covers three years and the citations are all weighted, meaning that the value of the citation depends on the position occupied by the journal in which the citations are made

Metrics for journals: other indexes

The relatively small number of non-English-language journals and the fact that very few journals about Humanities and Social Sciences appear in the Web Of Science list and, by extension, in JCR has meant that in some countries, such as Spain, alternative tools have been created to measure the impact factor of journals.

We will now outline a series of tools which offer other indices that can be used to measure the quality of publications. This is a small selection of the tools that are currently available.

  • InRecs: http://ec3.ugr.es/in-recs/ provides impact indices of Spanish journals in the subject areas of Law and Social Sciences. Created by the Universidad de Granada. The information in InRecs has not been updated since 2014.
  • RESH: http://epuc.cchs.csic.es/resh/Revistas Españolas de Ciencias Sociales y Humanidades [“Spanish Humanities and Social Sciences Journals”]: this is a system which integrates quality indicators for Spanish scientific journals in the subject areas of Social Sciences and Humanities. It was developed within the framework of the Valoración integrada de las revistas españolas de Ciencias Sociales y Humanas mediante la aplicación de indicadores múltiples [“Integrated evaluation of Spanish Social Sciences and Humanities journals using multiple indicators”] project and financed by the National Research, Development and Innovation Plan.
  • Dice: http://epuc.cchs.csic.es/dice/this is the product of a partnership agreement between theSpanish National Research Council (CSIC) http://www.csic.es/ and theNational Agency for Quality Assessment and Accreditation of Spain (ANECA).http://www.aneca.es/It aims to promote awareness of, and research into, some of the editorial characteristics and indirect quality indicators of Spanish journals in the subject areas of Humanities and Social Sciences. ANECA uses this database as a quality benchmark for Spanish publications in their evaluations of teaching staff.
  • Scielo: http://www.scielo.org/php/level.php?lang=en&component=42&item=24 provides bibliometric indicators for Spanish journals in the area of Health Sciences. Created by Iberian-American institutions.
  • Latindex: http://www.latindex.org/latindex/inicio a regional online information system for scientific journals from Latin America, the Caribbean, Spain and Portugal. An initiative led by the Universidad Autónoma de Méjico. This tool provides the editorial characteristics that the system establishes through the application of editorial quality criteria.
  • Google Scholar Metrics: https://scholar.google.com/citations?view_op=top_venues this provides the metrics applied by Google Scholar and publishes a ranking of scientific journals.

 

Metrics for journals: the H-Index

The H-Index was created by Jorge E. Hirsch in 2005. It has since gained popularity and is applied to both metrics for journals and other metrics which apply specifically to authors. It can also be used to measure scientific productivity in different countries.

The h-index is based on a simple calculation which is used in a similar fashion regardless of the concept to be measured: this calculation involves ranking scientific articles from the highest to the lowest according to the number of citations that each one has received. The h-index is the point at which the article’s number in the ranking matches the number of citations that it has received.

 

Metrics used for journals: Google Scholar Metrics

Google Scholar Metrics includes the journals listed in Google Scholar that have published at least 100 articles receiving one or more citations.

In order to evaluate these journals,Google Scholar Metrics https://scholar.google.com/citations?view_op=top_venueshas drawn on the concept of the h-index and has created metrics known as the h5-index and h-median index. These indices calculate the citations received in the five complete calendar years prior to the realisation of the metric.

The main indicator adopted by Google is the h5-Index, which will be explained in greater detail below. Google Scholar Metrics understands the H-Index of a publication as being the highest h-number that arises when the articles cited in that publication are placed in descending order of more to less citations received and the highest number of citations received is still equal to or lower than the total number of articles on the list.For example, if a publication has 5 articles and these articles, when ordered by number of citations, have received 17, 9, 6, 3 and 2 citations respectively, we can conclude that the h-index of the publication is 3.

Using data based on the h-index, Google Scholar provides the h5- or ih5-index of publicationsby calculating the citations received in the previous five complete calendar years prior to the year in which the calculation was made (the publication date of the article is not considered in this calculation). The metrics are based on citations received by all of the articles indexed on Google Scholar up to a certain date.

Google Scholar also provides a ranking of journals based on the language in which they are published: it lists the 100 most important publications or the 100 journals with the most significant impact in each language. These publications are ordered by their h5 and h5-median indices. It is not currently possible to group and organise journals by their country of publication.