{"id":1743,"date":"2018-10-04T19:45:30","date_gmt":"2018-10-04T19:45:30","guid":{"rendered":"https:\/\/www.dimensions.ai\/?p=1743"},"modified":"2022-06-10T15:33:54","modified_gmt":"2022-06-10T15:33:54","slug":"dimensions-thoughts-on-how-can-bibliometric-and-altmetric-suppliers-improve","status":"publish","type":"post","link":"https:\/\/www.dimensions.ai\/blog\/dimensions-thoughts-on-how-can-bibliometric-and-altmetric-suppliers-improve\/","title":{"rendered":"Dimensions’ thoughts on “how can bibliometric and altmetric suppliers improve?”"},"content":{"rendered":"

This week, <\/span>UKSG Insights<\/span><\/i> published a <\/span>provocative new article<\/span><\/a> that contains important constructive criticism from research administrators and librarians that use bibliometrics and altmetrics tools, including Dimensions.<\/span><\/p>\n

The article rightly challenges metrics vendors on issues of transparency, coverage, responsible metrics, open metrics and services, and indicator development. It gave us a lot to think about and to celebrate, too–according to the article, apparently, one in four respondents were already using Dimensions only a month after we launched!<\/span><\/p>\n

After <\/span>author Lizzie Gadd invited our response on Twitter<\/span><\/a>, we thought it would be worthwhile to provide a lengthier, more detailed response than is possible in 280 characters.<\/span><\/p>\n

In this post, I highlight how we think that Dimensions does a good job of meeting the community\u2019s needs described in the article, and where we have room to grow.<\/span><\/p>\n

What we think we do well<\/strong><\/h5>\n

Partnering with the community<\/strong><\/p>\n

Dimensions was created from the ground up with input from over 100 community partners. We continue to develop Dimensions based on community feedback and suggestions. <\/span><\/p>\n

Moreover, we recognize and respect community ownership and stewardship for bibliometrics data. After all, open citation data is <\/span>what made Dimensions possible in the first place<\/span><\/a>. We do our best to give back in the form of open data that the community can use to validate existing citation-based metrics and innovate by creating new metrics.<\/span><\/p>\n

High-quality data as a starting point<\/strong><\/p>\n

As a \u2018new kid on the block\u2019, Dimensions has to focus on data quality. After all, we are up against other companies who have had decades to work on both coverage and quality. <\/span><\/p>\n

From the beginning, we have opted for precision over recall, for example in the areas of institutional affiliation matching and researcher disambiguation. In both areas, our approach is to use open standards and infrastructure: our own <\/span>GRID<\/span><\/a> for research organizations, <\/span>ORCID<\/span><\/a> for researcher disambiguation, and so on.<\/span><\/p>\n

Recall over precision means that we would rather split a researcher\u2019s body of work into several profiles rather than run the risk of assigning the wrong publication to a person. This data is easily amendable by the researcher herself, who can curate her ORCID record, which will be picked up in the next disambiguation run. <\/span><\/p>\n

Though we\u2019ve gotten some <\/span>good<\/span><\/a> feedback<\/span><\/a> from the community regarding our coverage and data quality to date, there is always room for improvement, which is why we have also listed \u201cdata quality\u201d as something we can improve upon–more below!<\/span><\/p>\n

More open, interoperable, and reusable data<\/strong><\/p>\n

We make our data available in open, standardized formats (primarily CSV and JSON), so that it is easy to repurpose our data in analyses, visualizations, and reports. We also make publication and citation data as open as possible (within the limits of some publishers\u2019 restrictions) through the <\/span>free Dimensions webapp<\/span><\/a> and the <\/span>open Dimensions Metrics API<\/span><\/a>. Scientometrics researchers can also apply for free access to Dimensions Plus and the Dimensions API, so they can use our data in their own research with minimal red tape and restrictions. We have also put a lot of effort into the <\/span>Dimensions API<\/span><\/a> and developed a domain-specific querying language to support the easy \u2018mash up\u2019 of Dimensions data with other data sources.<\/span><\/p>\n

Promoting the responsible use of metrics<\/strong><\/p>\n

We agree that metrics services have a \u201cduty of care\u201d to end users to help promote the responsible use of metrics. That\u2019s why we are signatories to <\/span>DORA<\/span><\/a>, and why we only include a limited number of carefully selected, community approved metrics in our products. We also have begun working with the academic scientometric community to redefine the metrics and their (responsible) presentation in Dimensions, which will be made public in an upcoming Dimensions release. We do our best to organize as many educational opportunities as possible for Dimensions users, including user days and webinars, and are constantly looking to expand and improve the educational services we offer.<\/span><\/p>\n

Article-level subject indexing<\/strong><\/p>\n

Survey respondents described the importance of accurate article-level subject indexing when benchmarking across institutions and niche subject areas. We agree that journal-level subject indexing (whereby an entire journal\u2019s identified subject area is applied to all the articles that are published in the journal, rather than each article being analyzed to determine its specific topic) is less than desirable, which is why we have taken an article-level indexing approach for Dimensions.<\/span><\/p>\n

We also have developed techniques to apply granular topics (based on the standardized Australian FOR subject areas) to journal articles, making the identification of research in niche subject areas much easier for those using our data in analyses. <\/span><\/p>\n

That said, we are well aware that our subject area coverage is not always 100% accurate nor as granular as it could be, so we are constantly working to improve what we do. For example, we are currently working on improving the training sets for our machine learning based classifications using various methods, from subject matter experts\u2019 input to journal level classifications implemented in the background, to broaden and improve the training sets. <\/span><\/p>\n

Finding the balance between innovation and the basics<\/strong><\/p>\n

When working on the concept for Dimensions, the balance between perfecting the basics and offering innovation was front and center in our discussions. It was clear that building a tool that allowed only citation-based analysis was too narrow, and that for any analysis a robust and relevant citation graph was required. <\/span><\/p>\n

We realized the value to be added if we created an inclusive, world-class publications index, and \u2018bolted on\u2019 other highly relevant data sources (such as a global grant database, patents, clinical trials and policy documents), consistently linking all the data together to one large dataset. In doing so, Dimensions allows a broader view on the trajectory of research from funding to later impact reflected in policy papers (to name just one example). <\/span><\/p>\n

What we\u2019ve ended up with is a tool that offers citation data for \u2018old fashioned\u2019 basic analysis, as well as a larger, comprehensive data set to be used by the scientometric community for the development of new metrics.<\/span><\/p>\n

We have put also a lot of care into taking fresh approaches to common bibliometrics challenges, for example by developing machine learning approaches that make possible subject-level article classification based on textual analysis. We also recognize the importance of getting right \u201cthe basics\u201d like researcher disambiguation and accurate article metadata, and work hard to do so.<\/span><\/p>\n

How we think we can improve<\/strong><\/h5>\n

Expanded coverage<\/strong><\/p>\n

Complete, accurate disciplinary coverage is in high demand from survey respondents, and rightly so. After all, it\u2019s difficult to do departmental and institutional bibliometric analyses using incomplete datasets!<\/span><\/p>\n

Providing complete coverage of all published research in all disciplines has major challenges, not the least of which include open data that can be reused without commercial restrictions, adequate metadata, and discoverability. Though we are currently one of the largest research indexes (at 96 million publications and counting!), we recognize that we are not as comprehensive as we could be. <\/span><\/p>\n

The concept behind Dimensions is to provide a true database, which is as inclusive as possible, and not a curated database, where a decision-making body decides which research \u201cmakes the cut\u201d for inclusion. <\/span><\/p>\n

By \u201cinclusive\u201d, we mean that we strive to strike the balance between providing comprehensive coverage (i.e. all scholarly work ever produced, no matter the publisher, source, or quality) and providing access to high-quality research (i.e. a very selective, restricted subset of research deemed \u201cexcellent\u201d by a small group of appointed experts).<\/span><\/p>\n

We believe that the decision-making power over what research is relevant belongs in the hands of the user – different use cases require different data scopes.<\/span><\/p>\n

In Dimensions, this has been realized by implementing various journal lists, like Pubmed and the Australian ERA 2015 journal list, which can be used to refine searches. And Dimensions is prepared to host community-provided lists to support discipline-specific journal sets, national selections, or quality-driven subsets of research; we are working with the community on integrating some at the moment.<\/span><\/p>\n

We\u2019ll continue to work within these constraints by partnering with like-minded organizations to get research into Dimensions so that it can be used by the larger community.<\/span><\/p>\n

Improved education and in-product \u201csignposts\u201d<\/strong><\/p>\n

The article authors point out that all vendors can promote the responsible use of metrics by \u201cmaking it very clear what their sources are, how the indicators are calculated and what their limitations are (e.g. sample sizes and confidence intervals)\u201d and offering \u201ceasy-to-find and comprehensive list of data sources for [their products].\u201d<\/span><\/p>\n

We enthusiastically agree, and add that in-product \u201csignposts\u201d could be added to most research indices to offer additional context for end users–for example, listing metrics\u2019 definitions, calculations, and limitations in pop-ups and knowledge base articles that are easily accessible wherever any metrics are displayed.<\/span><\/p>\n

We launched Dimensions with in-depth documentation to help our end users use our data responsibly, and we recognize that there are many ways we can make this information more accessible and robust. <\/span><\/p>\n

What do you think?<\/strong><\/h5>\n

Given the thoughtful challenges put to Dimensions and other vendors by the <\/span>UKSG Insights<\/span><\/i> article<\/span><\/a>, what do you think we are doing well? How do you think we can improve? Please do share your thoughts here in the comments, or by tweeting us at @DSDimensions on Twitter!<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Our response to a recent survey highlighting how we think that Dimensions does a good job of meeting the community\u2019s needs described in the article, and where we have room to grow.<\/p>\n","protected":false},"author":1,"featured_media":1744,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"latestblog_background":"","latestblog_bgcolor":"","latestblog_textcolor":"","latestblog_overlay":false,"inline_featured_image":false,"footnotes":""},"categories":[8],"tags":[],"resource_audience_segment":[],"class_list":["post-1743","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog"],"acf":{"author_name":"Stacy Konkiel","author_image":9277},"featured_image_urls":{"full":["https:\/\/www.dimensions.ai\/wp-content\/uploads\/2018\/10\/AdobeStock_189582516-1-e1559934852610.jpeg",800,300,false]},"post_excerpt_dimensions":"

Our response to a recent survey highlighting how we think that Dimensions does a good job of meeting the community\u2019s needs described in the article, and where we have room to grow.<\/p>\n","category_list":"Blog<\/a>","author_info":{"name":"admin","url":"https:\/\/www.dimensions.ai\/blog\/author\/admin\/"},"comments_num":"0 comments","_links":{"self":[{"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/posts\/1743","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/comments?post=1743"}],"version-history":[{"count":0,"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/posts\/1743\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/media\/1744"}],"wp:attachment":[{"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/media?parent=1743"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/categories?post=1743"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/tags?post=1743"},{"taxonomy":"resource_audience_segment","embeddable":true,"href":"https:\/\/www.dimensions.ai\/wp-json\/wp\/v2\/resource_audience_segment?post=1743"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}