{"id":12269,"date":"2020-03-04T08:39:48","date_gmt":"2020-03-04T08:39:48","guid":{"rendered":"https:\/\/www.dimensions.ai\/?p=12269"},"modified":"2022-06-10T15:50:37","modified_gmt":"2022-06-10T15:50:37","slug":"dimensions-api-story-taking-a-look-at-datasets-the-all-new-content-type-in-dimensions","status":"publish","type":"post","link":"https:\/\/www.dimensions.ai\/blog\/dimensions-api-story-taking-a-look-at-datasets-the-all-new-content-type-in-dimensions\/","title":{"rendered":"Dimensions API story: taking a look at datasets, the all-new content type in Dimensions"},"content":{"rendered":"\n

At Digital Science, we think that only looking at publications being the single measure for research impact does not live up nor pay tribute to all the amazing work that is done along the research cycle. That is why we have equipped Dimensions with all its content types to reflect all the contributions in all their facets. With the latest release in January, we added a new content type to Dimensions to further complete the picture: Datasets. <\/p>\n\n\n\n

Datasets play a vital role in the modern academic ecosystem, so much so, that nowadays funders begin to mandate publishing non-traditional outputs like datasets for achieving better reproducibility and replicability in research. In this blog post, we’ll take a look at datasets and what the Dimensions API has to offer, for further information about datasets in Dimensions, we recommend this blog post<\/a> to you.<\/p>\n\n\n\n

Taking a look at an all-new content type<\/strong><\/h2>\n\n\n\n

Since Datasets are a completely new content type in Dimensions, let\u2019s at first take a more general look at it: As you might know, with the Dimensions Search Language (DSL<\/a>), that was specially created for the Dimensions API to make it easy and comfortable to query, there is a certain command that can give you information about available fields, their type and how they can be used:<\/p>\n\n\n\n

%dsldocs datasets<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n


It will return a list of all fields, their type and where they can be used, including a written description. It will provide a neat overview of the new content type:<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

As a next step, we\u2019ll create an overview of how well they are populated for all of the nearly 1.5 million datasets records available in Dimensions.<\/p>\n\n\n\n

For this, we run through all of the fields above and have the API return the number of datasets that have this information present:<\/p>\n\n\n\n

search datasets where {field_name} is not empty return datasets limit 1<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

<\/p>\n\n\n\n

We\u2019ll have Jupyter\/Colab help us to run through all the fields available by replacing the {field_name} with the individual field from above. Once that is completed, we\u2019ll use the total_count<\/em> field in the response and have us create a bar chart for better visualization:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

We can see that half of the available fields on datasets are always populated while the other half holds more specific information when it\u2019s available like the associated journal, in case a journal article has referenced a certain dataset.<\/p>\n\n\n\n

As a last step of the general overview, let\u2019s see how this information is distributed over time:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n


Here we can see a trend of more and more metadata becoming available for datasets within recent years and we hope to see this trend continue!<\/p>\n\n\n\n

Conducting a topic search with datasets<\/strong><\/h2>\n\n\n\n

With a general understanding of how the data looks like and what can be seen in our query results, let\u2019s dig into some specific queries and use cases.<\/p>\n\n\n\n

We have prepared an example in our notebook<\/a> that queries the Dimensions API for datasets around a certain topic. We have chosen \u201cgraphene<\/em>\u201d as the topic for our blog post, if you\u2019re interested in a different topic or you would like to tweak the queries and reports differently, that is not a problem: We have created a Colab\/Jupyter Notebook<\/a> for you that you can use to conduct your own investigations. You can find a link at the end of the blog post. Something worth mentioning is, that with Colab you won\u2019t need to install any software on your computer, it works by only using your browser, so there is really nothing holding you back!<\/p>\n\n\n\n

At first, we\u2019ll query the API and retrieve the 100 most recent datasets related to the topic \u201cgraphene<\/em>\u201d. For this we\u2019ll use the following query:<\/p>\n\n\n\n

search datasets in full_data for “graphene”
return datasets[basics+license]  sort by date_created limit 100<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n


The DSL<\/a> makes it really easy to do and to tweak if you\u2019re more interested in another subject, like malaria or maybe you want to limit the results to datasets of the last three years?<\/p>\n\n\n\n

We run the query and have it plot the license information of the result:<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Great to see, all of the 100 most recent datasets have a CC license!<\/p>\n\n\n\n

Let\u2019s dig a little deeper and see how we can shed some light on the relations of datasets included in Dimensions. The next query is a little longer, but thanks to the DSL<\/a> it still is easy to read and understand. What we want to retrieve is a list of datasets that is related to \u201cgraphene\u201d (in full_data for “graphene”<\/em>) and has relations to grants and publications (where associated_grant_ids is not empty and associated_publication_id is not empty<\/em>).
In the result set, we would like to include the basic information on datasets (basics<\/em>), related publications and grants (associated_publication_id+associated_grant_ids<\/em>), as well as the FoR research category (category_for<\/em>):<\/p>\n\n\n\n

search datasets in full_data for “graphene”     
where associated_grant_ids is not empty and  associated_publication_id is not empty
return datasets[basics+associated_publication_id+associated_grant_ids+category_for]     
sort by date_created desc limit 50<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n


At first, we\u2019ll have a look at the resulting journal of our query:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

We can see, that we a large percentage of the datasets in question have been related to three journals: Frontiers of Chemistry, PLoS ONE and ACS Applied Materials & Interfaces.<\/p>\n\n\n\n

To view the funding side of our result set, we’ll have Colab\/Jupyter fetch the grant information related to our datasets from our query to see who funds grants that are related to our “graphene” datasets:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

We can see, that within the 50 most recent datasets, the National Natural and Science Foundation of China funds the majority of grants with a total count of 46, quite an impressive number! We also have the grant amount readily available, so we\u2019ll also look a the how the funding amounts are distributed:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n


In this diagram, we have each line represent a grant with the individual height being the funding amount. One thing that catches the eye is a large grant of 35 million euro of the NSF Directorate for Engineering we see related.<\/p>\n\n\n\n

<\/p>\n\n\n\n

As mentioned in our initial statement, we can observe that more and more funders are now mandating to also publish non-traditional outputs: An interesting question comes to mind, how is funding overall related to datasets? Can we see a pattern? We use another query to retrieve a list of top 100 funders being related to datasets and the term \u201cgraphene\u201d:<\/p>\n\n\n\n

search datasets in full_data for “graphene” 
return funders limit 100<\/td><\/tr><\/tbody><\/table>


<\/figcaption><\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

<\/p>\n\n\n\n

With this visualization, we can see that the majority of funders are related to several datasets, that is excellent and we hope to see this trend continue!<\/p>\n\n\n\n

<\/p>\n\n\n\n

Since all the research information in Dimensions is interconnected, and access is readily available, let\u2019s see what research organisations contribute to the \u201cgraphene\u201d datasets as the last step. We\u2019ll use another query to retrieve the ten research organisations with the most “graphene” datasets related:<\/p>\n\n\n\n

search datasets in full_data for “graphene” return research_orgs limit 10<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n


We\u2019ll convert that to a pie chart right away to get to the following overview:<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

<\/p>\n\n\n\n

<\/p>\n\n\n\n

We hope with this blog post we should share some insights on the new dataset content type in Dimensions and how you can utilize the API to generate helpful reports and uncover interesting connections. Of course, there is much more to explore, so we hope you take advantage of your notebook<\/a> for a headstart on your own report.<\/p>\n\n\n\n

How to do this yourself<\/strong><\/h2>\n\n\n\n

If you are interested and want to create your own patent-publication analysis, all you need is:<\/p>\n\n\n\n