neuropapers db

2010 - 2023 Neuroscience Scientific Publications

Explore the documents »

This project involved the creation of a database of neuroscience publications. To achieve this, the first step was to gather information about journals with the highest impact factor in the field of neuroscience (SJR). Afterward, PubMed was scraped to retrieve all publications published in these journals between 2010 and 2023. Subsequently, the data was enriched with geographical coordinates, links to journals, and other relevant information

(Back to top)

The type of data collected from a scientific publication is illustrated in the following example and subsequently listed

• Title: Clear and concise, reflecting the content of the study.
• Authors: List of individuals responsible for the research.
• Abstract: A comprehensive summary of the main objectives, methodologies, findings, and implications of the study.
• DOI (Digital Object Identifier): For precise identification and referencing.
• Journal: Information about the journal of publication, including name, publication year, and month.
• Volume: Specific volume where the article can be found.
• Publication Date:
• Pages: Page numbers where the article can be found.

And other types of information as:

• Affiliations: Information about the affiliations of the authors.
• PubMed ID (PMID): Unique identifier for efficient retrieval in relevant databases.
• ISSN: International Standard Serial Number (ISSN) for electronic and linking formats.

(Back to top)

For the creation of neuropapers database, a diverse range of sources has been utilized to ensure comprehensive and robust data compilation.

(Back to top)

With pandas, data cleansing involved the process of detecting and rectifying any inconsistencies, inaccuracies, or anomalies within a dataset. Leveraging functionalities such as data filtering, handling missing values, and identifying outliers, data cleansing ensured the accuracy and reliability of the dataset.

Together with the data cleansing process, additional information was enriched in the dataset, including geographical coordinates of research centers. This enrichment enhances the dataset by providing supplementary details for a more comprehensive analysis.

(Back to top)

Thanks to the powerful SQLAlchemy library, the relational SQL database was created, relationships between tables were established, and tables were filled. Finally, MySQL software was used to check the viability of the data. The database architecture adheres to the following schema.

(Back to top)

I performed data visualization using a variety of tools, ranging from Python libraries like pylab and seaborn to software such as Tableau and MySQL. Additionally, I utilized mapping libraries like folium and pydeck for the creation of maps. Not forgetting the development of a web application using streamlit.

(Back to top)

I have created a model to predict in which journal it is more likely to submit your publication given the title and abstract. To achieve this, I preprocessed data from titles and abstracts, converted them into tensors with tensorflow, and reduced their dimensionality. Finally, the data has been fed into various classification models. Among them, I ultimately chose to use QuadraticDiscriminantAnalysis().

(Back to top)

• GitHub
• LinkedIn
• Website

(Back to top)

Requirements from Ironhack Spain for the Final Project.

Basic Knowledge:

• Proficiency in Python
• Proficiency in Data Transformation

Project Submission:

• README.md
• Folder structure with all the necessary documentation for the project
• Support material for the presentation

Specific Criteria:

Below is a table with 4 main sections. Each project must comply with the indication of at least 2 of the sections, as they fit each project:

Extraction:

• APIs
• WebScraping
• Automated file download
• Queries to Mongo
• Queries to SQL

Loading:

• Structured database creation (SQL) or Cloud creation
• Unstructured database creation (Mongo)
• API creation

Visualization:

• Use of at least one Python visualization library
• Use of PowerBI or Tableau
• Use of Streamlit or Flask

Machine Learning:

• Use of ML models to infer a numerical value
• Use of ML models to infer one or more categories
• Use of ML models for imputing null values

(Back to top)

I am sincerely grateful for the support and guidance provided by my teachers: Yona, Carlos, JeanCha, Rosella, and Jaime. Their dedication to fostering an environment of learning and nurturing their students' intellectual growth has profoundly impacted my educational journey at Ironhack.

• 💡 Yona
• 🏈 Carlos
• ⚡ Jean-Cha
• 🤖 Rosella
• 😊 Jaime

(Back to top)