VisualSphere: a Web-based Interactive Visualization System for Clinical

Research Data

Shiwei Lin, MS1,2,3, Shiqiang Tao, PhD1,2, Wei-Chun Chou, MS1,2,

Guo-Qiang Zhang, PhD1,2,*, Xiaojin Li, PhD1,2,*
1
Department of Neurology, McGovern Medical School, The University of Texas Health
Science Center at Houston, Houston, TX, 77030
2
Texas Institute for Restorative Neurotechnologies, The University of Texas Health Science
Center at Houston, Houston, TX, 77030
3
Department of Biostatistics and Data Science, School of Public Health, The University of
Texas Health Science Center at Houston, Houston, TX, 77030
*Corresponding authors: guo-qiang.zhang@uth.tmc.edu and xiaojin.li@uth.tmc.edu

Abstract
Clinical research data visualization is integral to making sense of biomedical research and healthcare data. The
complexity and diversity of data, along with the need for solid programming skills, can hinder advances in clinical
research data visualization. To overcome these challenges, we introduce VisualSphere, a web-based interactive
visualization system that directly interfaces with clinical research data repositories, streamlining and simplifying the
visualization workflow. VisualSphere is founded on three primary component modules: Connection, Configuration,
and Visualization. An end-user can set up connections to the data repositories, create charts by selecting the desired
tables and variables, and render visualization dashboards generated by Plotly and R/Shiny. We performed a
preliminary evaluation of VisualSphere, which achieved high user satisfaction. VisualSphere has the potential to serve
as a versatile tool for various clinical research data repositories, enabling researchers to explore and interact with
clinical research data efficiently and effectively.

1 Introduction
The widespread digitization of clinical research has yielded vast amounts of invaluable data1, including patients’
medical history, laboratory tests, diagnostic reports, medications, metabolomics and genomics profiles, disease
registries, and claims2. These are pivotal for advancing biomedical research, guiding clinical decision-making, and
improving healthcare delivery3. However, effectively utilizing clinical research data poses several challenges.
One major challenge is the limited programming and database background among clinicians and researchers4. Clinical
research data is typically stored in data repositories due to its size and diversity5, which can be difficult to access for
researchers unfamiliar with Structured Query Language (SQL) programming6. Although collaborating closely with
data management teams for data extraction can overcome this challenge, the procedure can be inefficient and time-
consuming due to the need for extensive communication7. Developing a user-friendly application that seamlessly
connects with clinical research data repositories would efficiently fill the gap between inadequate database knowledge
and the critical need for easy access to research data.
Another challenge is the lack of informative data rendering strategies8. Converting massive amounts of data into usable
information can be a daunting task. Incorporating visual analytics could enhance data representation in clinical settings,
reducing the cognitive burden for clinicians and researchers9. Information visualization can leverage researchers’
visual recognition capabilities to improve their interactions with the data and aid the identification of detailed insights
by reducing data complexity and revealing patterns9,10.
Many visualization tools have been developed to integrate and explore rapidly growing clinical research data11. Studies
such as HARVEST12 and RiskScape13 are problem-solving-oriented applications. HARVEST focuses on temporal
visualization of longitudinal patient records and data extraction from patient notes, while RiskScape facilitates
visualization and aggregation of near-real-time data from electronic health records (EHR) for investigating chronic
diseases. Another notable development is i2b2t214, a tailor-made visualization system for data from the clinical data
warehouse Informatics for Integrating Biology and Bedside (i2b2) using Tableau15. Tableau features a drag-and-drop
interface that eliminates the need for programming experience but requires commercial licenses for both development

603
and viewing by other researchers. Wiz16, alternatively, delivers high-quality interactive plots but requires dataset
uploads. Moreover, several libraries are available for user-customized data visualization, including D3.js17 and
Plotly.R18. D3.js (Data-driven Documents) is an open-source JavaScript library offering highly customizable and
interactive data-driven visualizations. However, it requires a solid programming background and prior knowledge of
data, which may pose challenges for researchers. Plotly.R is another open-source charting library in R19 that simplifies
the creation of interactive charts with less coding effort. While existing visualization tools and libraries provide
specific features for various aspects, as summarized in Table 1, none cover all the functionalities listed.
We introduce VisualSphere, a web-based interactive visualization system that aims to simplify and expedite clinical
research data visualization, making it accessible to researchers without requiring programming expertise.
VisualSphere comprises three integral component modules: Connection, Configuration, and Visualization. The
Connection module empowers users to establish connections to clinical research data repositories. The Configuration
module enables users to tailor and modify their desired visualization dashboards. The Visualization module generates
the interactive visualization dashboard for users to extract insights from the presented data. By eliminating the time-
consuming data downloads and uploads, the system streamlines the data visualization process. VisualSphere
recommends suitable chart types based on selected variables, facilitating the generation of appropriate charts.
Leveraging the R/Shiny20 framework, the system harnesses Plotly’s capabilities to produce interactive visualizations
that can be directly embedded on a website. VisualSphere is a versatile web system compatible with diverse clinical
research data repositories. It offers a comprehensive set of features (Table 1) to bridge the gap between data and
clinical researchers, enabling direct engagement with raw data, pattern identification, and cohort exploration for future
studies.
Table 1. Feature comparison among visualization systems and libraries for clinical research data.
System Library
VisualSphere Harvest i2b2t2 RiskScape Wiz Tableau Plotly.R D3.js
Pre-requisite
Commercial license free Yes Yes No Yes Yes No Yes Yes
No programming background required Yes Yes Yes Yes Yes Yes No No
Data
No database background required Yes Yes No Yes - No - -
Database compatibility Yes No No No - Yes - -
No prior data knowledge required Yes No No No No No No No
Visualization
Interactive visualization Yes Yes Yes Yes Yes Yes Yes Yes
No statistical background required Yes Yes Yes Yes No No No Yes
Chart recommendations by variables Yes No No Yes Yes Yes - -
Visualization results sharing Yes No Yes No No Yes Yes Yes
Visualization integration in other applications Yes No No No No Yes Yes Yes

2 Methods
2.1 System Architecture
Figure 1 illustrates the system architecture of VisualSphere, which consists of three core modules: Connection,
Configuration, and Visualization. The workflow begins with the user creating a connection to a data repository,
specifying the repository type, and inputting the requisite credentials. Upon a successful connection, users proceed to
the Configuration module to shape their visualization dashboard by adding charts. Within each chart, users select the
associated table, designate specific variables, and determine the chart type for display. The system offers
recommendations for chart types based on the number of variables and variable types. Finally, users can access the
Visualization module, enabling a detailed exploration of the clinical research data.
VisualSphere is developed using Ruby on Rails21, MongoDB22, R/Shiny, and Plotly. Ruby on Rails forms the
foundational framework of the website and handles server-side tasks. MongoDB serves as the backend database,
storing connected clinical research database information, user configurations for charts, ensuing modifications, and

604
logging activities. The system leverages the capabilities of Shiny and Plotly within the R environment to generate
interactive visualization dashboards.

Figure 1. System architecture of VisualSphere.

2.2 Connection
VisualSphere is designed to work with a relational database, MySQL23, and a non-relational database, MongoDB. The
connection process involves choosing the database type and entering authentication information. Users can upload a
data dictionary that outlines variables, codes, and values to augment visualization capabilities. This feature ensures
accurate labeling and heading display during visualization. VisualSphere provides straightforward ways to modify
connections and manage data dictionary files as needed. To maintain the integrity of each connection, VisualSphere
detects and prevents duplicate connections.
2.3 Configuration
After connecting to the data repository, users tailor their dashboards by specifying names, adding descriptions, and
integrating charts. For each chart, users must choose the relevant table, determine the number of variables, select the
corresponding variables, and decide on the chart type. To facilitate the configuration process, the system extracts
metadata from the connected database and compiles a list of tables and variables that users can navigate and select
from. VisualSphere employs a chart type model, as depicted in Figure 2, to assist users in making informed chart type
selections. Each variable is classified as categorical, continuous, or date. After users designate the variables for the
chart, the model recommends appropriate chart types for users to choose from. Users can modify their selections and
switch between chart types to customize the chart further. The backend database generates a unique ID for each
dashboard, which the R/Shiny application uses for visualization, as shown in Figure 3.

Chart Type Model

One Variable Two Variables

Categorical Continuous Date Categorical + Categorical + Continuous + Date + Categorical/

Categorical Continuous Continuous Continuous

Bar Chart Box Plot Histogram Grouped Bar Box Plot Scatter Plot Line Chart
Chart

Donut Chart Histogram Line Chart Stacked Bar Strip Plot Bubble Chart Scatter Plot
Chart

Pie Chart Violin Plot Scatter Plot Heatmap Violin Plot Regression Plot Area Chart

Figure 2. Chart type model. The model recommends chart types for both one-variable and two-variable scenarios,
enabling users to select suitable charts.
2.4 Visualization
The core components driving the visualization module encompass Shiny and Plotly. The Shiny application, hosted on
a dedicated Shiny server, provides the framework for interactive visualization. Once the visualization dashboard is

605
configured, the system passes the parameters to the application via a URL. The application executes R scripts to query
relevant data and generate visualizations based on user input, including connection and configuration details. Plotly
is used to generate the resulting visualizations automatically. Pre-designed templates for each dashboard simplify the
layout for users, allowing them to view and explore relevant data efficiently.
VisualSphere currently features fifteen unique graph types that deliver insights through hover- and click-driven
interactions. Powered by Plotly, the visualization module is equipped with various interactive functions, including
hovering, panning, zooming, selecting, and downloading plots. Moreover, the system harnesses the reactive function
of Shiny to offer interactive filtering. Mouse clicks trigger data filtering when users select specific data points,
categories, or time periods on the graphs. The data is filtered with each click based on the chosen points, and the
graphs are regenerated with updated data in real time. This interactive filtering function allows users to examine data
from multiple perspectives and identify potential patterns, ultimately yielding meaningful insights.

Figure 3. The workflow of the Visualization module. After the dashboard configuration is complete, both the database
connection information and user selections are passed to the Shiny application via a URL. The Shiny application
fetches the relevant data from the database and constructs the dashboard according to the user’s preferences.
2.5 Privacy
Ensuring data privacy and protecting sensitive information are paramount in systems dealing with clinical research
data. Institutions with stringent data governance policies or individual research groups with granted data access
permissions can host VisualSphere in their secure environment. During the visualization rendering process,
VisualSphere does not persistently store any clinical research data on its server. Data is transiently fetched for the sole
purpose of visualization and is not retained post-rendering.
2.6 System Evaluation
To assess the usability of VisualSphere, we invite participants from diverse research backgrounds and institutions.
The evaluation is divided into two sessions: one-on-one interviews and evaluation surveys. In the first session, we
introduce participants to the system, guiding them through tasks detailed in Table 2. These tasks mimic a clinical
scenario where researchers investigate patient demographics. We furnish precise and step-by-step instructions for each
task to guarantee participant consistency. Additionally, we document the time and steps taken by each participant to
complete the tasks.
After completing the tasks, we survey participants’ feedback. The survey questions, presented in Table 3, are adapted
from System Usability Scale (SUS)24. SUS, a widely utilized and validated evaluation, is appropriate for assessing the

606
Table 2. Tasks for evaluating VisualSphere.
Task Description
1. Database Connection Sign-up and log-in, connect to a clinical research data repository, and upload a data dictionary.
2. Dashboard Creation Create a dashboard and select charts with 3 demographics variables (age, race, and campus).
3. Dashboard Rendering Render a visualization dashboard and explore the patient demographics.
4. Dashboard Modification Modify the visualization dashboard: i. by adding one more chart; ii. switching the chart types.
5. Dashboard Exploration Render the updated dashboard and explore the visualization to identify patterns or outliers.

usability of general-purpose visualization systems like VisualSphere, especially with limited participants. Participants
rate each SUS question (Item 4 to Item 13, Table 3) on a scale of one to five, with one indicating strong disagreement
and five indicating strong agreement. Furthermore, the questionnaire includes three questions learning participants’
knowledge of clinical research data visualization. We also incorporate an open-ended comment section to gather
participants’ suggestions.
Table 3. Questions for the survey.
Item Question
1 Do you have any experience with clinical research data?
2 Do you have any experience with data visualization?
3 Do you have any experience with database operation?
4 I think that I would like to use this system frequently.
5 I found the system unnecessarily complex.
6 I thought the system was easy to use.
7 I think that I would need the support of a technical person to be able to use this system.
8 I found the various functions in this system were well integrated.
9 I thought there was too much inconsistency in this system.
10 I would imagine that most people would learn to use this system very quickly.
11 I found the system very cumbersome to use.
12 I felt very confident using the system.
13 I needed to learn a lot of things before I could get going with this system.
14 Comment or suggestion on the VisualSphere.

3 Results
VisualSphere has been deployed at The University of Texas Health Science Center at Houston (UTHealth Houston) 25
and adheres to the UTHealth Houston data use agreement. At present, access to the system is exclusive to the internal
research community at UTHealth Houston. To demonstrate the system’s functionality, we utilize it to visualize and
explore two distinct datasets, CVSToMe and the OPTUMâ de-identified COVID-19 Electronic Health Record dataset.
CVSToMe is a database system hosted at UTHealth Houston, consisting of 19,498 Cerebrovascular and Stroke
patients from eleven clinical campuses. The OPTUMâ dataset comprises large-scale EHR data (over 21 billion EHR
records) from approximately seven million patients drawn from dozens of healthcare providers in the United States,
including more than 700 hospitals and 7,000 clinicians7, 26. The results are presented following the three primary
modules of VisualSphere: Connection, Configuration, and Visualization.
3.1 Connection Interfaces
The initial step entails establishing connections to clinical research data repositories. Figure 4A displays active clinical
research database connections. Users can set up new connections by clicking the “New Connection” button and modify
existing connections by clicking the “Edit” button. Creating a data repository (Figure 4B) connection involves
selecting the database type and entering the required credentials. An illustrative example data dictionary has been
provided for user reference. We have integrated a data dictionary to improve the understanding of visualization since
the raw CVSToMe dataset contains encoded variables. Once the connection to the data repository is set up, users can
create and manage the visualization dashboard with the configuration interface.
3.2 Configuration Interfaces

Dashboard configuration begins with creating an empty dashboard, followed by assigning it a name and description
(Figure 5). The “Details” column lists charts and associated information for each dashboard. Users can update the
dashboard’s name and description by clicking the “Edit” button. Clicking the “Chart” button directs users to the charts
management interface (Figure 6A), where they can add new charts by clicking “Create Chart” and alter existing charts

607
by clicking “Edit.” The configuration process involves naming the chart, selecting a data table for connection,
determining the number of variables to include, and specifying the required variables. The system then recommends
the chart based on these variables. As depicted in Figure 6B, we employed the search function to select the “patients”
table and then chose “race” and “gender” for visualization. The chart type model automatically recommends suitable
chart types, defaulting to a “Stacked Bar Chart” in this case.

Figure 4. Connection interfaces. (A) The interface shows the established connections to clinical research databases.
(B) Setup process for connecting to a clinical research data repository.

Figure 5. Dashboard configuration interface.

3.3 Visualization Interfaces
After completing the configurations, the user can access the visualization dashboard. Figure 7 presents interactive
visualization tailored for the CVSToMe and OPTUMâ COVID-19 EHR datasets. The dashboard shown in Figure 7A
provides interactive visualization for the CVSToMe dataset with variables “gender,” “race,” and “campus” selected.
The “Back” button directs users to the configuration interface. The interactive filtering function is enabled as these
variables are from the same data table. Clicking on the “Male” of “gender” and “Black” of “race” dynamically updates

608
Figure 6. Configuration interfaces. (A) Chart management interfaces. (B) Chart configuration process.

Figure 7. Visualization interfaces. (A) Visualization dashboard for CVSToMe dataset. (B) Interactive filtering
function. (C) Visualization dashboard for OPTUMâ COVID-19 EHR dataset.

609
the “campus” plot, which then shows the count of black male patients across different campuses (Figure 7B). The
“Clear” button is provided to remove any applied filters. Figure 7C displays another dashboard for the OPTUMâ
COVID-19 EHR dataset, featuring three advanced charts: a grouped bar chart, a stacked bar chart, and a heatmap.
Zooming in and panning functions allow users to explore the population of interest. Hovering over the stacked bar
chart reveals the count of patients within each group. Table 4 highlights the system’s efficiency: the CVSToMe
dashboard renders demographics for 19,498 patients in 2.2 seconds, and the OPTUMâ COVID-19 dashboard takes
5.3 seconds to present advanced visualizations for 6,963,774 patients.
Table 4. Rendering performance evaluation using two datasets.
CVSToMe dataset OPTUMâ COVID-19 EHR dataset
Type of connected database MySQL MongoDB
Number of patients visualized 19,498 6,963,774
Number of charts displayed 3 single-variable charts 3 two-variable charts
Time for rendering the dashboard 2.2 seconds 5.3 seconds

3.4 System Evaluation

Ten researchers were recruited to evaluate VisualSphere. These researchers were from three health institutes, including
two biostatisticians, two bioinformaticians, three clinicians, and three data scientists. While all participants had
experience with clinical research data and information visualization, four had no prior experience with database
operation. Table 5 presents the completion rate, along with the average time and steps each researcher took to complete
the tasks listed in Table 2. Since exploring and modifying the visualization dashboards involved the user’s preference,
the average steps of tasks 3, 4, and 5 were not counted. All participants completed the assigned tasks, and most of the
participants (80%, 8/10) reached the minimum required steps for the first two tasks. On average, it took the ten
participants six minutes to connect to the clinical research data repository and explore the first visualization dashboard.
Table 5. Average time and steps taken for performing usability evaluation tasks.
Task Average time Average steps (Minimum steps) Completion rate
1 1min45s 11 (11) 100%
2 46s 7 (6) 100%
3 3min28s - 100%
4 25s - 100%
5 1min10s - 100%

The usability testing survey yielded positive results, as shown in Figure 8. Most participants (90%, 9/10, Figure 8.
Q1) expressed the desire to use the system frequently. All participants reported confidence in using VisualSphere,
with 90% (9/10, Figure 8. Q3) finding it easy to use. Regarding the system’s functionality, 90% of participants (9/10,
Figure 8. Q5) found the various functions of VisualSphere well-integrated. The four participants with limited
knowledge of database operation noted that VisualSphere could save significant time in future research endeavors.
VisualSphere is considered excellent with a final SUS score of 85.5, as SUS scores over 85 are typically viewed as
outstanding, and scores over 75 are deemed good27.

4 Discussion
Preserving data privacy stands paramount in today’s data-centric environments. Currently, VisualSphere is deployed
at The University of Texas Health Science Center at Houston. When processing, analyzing, and visualizing clinical
research data, VisualSphere strictly complies with the data policies outlined by UTHealth Houston. Access to the
system is exclusively granted to UTHealth Houston’s internal research community, reinforcing its dedication to
upholding the highest privacy and security standards.
Compared with seven other visualization systems and libraries, VisualSphere stands out with distinct features and
advantages. While tools like Tableau require commercial licenses for viewing and editing the visualization dashboard,
VisualSphere offers these capabilities without cost. Unlike many database-specific tools, VisualSphere boasts
versatility, catering to relational and non-relational data repositories. The modular design in VisualSphere enables
various customization, addressing the specific requirements of clinical researchers. The proposed chart type
recommendation model empowers users to delve into raw research data and engage in visual analytics without prior
data understanding, statistical expertise, and programming knowledge. The evaluation results demonstrated that

610
VisualSphere’s intuitive interfaces and user-centric design are crucial in reducing the learning curve and enhancing
accessibility for users to navigate and utilize the platform. Furthermore, VisualSphere distinguishes itself by the ability
to generate downloadable charts and embeddable dashboards for other web applications. These unique features
indicate that VisualSphere is a comprehensive, adaptable, and user-friendly tool for clinical research data visualization.
One limitation of VisualSphere is the chart type model, which is optimized for single-variable and two-variable
visualizations. While adequate for preliminary data exploration, the model needs to be refined to accommodate multi-
variable and high-dimensional data visualizations. Another limitation is VisualSphere’s data extraction capability.
Currently, it allows users to select variables from the same table while creating charts, limiting the depth of analysis,
especially in clinical research where multi-table data integration is common. We plan to develop an advanced query
builder that allows comprehensive queries across multiple tables and datasets, incorporating conditional logic and
aggregation functionalities.
Future work also includes expanding data connectivity to more clinical research data repositories, such as the
Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM)28 databases. This integration
aims to increase VisualSphere’s versatility and applicability in clinical research by harmonizing OMOP-CDM’s
standardized data structure for large-scale observational studies. We envision introducing a statistical analysis module
in the future. This module would empower users to access summary statistics for their target data, perform statistical
tests such as t-tests and chi-square tests, and construct statistical models like linear and logistic regression models.
Built on the R/Shiny framework, VisualSphere can leverage extensive packages customized for data manipulation,
visualization, and statistical modeling, improving the system to better meet the needs of clinical researchers.

Figure 8. Survey responses for the usability evaluation of VisualSphere.

5 Conclusion
The effective utilization of the vast amounts of data produced by modern clinical research is essential for advancing
biomedical research and enhancing healthcare outcomes. However, the massive volume of this data poses challenges
in its processing, analysis, and visualization. VisualSphere addresses these obstacles by simplifying the process of
exploring clinical research data. The system streamlines the visualization workflow for clinical researchers with its
user-friendly interface, eliminating the need for data re-entry or uploads by connecting to clinical research data
repositories. The proposed chart type model accelerates exploration by classifying variables and recommending
suitable chart types. Interactive functions and visualizations by Plotly and R/Shiny allow users to navigate clinical
research data and extract insights effortlessly. The preliminary evaluation of VisualSphere indicated high user
satisfaction, underscoring its potential as a comprehensive tool for diverse clinical research data repositories and its
ability to foster active engagement and interaction with clinical research data among researchers.

611
Acknowledgment
This work was supported in part by the National Institutes of Health (NIH) grant R01NS126690. The content is solely
the responsibility of the authors and does not necessarily represent the official views of the NIH.
References
1. Caban JJ, Gotz D. Visual analytics in healthcare – opportunities and research challenges. J Am Med Inform Assoc.
2015;22(2):260–2.
2. Wanderer JP, Nelson SE, Ehrenfeld JM, Monahan S, Park S. Clinical data visualization: The current state and
future needs. J Med Syst. 2016;40(12).
3. Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: Enabling
Reuse for Clinical Research. J Am Med Inform Assoc. 2013;20(1):144–51.
4. Glicksberg BS, Oskotsky B, Thangaraj PM, Giangreco N, Badgeley MA, Johnson KW, et al. PatientExploreR:
An extensible application for dynamic visualization of patient clinical history from electronic health records in
the OMOP common data model. Bioinformatics. 2019;35(21):4515–8.
5. Tao S, Cui L, Chou W-C, Lhatoo S, Zhang G-Q. DaT3M: A Data Tracker for Multi-faceted Management of
Multi-site Clinical Research Data Submission, Curation, Master Inventorying, and Sharing. AMIA Summits
Transl Sci Proc. 2022;2022:466–75.
6. Lee J, Ribey E, Wallace JR. A web-based data visualization tool for the MIMIC-II database. BMC Med Inform
Decis Mak. 2016;16(1).
7. Li X, Huang Y, Cui L, Zhang G-Q. A Query Engine for Self-controlled Case Series, with an application to
COVID-19 EHR data. AMIA Summits Transl Sci Proc. 2023;2023:350–9.
8. West VL, Borland D, Hammond WE. Innovative Information Visualization of Electronic Health Record Data: A
systematic review. J Am Med Inform Assoc. 2015;22(2):330–9.
9. Khairat SS, Dukkipati A, Lauria HA, Bice T, Travers D, Carson SS. The impact of visualization dashboards on
quality of care and clinician satisfaction: Integrative Literature Review. JMIR Hum Factors. 2018;5(2).
10. Gotz D, Borland D. Data-Driven Healthcare: Challenges and opportunities for interactive visualization. IEEE
Comput Graph Appl. 2016;36(3):90–6.
11. Chishtie JA, Marchand J-S, Turcotte LA, Bielska IA, Babineau J, Cepoiu-Martin M, et al. Visual analytic tools
and techniques in Population Health and Health Services Research: Scoping review. J Med Internet Res.
2020;22(12).
12. Hirsch JS, Tanenbaum JS, Lipsky Gorman S, Liu C, Schmitz E, Hashorva D, et al. Harvest, a longitudinal patient
record summarizer. J Am Med Inform Assoc. 2015;22(2):263–74.
13. Cocoros NM, Kirby C, Zambarano B, Ochoa A, Eberhardt K, Rocchio, SB C, et al. RiskScape: A data
visualization and aggregation platform for public health surveillance using routine electronic health record data.
Am J Public Health. 2021;111(2):269–76.
14. Harris DR, Henderson DW. i2b2t2: Unlocking Visualization for Clinical Research. AMIA Summits Transl Sci
Proc. 2016;2016:98–104.
15. Tableau: Business Intelligence and Analytics Software [Internet]. Tableau. [cited 2023 Mar 10]. Available from:
https://www.tableau.com/
16. Balzer C, Oktavian R, Zandi M, Fairen-Jimenez D, Moghadam PZ. Wiz: A web-based tool for interactive
visualization of Big Data. Patterns. 2020;1(8):100107.
17. Data-driven documents [Internet]. D3.js. [cited 2023 Mar 10]. Available from: https://d3js.org/
18. Plotly R graphing library [Internet]. Plotly. [cited 2023 Mar 10]. Available from: https://plotly.com/r/
19. The Comprehensive R Archive Network. [cited 2023 Mar 10]. Available from: https://cran.r-project.org/
20. Shiny [Internet]. Rstudio.com. [cited 2023 Mar 10]. Available from: https://shiny.rstudio.com/
21. Ruby on rails [Internet]. Ruby on Rails. [cited 2023 Mar 10]. Available from: https://rubyonrails.org/
22. MongoDB [Internet]. MongoDB. [cited 2023 Mar 10]. Available from: https://www.mongodb.com/
23. Oracle. MySQL [Internet]. [cited 2023 Mar 10]. Available from: https://www.mysql.com/
24. Brooke J. SUS: A ‘quick and dirty’ usability scale. Usability Evaluation In Industry. 1996;:207–12.
25. VisualSphere. [cited 2023 Mar 10]. Available from: http://neuroaplv001.uth.tmc.edu/visual_sphere/
26. Huang Y, Li X, Zhang G-Q. ELII: A novel inverted index for Fast Temporal Query, with application to a large
covid-19 EHR dataset. J Biomed Inform. 2021;117:103744.
27. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput
Interact. 2008;24(6):574–94.
28. OMOP Common Data Model [Internet]. The Observational Health Data Sciences and Informatics. [cited 2024
Jan 09]. Available from: https://ohdsi.github.io/CommonDataModel/

612