Suchergebnisse

ABSTRACTBackgroundThe Institute for Clinical Evaluative Sciences (ICES) is a not-for-profit organization that conducts research to evaluate health care delivery and outcomes. Established in 1992, ICES houses a vast and secure array of linkable, coded health-related data on more than 13 million Ontarians, including health services data, health care provider data, registries and population-based health surveys. ICES has a reputation for generating strong evidence-based knowledge to inform policy and practice, however the use of the data was restricted to ICES' purposes. In March 2014, ICES launched the Data & Analytic Services (DAS) platform with the primary objective of increasing access to its data to publicly-funded researchers, health care providers and administrators, policymakers and students.
MethodDAS provides access to highly de-identified, risk-reduced datasets created from ICES' data holdings; analytic support; and complete data analysis and report writing services. DAS also enables the importation of external data for linkage to ICES' data holdings. Research objectives and methodology are led by the requestor and ICES analysts rely on their subject matter expertise to direct the deliverables.
ResultsSince launch, over 200 requests from Canada, United States and United Kingdom have been adjudicated, of which 187 have been deemed feasible and eligible. Over the same period of time, ICES as an organization had over 700 active projects of which 348 were initiated, an increase in capacity of 26%. Though Toronto-based researchers represent the majority of the requests (62%), there have been requests from outside Ontario interested in comparing aspects of Ontario's healthcare to their home province. Research topics have varied and include assessments of health care provision by sector, disease prevalence and treatment, and statistical methods. An unexpected outcome of increasing access has been the large interest from small physician groups and knowledge users who are not typically involved in research for academic purposes. Access to ICES' data holdings provides an opportunity to examine a larger cohort of patients who share the same characteristics as their clinic patients or group. Furthermore, by enabling remote access to the data, DAS is able to leverage the capabilities of ICES' data holdings and increase research capacity in a short period of time.
ConclusionIn making one of the most comprehensively linked health administrative data repositories in the world widely available to the broader research and healthcare community, DAS engages investigators involved in front-line care, stimulates new avenues of research and fosters collaboration that was previously unachievable.

Overall objectives or goalMost of the organizations that use population administrative data for research purposes have internal repository of validated definitions and algorithms of their own. Many of these concepts and definitions are applicable or at least adaptable to other organizations and jurisdictions. A comprehensive National (and potentially International) Concept Dictionary could help investigators to carry out methodologically sound work using consistent and validated algorithms using a shared pool of knowledge and resources.
The Institute for Clinical Evaluative Sciences (ICES) in Ontario, Canada has recently modernized its internal Concept Dictionary by adopting standard templates based on the Manitoba Centre for Health Policy (MCHP) Concept Dictionary, reviewing and updating existing content and tagging the concept entries with appropriate MeSH terms and data sources, and adding standard computer code (e.g., SAS coding) where appropriate. A SharePoint® web-based application has been developed to provide advanced tagging, searching and browsing features.
We envision a wiki-based Concept Dictionary hosted on a cloud-based environment with very granular access controls to provide enough flexibility for each participating organization to control their own content. This means each organization will be able to decide on how to share their own concepts (or part of them) with the public or internal users.
All content will be tagged with MeSH terms and as well with the organization's name that initially posts each entry. Other organizations which find the same concept applicable to their own use can tag the same entry with their organization name or refer to a secondary adapted entry if adaptation to fit their data and methodologies is required.
The Search feature will allow refining the search criteria by MeSH terms, data sources, and also organization/jurisdiction name.
Multiple layers of access controls will allow each organization to have their own groups of users with different standard privileges such as Local Administrators, Authors and Approvers (or Publishers).
The Approver (Publisher) users within each organization can publish each entry for internal or public view. This way, for example, a definition/algorithm can be viewable only within the organization until the validation process is complete, and then the entry can be made publically available, while some sections, such as computer code, can remain restricted to the organization.
We will discuss challenges in developing and maintaining such a platform including the costs, governance, intellectual property rights, copyrights and liabilities for the participating organizations.
The intended output or outcomeWe aim to use this opportunity to form a working group from the interested organizations that are ready to participate and commit in developing this collaborative platform. After the conference, there will be follow up sessions with the members of the working group to plan and develop the online application.

IntroductionIntegrating health and social services data is critical to understanding social determinants of health and responding to public expectations for evidence-based policies amidst changing demographics and fiscal constraint.
While academia has long understood the importance of social determinants of health, real and perceived obstacles have slowed their evaluation in Ontario.
Objectives and ApproachThis report describes how the Institute for Clinical Evaluative Sciences (ICES) and the Ministry and Community and Social Services (MCSS) have partnered to bring social services data and health data together to better understand the Ontario population and better support decision makers across various sectors.
We present how ICES and MCSS tackled barriers to data access and cultural challenges to data sharing in the Ontario context, provide an overview of their unique data and research partnership - including the new collaboration research and data access platforms created, highlight research findings to date, and identify key topics of interest moving forward.
ResultsOver the last decade, ICES and MCSS have led the way in Ontario linking health administrative and social services data. An initial single year linkage enabled the success of the Health Care Access Research and Developmental Disabilities project. This cross-sectoral initiative provided a clearer sense of how people with developmental disabilities experienced health care in Ontario.
Building on this work, ICES and MCSS recently expanded their partnership bringing together 15 years of social services and health data through a broader data sharing agreement. This agreement allows greater data access to researchers. In addition, ICES and MCSS have been successful in creating a new integrated research platform that will increase the depth and quality of health and social services research and policy evaluation in Ontario.
Conclusion/ImplicationsA broader collaborative research community will now be able to answer questions of interest, do self-directed integrated data analytics and leverage respective program data expertise to tackle joint research projects. Importantly, MCSS analytics teams will now also have access to linked data on this platform to conduct their own research.

ObjectiveIn March 2014, ICES launched Data & Analytic Services (DAS), expanding the access to ICES data and analytics beyond ICES scientists and analytic staff. In eight years, DAS has grown and evolved to increase high quality services offered to an expanding client base of external researchers.
ApproachAt the inception of DAS, two services were offered to public sector researchers: data access and analytics. Data access enabled researchers to analyze coded record-level data through a secure virtual environment. Analytics, conducted by DAS staff in ICES analytic environment, provided researchers with risk-cleared summary level reports. In response to growing demand from an increasingly diverse range of researchers, ICES engaged in extensive consultations with internal and external stakeholders to re-evaluate and operationalize new services. Compliance with contractual obligations and Ontario law, organizational capacity to scale up, alignment with ICES' mission, vision and values, were cornerstones in establishing new offerings.
ResultsAnalytic services became available to private sector researchers in June 2016. In March 2017, support for cohort and longitudinal follow-up studies became the newest service offering (researchers provided with a list of applicable individuals defined for the purposes of conducting publicly funded research). As more data assets become available to researchers, requests continue to increase in volume and complexity, particularly of projects seeking to import external data for linkage to ICES data. A second high performance computing virtual environment onboarded researchers September 2021 while the original analytic environment has undergone multiple upgrades, and will soon be fully refreshed. Regular solicitation of feedback has enabled DAS to increase staffing and diversify resources available which improves the client experience at all stages.
ConclusionsSince its inception, DAS has expanded from five to thirty personnel, grown and diversified its new and returning client base and has responded to demand for new services. DAS continues to provide high quality services which enable impactful research and is responsive to new opportunities for collaboration and service provision.

Overall objectives or goalMany health data linkage ecosystems across the world have designed and implemented secure data analysis environments as one of their controls to protect patient privacy and confidentiality. These have been shaped by local legislation and data governance policies, available IT infrastructure and resources, and the skills and imagination of their architects. However, at present their various features and functionalities have not been reviewed, synthesised or contrasted. Burton et al [1] have proposed 12 criteria for Data Safe Havens in health and healthcare, which they conceptualise broadly as encompassing data governance and ethics, quality and curation of data repositories, and data security. Under this definition, secure analysis environments, which may or may not be integrated with data repositories, are a component of a Data Safe Haven, addressing the criterion "Appropriate secure access to individually identifying data". To guide those building and operating these environments, and data custodians and stewards who need to assess their fitness-for-purpose, it would be of great value to discuss and agree an aggregate term (e.g. "Secure Data Lab") that describes them, and to develop a more detailed set of criteria for what entails "Appropriate secure access" to linked health data.
The goal of this session is to describe and document the approaches that have been taken by flagship secure data analysis environments internationally, including their approaches to authentication, assigning permissions, managing the ingress and egress of files and auditing transactions, and their responses to emerging opportunities, including cloud computing and national and international data sharing. We will explore how the interplay of physical, technical and procedural controls have been combined to create existing models, and the extent to which these can balance each other and be applied with flexibility depending on perceived risk and regimes.
Session structurePrior to the session, we will develop a draft set of criteria for "Appropriate secure access" to linked health data. The session will comprise presentations describing existing secure analysis environments against the draft criteria, followed by a facilitated discussion. The secure data analysis environments that will be presented include:

UNSW Sydney E-Research Institutional Cloud Architecture (ERICA)
PopData BC Secure Research Environment (SRE)
Institute for Clinical Evaluative Sciences (ICES) Data and Analytic Virtual Environment (IDAVE)
Secure Anonymised Information Linkage (SAIL) Gateway

Intended output or outcomeWe will write up the outcomes of the session as a scientific paper that proposes an aggregate term for secure data analysis environments for linked health data and a set of criteria for what entails "Appropriate secure access" to linked health data.
Presenters and Facilitators
Professor Louisa Jorm, Centre for Big Data Research in Health, UNSW Sydney, Australia
Dr Tim Churches, South Western Sydney Clinical School, UNSW Sydney, Australia
Professor Kim McGrail, Population Data BC, The University of British Columbia, Vancouver, Canada
J. Charles Victor, Institute for Clinical Evaluative Sciences, Toronto, Canada
Dr Kerina Jones, Swansea University Medical School, Wales, United Kingdom
Professor David Ford, Swansea University Medical School, Wales, United Kingdom
1. Burton PR, Murtagh MJ, Boyd A, et al. Data Safe Havens in health research and healthcare. Bioinformatics 2015; 31(20): 3241–3248

OBJECTIVE: : On May 31, 2006, Ontario joined a small group of international jurisdictions to implement legislative restrictions on tobacco point of sale promotions. This study compares the presence of point of sale promotions in the retail tobacco environment from three surveys: one prior to and two following implementation of the legislation. METHOD: : Approximately 1,575 tobacco vendors were randomly selected for each survey. Each regionally-stratified sample included equal numbers of tobacco vendors categorized into four trade classes: chain convenience, independent convenience and discount, gas stations, and grocery. Data regarding the six restricted point of sale promotions were collected using standardized protocols and inspection forms. Weighted estimates and 95% confidence intervals were produced at the provincial, regional and vendor trade class level using the bootstrap method for estimating variance. RESULTS: : At baseline, the proportion of tobacco vendors who did not engage in each of the six restricted point of sale promotions ranged from 41% to 88%. Within four months following implementation of the legislation, compliance with each of the six restricted point of sale promotions exceeded 95%. Similar levels of compliance were observed one year later. Grocery stores had the fewest point of sale promotions displayed at baseline. Compliance rates did not differ across vendor trade classes at either follow-up survey. Point of sale promotions did not differ across regions in any of the three surveys. CONCLUSION: : Within a short period of time, a high level of compliance with six restricted point of sale promotions was achieved.

IntroductionCanadian provincial health systems have a data advantage – longitudinal population-wide data for publicly funded health services, in many cases going back 20 years or more. With the addition of high performance computing (HPC), these data can serve as the foundation for leading-edge research using machine learning and artificial intelligence.
Objectives and ApproachThe Institute for Clinical Evaluative Sciences (ICES) and HPC4Health are creating the Ontario Data Safe Haven (ODSH) – a secure HPC cloud located within the HPC4Health physical environment at the Hospital for Sick Children in Toronto. The ODSH will allow research teams to post, access and analyze individual datasets over which they have authority, and enable linkage to Ontario administrative and other data. To start, the ODSH is focused on creating a private cloud meeting ICES' legislated privacy and security requirements to support HPC-intensive analyses of ICES data. The first ODSH projects are partnerships between ICES scientists and machine learning.
ResultsAs of March 2018, the technological build of the ODSH was tested and completed and the privacy and security policy framework and documentation were completed. We will present the structure of the ODSH, including the architectural choices made when designing the environment, and planned functionality in the future. We will describe the experience to-date for the very first analysis done using the ODSH: the automatic mining of clinical terminology in primary care electronic medical records using deep neural networks. We will also present the plans for a high-cost user Risk Dashboard program of research, co-designed by ICES scientists and health faculty from the Vector Institute for artificial intelligence, that will make use of the ODSH beginning May 2018.
Conclusion/ImplicationsThrough a partnership of ICES, HPC4Health and the Vector Institute, a secure private cloud ODSH has been created as is starting to be used in leading edge machine learning research studies that make use of Ontario's population-wide data assets.

IntroductionThere is widespread enthusiasm to improve health through the application of artificial intelligence and machine learning (AI/ML) methods to large population-level health datasets. Achieving this may require successful collaboration between institutions as well as between computer scientists (CS), machine learning researchers (MLR) and health service researchers (HSR). The objective is to describe lessons learned in creating the Health Artificial Data and Analysis Platform (HAIDAP) in Ontario, Canada.
Objectives and ApproachA partnership between a HSR institute (ICES), an AI/ML institute (Vector) and a high-performance computing center (HPC4H) was initiated in 2017 to enable the application of AI/ML methods to population-level health data for the province of Ontario (population 14M). The HAIDAP was launched in 2019. We describe lessons learned (and being learned) following the HAIDAP's launch.
ResultsMajor learnings include: 1)importance of institutional partnerships and alignment with institutional strategies; 2)potential of joint institutional risk-sharing models; 3)need for scientific collaborations bridging disciplines around joint research projects; 4)sensitivity to different scientific cultures (e.g., academic prestige of conference proceedings for MLR vs journal publications for HSR; traditional statistical vs. ML model assumptions); 5)differences in research timeline expectations; 6)different experience with and expectations for access to de-identified routinely collected data (e.g., need for research ethics committee project approvals and privacy impact assessments); 7)developing data access models that enable greater flexibility (e.g. importing code or using open source tools); 8) broadening data access models to allow modern high-dimensional exploratory data analysis; 9)obtaining support of information/privacy regulator; 10)hardware is (relatively) easy part compared to other success factors.
Conclusion / ImplicationsThe HAIDAP has enabled multi-disciplinary collaborations and novel AI/ML research of Ontario's population-level health data. Collectively we have learned that additional effort is required to develop systems and processes enabling more efficient access to data and analytic tools for the analysis of administrative health data.

IntroductionResearchers and decision makers from across Canada use linked provincial administrative data for analysis and to address research and policy questions. Currently there are several impediments to working harmoniously across provincial boundaries. A group of academic and policy researchers are working to address these multi-jurisdictional obstacles.
Objectives and ApproachResearchers and data organizations from across Canada are working together as the Pan-Canadian Real-World Health Data Network PRHDN). PRHDN aims to: (1) create harmonized data, algorithms and analytic protocols, and (2) link administrative databases to other types of data, including electronic medical records, clinical trials records, "omics data" and records from pan-Canadian cohort studies. PRHDN's vision is to construct a unified, documented infrastructure to advance pan-Canadian population-based research and analysis. This presentation incorporates material that is part of PRHDN's response to a funding call to create national, collaborative infrastructure.
ResultsScientists and staff at PRHDN organizations will create three main categories of infrastructure: 1) Algorithms: Reusable processes, ideally in the form of documented code, which implement a common approach or definition, e.g. to define cases or to create derived variables; 2) Harmonized Common Data: Based on the Sentinel model, we will establish a standardized subset of harmonized common data that are analysis-ready; 3) Common Analytic Protocols: Complementing work of the Canadian Network for Observational Drug Effect Studies (CNODES), we will establish processes for distributed analysis with common analytic protocols and meta-analysis of results to provide pan-Canadian estimates. Source data would remain within jurisdictional boundaries and only aggregate results would be pooled across jurisdictions. Details of these approaches will be presented.
Conclusion/ImplicationsThis initiative will improve coordinated access to distributed data from across Canada that is built once then used by many stakeholders for a variety of purposes including: research, benchmarking, performance monitoring to identify gaps and opportunities for improvement, multi-jurisdictional evaluations of novel interventions and inter-jurisdictional comparisons.

IntroductionImproving the care and management of patients with diabetes, particularly those with extreme blood glucose and/or cholesterol levels, has been identified as a key priority area for healthcare in Ontario. A multi-organizational collaboration produces audit-and-feedback reports distributed to consenting primary care physicians across the province for quality improvement purposes.
Objectives and ApproachWe examined the feasibility of linking the Ontario Laboratory Information System (OLIS), a large and nearly population-wide database of laboratory test results in Ontario, with the existing provincial audit-and-feedback reporting structure to integrate aggregated, physician-level measures of glycemic and cholesterol control among patients with diabetes.
All Ontario residents alive on March 31, 2014, attached to a primary care physician, and diagnosed with diabetes for at least two years were included. These patients were linked to OLIS to extract laboratory test orders and results for glycated hemoglobin (HbA1C) and low-density lipoproteins (LDL) between April 1, 2013 and March 31, 2014.
ResultsThere were 1,108,530 diabetes patients included who were assigned to 10,085 primary care physicians. During fiscal year (FY) 2013, 70%, 64%, and 59% of diabetes patients were tested for HbA1C, LDL, and both measures, respectively. Among the 648,238 diabetes patients with at least one of each test in FY2013, 13% had a HbA1C test exceeding a threshold of 9%, 4% had a LDL test exceeding a threshold of 4 mmol/L, and 0.8% exceeded both thresholds. At the physician-level, the median (Interquartile Range) proportions of diabetes patients exceeding the testing thresholds were 12% (9%-16%) for HbA1c and 4% (2%-6%) for LDL. In a multilevel logistic regression model, there was significant between-physician variability in the proportions of diabetes patients exceeding the HbA1C (p
Conclusion/ImplicationsWe developed a mechanism for integrating population-wide, clinical laboratory test results into physician audit-and-feedback reports to improve diabetes care in Ontario. Significant variation observed in the aggregated, physician-level proportions of diabetes patients testing above clinical thresholds for HbA1C and LDL highlights the importance of reporting such information to physicians.

IntroductionThe Ontario Brain Institute has developed Brain-CODE, an informatics platform, to support the acquisition, storage, management and analysis of multi-modal data. The standardized research data within Brain-CODE spans several brain disorders, allowing for integrative analyses, while also providing the opportunity to leverage existing clinical administrative data holdings through external linkages.
Objectives and ApproachWithin Ontario, the majority of individuals who access the healthcare system have a unique identifier, the Ontario Health Insurance Plan (OHIP) number. The OHIP number can facilitate linkages with administrative data holdings, such as those at the Institute for Clinical Evaluative Sciences (ICES). Given that OBI is not permitted under Ontario's privacy legislation to hold OHIP numbers, identifiers for consented participants are encrypted using a public key mechanism upon entry into Brain-CODE, where the private key is inaccessible. To facilitate linkages involving OHIP numbers between Brain-CODE and ICES, Brain-CODE Link software was co-developed by members of the Indoc Consortium.
ResultsBrain-CODE Link allows a deterministic linkage between encrypted identifiers (OHIP numbers), without revealing participant identity. The same homomorphic encryption algorithm applied to identifiers upon entry to Brain-CODE, is applied to relevant identifiers within ICES data holdings. Encrypted identifiers from Brain-CODE are securely transferred to ICES, where a comparison computation calculates differences between the encrypted sets. These differences are sent to a semi-trusted third party, who has no access to the original data, to decrypt the differences using the private key. A zero difference indicates a set of matching identifiers. One of the main challenges during testing and development of Brain-CODE Link was ensuring the software was capable of scaling to a population level, performing a large number of comparisons, in a computationally efficient manner.
Conclusion/ImplicationsOngoing pilot projects within the areas of epilepsy, neurodevelopment disorders, and neurodegeneration will be the first examples of linkages between Brain-CODE and ICES. Brain-CODE Link has successfully performed several billion test comparisons, indicating its suitability to function as a scalable privacy preserving record linkage to support comprehensive analyses.

IntroductionHealth care systems have faced unprecedented challenges due to the COVID-19 pandemic. Access to timely population-based data has been vital to informing public health policy and practice.
MethodsWe describe how ICES, an independent not-for-profit research and analytic institute in Ontario, Canada, pivoted existing research infrastructure and engaged health system stakeholders to provide near real-time population-based data and analytics to support Ontario's COVID-19 pandemic response.
ResultsSince April 2020, ICES provided the Ontario COVID-19 Provincial Command Table and public health partners with regular and ad hoc reports on SARS-CoV-2 testing and COVID-19 vaccine coverage. These reports: 1) helped identify congregate care/shared living settings that needed testing and prevention efforts early in the pandemic; 2) provided early indications of inequities in testing and infection in marginalized neighbourhoods, including areas with higher proportions of immigrants and visible minorities; 3) identified areas with high test positivity, which helped Public Health Units target and evaluate prevention efforts; and 4) contributed to altering the province's COVID-19 vaccine roll-out strategy to target high-risk neighbourhoods and helping Public Health Units and community organizations plan local vaccination programs. In addition, ICES is a key component of the Ontario Health Data Platform, which provides scientists with data access to conduct COVID-19 research and analyses.
Discussion and ConclusionICES was well-positioned to provide rapid analyses for decision-makers to respond to the evolving public health emergency, and continues to contribute to Ontario's pandemic response by providing timely, relevant reports to health system stakeholders and facilitating data access for externally-funded COVID-19 research.

IntroductionAround the world, many organisations are working on ways to increase the use, sharing, and reuse of person-level data for research, evaluation, planning, and innovation while ensuring that data are secure and privacy is protected. As a contribution to broader efforts to improve data governance and management, in 2020 members of our team published 12 minimum specification essential requirements (min specs) to provide practical guidance for organisations establishing or operating data trusts and other forms of data infrastructure.
Approach and AimsWe convened an international team, consisting mostly of participants from Canada and the United States of America, to test and refine the original 12 min specs. Twenty-three (23) data-focused organisations and initiatives recorded the various ways they address the min specs. Sub-teams analysed the results, used the findings to make improvements to the min specs, and identified materials to support organisations/initiatives in addressing the min specs.
ResultsAnalyses and discussion led to an updated set of 15 min specs covering five categories: one min spec for Legal, five for Governance, four for Management, two for Data Users, and three for Stakeholder & Public Engagement. Multiple changes were made to make the min specs language more technically complete and precise. The updated set of 15 min specs has been integrated into a Canadian national standard that, to our knowledge, is the first to include requirements for public engagement and Indigenous Data Sovereignty.
ConclusionsThe testing and refinement of the min specs led to significant additions and improvements. The min specs helped the 23 organisations/initiatives involved in this project communicate and compare how they achieve responsible and trustworthy data governance and management. By extension, the min specs, and the Canadian national standard based on them, are likely to be useful for other data-focused organisations and initiatives.