Search results

IntroductionIn observational health data, phenotyping algorithms are needed to process raw information into clinically relevant features. Validation studies traditionally estimate sensitivity and specificity by comparing the phenotyping algorithm with a reference standard on a population sample. There are challenges to conduct validation studies for conditions with low prevalence.
Objectives and ApproachWe propose a novel and efficient method for conducting validation studies to indirectly estimate the sensitivity and specificity. We simulated datasets with different levels of disease prevalence and phenotyping algorithms with different sensitivities and specificity. We applied both the traditional (direct) and new (indirect) method on simulated data to estimate the sensitivity and specificity and compare the performance of the two methods. We also designed a gate to exclude true negatives to improve study efficiency on conditions with low prevalence and sensitive analysis was conducted on the imperfect gate.
ResultsThe new (indirect) method provided better or comparable accuracy in estimating both sensitivity and specificity compared to the traditional (direct) method. Applying a gate enabled us to conduct validation study in conditions with very low prevalence. An imperfect gate results in the overestimation of sensitivity but has minimal effect on specificity.
Conclusion/ImplicationsThe new (indirect) method provides an alternative way to conduct validation studies in observational health data with improvement in estimating accuracy.

ABSTRACTObjectivesAdministrative health data including hospital discharge abstract data have been widely collected and analyzed for various purposes, including disease surveillance, case-mix costing, tracking healthcare system performance, policy-making and research. This study examined the coding validity of hypertension, diabetes, obesity and depression related to the presence of their co-existing conditions, death status and number of diagnosis codes in hospital discharge abstract data (DAD).
ApproachWe randomly selected around 4000 DAD records from four teaching hospitals in Alberta, Canada and reviewed their charts to extract 31 conditions listed in Charlson and Elixhauser comorbidity indices. Conditions associated with the four study conditions were identified through multivariable logistic regression. We examined the coding validity of the four study conditions related to whether their co-existing conditions were coded, whether the patient died in hospital and the total number of diagnosis codes recorded in a DAD record.Results Hypertension, diabetes, obesity and depression are generally secondary diagnosis and their validity are affected by the coding of their co-existing conditions. The sensitivity for the four conditions increased as the total number of diagnosis codes in the record increased. The impact of death status on coding validity for the four conditions was minimal.ConclusionThe coding validity of conditions is closely related to its clinical importance and complexity of patients' case mix. We recommend mandatory coding of certain secondary diagnosis to meet the need of health research based on administrative health data.

IntroductionSepsis is not well known. Only 58% of Americans know the word sepsis, less than 1% can identify its symptoms and one-third wrongly believe the disease is contagious (1). What if social media educated users about sepsis? There are at least 500 million tweets worldwide per day on Twitter.
Objectives and ApproachEarly detection of sepsis with early treatment is associated with a decrease in mortality (2). The current study aims to use Twitter to share sepsis patients' experiences. The approach consisted of using text data mining techniques by randomly extracting tweets (N =150) with the hashtag #sepsissurvivor (3) using R software (4). The study retrieved and quantified sepsis patients' tweets into word frequency distributions using documentation summarization and word cloud techniques (5) for visual representation of Twitter data. Sepsis patients used images symptoms cards (6) to raise awareness. The study used the R package "tesseract" to extract text from images (6).
ResultsPatients sharing their experiences frequently used the word "sepsis." Cardiorespiratory compromise (septic shock—the highest mortality risk) was illustrated in the words "my heart stops" or elevated "heart" rate or "low blood pressure." Several studies have reported increased mortality associated with delays in antibiotic administration (7). Many sepsis survivors had antibiotics exposure, both in a timely manner or delayed in use. Sepsis patients experienced long stays in the hospital. Tweets mentioned "infection" 39 times (8), which supports patients' diagnoses in addition to high rates of "fever." The clustering technique using word association indicated infection was highly correlated with sepsis (9). Sepsis survivors shared the "pain" they went through.
Conclusion/ImplicationsTwitter presents an opportunity for patients to disseminate information about sepsis raising awareness about important symptoms. The information tweeted explores the impact of this diagnosis, and the need for early treatment. The current study demonstrates the opportunity to raise awareness through the learned experiences of patients in a novel medium.

IntroductionFrailty is a state of vulnerability to diverse stressors emphasizing the importance of identifying the frail to support them. The burden of frailty in Canada is steadily growing. Today, approximately 25% of people over age 65 and 50% past age 85 – over one million Canadians – are medically frail.
Objectives and ApproachTo develop an administrative data definition of frailty to facilitate clinical and health system planning. We will validate the definition by linking the administrative data to electronic medical records (EMR) data. The EMR definition is based on a Machine Learning binarized frailty flag for patients with a Rockwood Clinical Frailty Score > 5 on physician chart audit. The sensitivity of the Machine Learning was disappointing: 28% (95% CI: 21% to 36%).specificity was: 94% (95% CI: 93% to 96%), positive predictive value: 53% (95% CI: 42% to 64%), negative predictive value: 86% (95% CI: 83% to 88%).
ResultsThere was little overlap between the EMR and administrative data definitions using the same population. Of the 29,382 eligible administrative data community dwelling patients over 65 years old, with a linkable EMR record, 2398 (8.15%) were identified as frail using the administrative data definition, but only 16.1% of these were frail according to the EMR definition. Of the 2396 who were identified as frail in EMR data, only 375 (15.7%) were identified as frail using the administrative data definition.
Conclusion/ImplicationsWe are not yet able to develop a reliable administrative data definition of frailty to identify community living individuals to support health service planning. The lack of agreement between the results obtained from EMR and administrative data definitions suggests that further refinement is necessary. Identification of frailty remains complex.

IntroductionThe problem list of a patient's primary care electronic medical record (EMR) generally reflects their important medical conditions. We will use association rule mining to assess between-provider and between-clinic variation in the coding of select conditions in the EMR problem list, in order to identify possible under-coding outliers.
Objectives and ApproachEMR data from participating clinics in the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) will be used, with a focus on three commonly-occurring conditions (hypertension, diabetes, and depression). Association rule mining will be used to develop association rules between these conditions and other clinical information available in the EMR, such as other diagnoses in the problem list, billing codes, medications, and laboratory results (e.g., a rule of "diabetic medication→diabetes" indicates that patients prescribed a diabetic medication are likely to have diabetes in the problem list). Under-coding outliers at the provider and clinic levels will be identified by comparing rule enforcement.
ResultsResults from this work in progress will be presented at the conference. An estimated 270 clinics, 1340 providers, and 1.8 million patients will be included from the CPCSSN database. Rule 'confidence' will be used to identify outliers; the confidence of a rule X→Y is the proportion of individuals with X who also have Y (Pr(Y|X)). For example, we may find that on average 80% of patients prescribed a diabetic medication will also have a diagnosis of diabetes in the problem list (average confidence of 80%), but an outlier clinic may have a confidence of 40%; this low rule confidence may indicate under-coding of diabetes in the problem list. Confounding by patient demographics (e.g., age, sex, urban/rural) will be assessed and adjusted for, if necessary.
Conclusion/ImplicationsThis work examines a novel method to identify potential under-coding in the EMR problem list. Providers/clinics could use this information to update patients' problem list or inform quality improvement interventions. Researchers using primary care EMR data need to be aware of potential under-coding and take steps to mitigate the effects.

BackgroundIn Canada, legislative restrictions exist for the disclosure and transfer of patient data across provincial boundaries. Often, meta-analysis techniques are employed to pool province-specific results to enable national results while maintaining compliance with privacy legislation. However, the effectiveness of this technique remains unexamined using electronic data. To evaluate its performance, we compare results obtained through meta-analysis with those from a fully pooled model.
MethodsUsing chronic kidney disease as a case study, this retrospective cohort study simulates the meta-analysis method to evaluate its performance using real data. We analyze data from the Canadian Primary Care Sentinel Surveillance Network (CPCSSN), a nationwide database of electronic medical records. We meta-analyze provincial logistic regression models to predict nephropathy over a 5-year period at a national level. Estimates are compared to a fully pooled national model.
ResultsAnalyses are currently ongoing. We expect the meta-analysis technique to produce similar estimates compared to the fully pooled model. However, we expect certain differences as meta-analysis techniques are insensitive to heterogeneity within provinces that impact the precision of estimates. Observed differences will help to inform future work where we will examine a potential new methodology to analyze health information without sharing patient data.
ConclusionsBased on our results, we will reveal strengths and limitations of the meta-analysis technique for interprovincial analyses. Limitations of the meta-analysis technique may underscore the need for further exploration into alternative methodologies that can effectively analyze health information without compromising patient privacy to facilitate better healthcare decision-making and policy development in Canada.

ObjectivesIn Alberta, Canada, we quantified the rate of potentially inappropriate oral antibiotic prescribing in emergency departments for viral infections or conditions not likely requiring antibiotics from 2010-2020 and assessed the impact of two Choosing Wisely Canada (CWC) campaigns (2015/2016 and 2018) discouraging inappropriate antibiotic prescribing in emergency medicine.
ApproachWe linked emergency department adult and pediatric records from the National Ambulatory Care Reporting System and medication dispensations from community-based pharmacies in the Pharmaceutical Information Network. From January 2010 to February 2020, we identified emergency department visits for 5 conditions that were potentially inappropriately treated using antibiotics per CWC recommendations (bronchitis, asthma, bronchiolitis, pharyngitis, and acute otitis media). We used an interrupted time series design to detect changes in antibiotic prescribing by fitting Autoregressive Integrated Moving Average (ARIMA) models to account for secular trends and seasonality, allowing for change in slopes to measure the effect of each CWC intervention.
ResultsAntibiotics were commonly prescribed in emergency departments for bronchitis (proportion of visits with antibiotics: 57%) and asthma (22%) in adults; bronchiolitis in children (43%); pharyngitis (39%) and acute otitis media (54%) in adults and children. Based on visual inspection, the proportion of emergency department visits for each condition where antibiotics were dispensed remained relatively consistent. The ARIMA models demonstrated mixed impacts on potentially inappropriate antibiotic prescribing associated with two interruptions: the 2015/2016 CWC recommendations and subsequent 2018 CWC Using Antibiotics Wisely campaign. Following each interruption, antibiotic prescribing was slightly reduced for bronchitis (-1.0%/year,p=0.03; -4.4%/year,p=0.004, respectively) and bronchiolitis (not significant) (-0.7%/year,p=0.57; -2.5%/year,p=0.34), but unchanged for asthma (-0.6%/year,p=0.30; 0.7%/year,p=0.74) and pharyngitis (0.0%/year,p=0.95; -0.2%/year,p=0.93), and slightly increased for acute otitis media (not significant) (1.4%/year,p=0.07; 5.9%/year,p=0.052).
ConclusionRates of potentially inappropriate antibiotic prescribing remained constant over the past 10 years in Alberta. Campaigns to rethink antibiotic use in emergency medicine may have resulted in small decreases in antibiotic use; however, further initiatives building upon existing campaigns are required to substantially reduce rates of inappropriate antibiotic prescribing.

IntroductionThe Discharge Abstract Database (DAD) associates ICD-10-CA diagnosis codes with inpatient care episodes at acute-care facilities. Codes are assigned by human coders, based on chart review. Coding guidelines stipulate mandatory coding of major and fatal conditions but only optional coding of secondary conditions, which results in undercoding for many conditions.
Objectives and ApproachThis research evaluates machine learning approaches for identifying and completing records with missing codes, to improve data quality. The Alberta Hospital DAD for 2013-14 was used in this study. We assumed that the existing ICD-10-CA codes in the DAD are correct, and used them as training examples. Several ML classifiers, including logistic regression and random forest, were used to develop models to assess the coding probability, using existing codes and demographic information. 3300 chart-review records were used as the reference standard. We focused on hypertension-related codes. Validity of raw diagnosis codes in the DAD was used as the baseline.
ResultsA record is deemed to have a missing hypertension diagnosis code if the predicted probability is high, but without the diagnosis codes having been assigned by the coders. In the baseline, the original hypertension codes have high PPV (ranging from 0.902 for the age group 35-54 to 1.000 for the age group 18-34) but low sensitivity (ranging from 0.200 for the age group 18-34 to 0.565 for the age group 75+). The most successful models that we have tested so far have provided improvements of 2-6% in the sensitivity, while maintaining the PPV. More improvement is generally seen for the younger age groups. Initial experiments indicate greater improvements in sensitivity may be possible for other conditions, such as peptic ulcer disease and cerebrovascular disease.
Conclusion/ImplicationsMachine learning approaches can be useful and cost-effective for improving data quality in DAD. While the improvements in sensitivity relative to the baseline are modest at present, further experiments with different models and feature sets are warranted. Experiments with other conditions may also be fruitful.

IntroductionFrailty is a combination of factors that increase vulnerability to functional decline, dependence and/or death. Frailty cannot easily be defined by comorbidities or medical treatment alone. Accurate detection of frailty in practice and at a population level is needed. This may be achieved using a combination of data sources.
Objectives and ApproachWe construct algorithms that can identify frailty using electronic medical record (EMR) and administrative data. We linked EMR data from the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) nodes and the administrative (e.g. billings, hospitalizations) from Population Data BC and the Manitoba Health Policy Centre. Frailty was defined as individuals 65+ who were receiving home services, had specific chronic conditions, received specific diagnoses, and/or had specific lab or other clinical indicators. We describe sociodemographic characteristics, risk factors, prescribed medications, use and costs of healthcare for those identified as frail.
ResultsPeople were identified as frail in 2014 and all analysis was completed with 2015 data. Among those who were > 65 years, who had a record in both EMR and administrative data, 5\%-8\% (n=191 of 3,553, BC; n=2,396 of 29,382, MB) were identified as frail. There was a higher likelihood of being frail with increasing age and being a woman. In BC, those identified as frail have higher contacts with primary care (n=20 vs. n=10) and more days in hospital (n=7.4 vs. n=2.0) compared to those who are not frail. Twenty two percent of those identified as frail in 2014 died in 2015, compared to a mortality rate of 2\% among those who are not frail.
Conclusion/ImplicationsIdentifying and reporting on those who are frail in primary care as well as in communities could enable targeted communications with patients and families and community based resources in order to improve patient care, patients' and caregivers' quality of life and better use of the healthcare system.

IntroductionRisk prediction models can be used to inform decision-making in clinical settings. With large and detailed electronic medical record data, machine learning may improve predictions. The objective of this work is to determine the feasibility and accuracy of machine learning versus logistic regression to predict unplanned hospital admissions.
Objectives and ApproachData from primary care electronic medical records for community-dwelling adults in Alberta, Canada available from the Canadian Primary Care Sentinel Surveillance Network will be linked to acute care administrative health data held by Alberta Health Services. Two regression methods (forward stepwise logistic, LASSO logistic) will be compared with three machine learning methods (classification tree, random forest, gradient boosted trees). Prior primary and acute care use will be used to predict three outcomes: ≥1 unplanned admission within 1 year, ≥1 unplanned admission within 90 days, and ≥1 unplanned admission within 1 year due to an ambulatory care sensitive condition.
ResultsThe results of this work in progress will be presented at the conference. 41,142 patients will have their primary and acute care data linked. We anticipate that the machine learning methods will improve predictive performance but will be more challenging for clinicians and patients to understand, including why a given patient is predicted to be at higher risk. The primary comparison of machine learning and regression methods will be based on positive predictive values corresponding to the top 5% predicted risk threshold, and estimated via 10-fold cross-validation.
Conclusion/ImplicationsThis project aims to help researchers decide which statistical methods to use for risk prediction models. When considering machine learning methods the best approach may be to try multiple methods, compare their predictive accuracy and interpretability, and then choose a final method.

IntroductionWe previously investigated the frequency of screening for celiac disease (CD) based on tissue transglutaminase antibody testing (tTG-IgA) and developed the first incident cohort of celiac autoimmunity in Canada.
MethodsAdministrative data sources used in this study were population-based and covered the entire province of Alberta (~4.3M residents during study period). Various approaches to querying data were employed within a diverse team of health information managers, data analysts, clinical scientists, and gastroenterologists. This process involved a broad search for CD screening tests, thorough data inspection/cleaning, and numerous discussions to determine potential explanations for the findings.
ResultsApproximately ~950,000 records for tTG-IgA were first identified. Records were then excluded due to missing/invalid patient identifiers or test results (0.8%), non-Alberta residency (0.4%), and duplicate records (1.1%). A final dataset included ~920,000 tTG-IgA tests on ~680,000 unique patients, which was also validated through a separate query performed by an analyst external to the study team. A conservative approach to excluding as many potential prevalent cases of CD was applied given a robust algorithm for CD has not yet been established in Canada. The final rate of celiac autoimmunity (34 per 100,000) offered further face validity based on prior estimates of diagnosed CD in Alberta and other countries reporting on celiac autoimmunity.
ConclusionWhen developing a novel case definition or investigating unfamiliar outcomes using routinely collected data, collaboration across several disciplines is highly recommended. Certain stages of the project may require additional scrutiny and discussion to ensure findings are valid and reliable.

Precision Medicine and Precision Public Health are approaches to improve population health. Achieving these goals requires innovation in health informatics. The Centre for Health Informatics (CHI) within the Cumming School of Medicine (CSM) at the University of Calgary (UC), Canada, was created to respond to this need by fostering multidisciplinary collaborations, building capacity by recruiting and training outstanding faculty and students, and harnessing Alberta's rich health data to advance health informatics. To establish CHI as a health informatics leader, CHI has struck partnerships with stakeholders, including Alberta Health Services (AHS), Alberta Health (AH), and the Alberta Strategy for Patient-Oriented Research Unit (AbSPORU) among others. Through these close relationships, the CHI intake team facilitates access to Alberta's rich health data sources and educates researchers on the available health data in Alberta. The concept of a "One Stop Shop" for CSM and UC researchers encourages multidisciplinary collaboration, helps investigators access a wide range of datasets, and receive analytical support. Population-based data sets enable the development of methods to turn raw data into health information, improve health data collection, linkage, analysis, and quality, and applied studies creating clinical decision-support tools, prognostic tools, improved health surveillance methods, and health system performance indicators. CHI's ecosystem of diverse research expertise, cutting-edge technology, and embedded AHS analysts to support data access via a wide-ranging network of partnerships allows our provincial researchers, national and international collaborators tremendous opportunities for empirical research. It paves the way for implementing Precision Medicine in the real world.

IntroductionPooling data from cohort studies can be used to increase sample size. However, individual datasets may contain variables that measure the same construct differently, posing challenges in the usefulness of combined datasets. Variable harmonization (an effort that provides comparable view of data from different studies) may address this issue.
Objectives and ApproachThis study harmonized existing datasets from two prospective pregnancy cohort studies in Alberta Canada (All Our Families (n=3,351) and Alberta Pregnancy Outcome and Nutrition (n=2,187)). Given the comparability of the characteristics of the two cohorts and similarities of the core data elements of interest, data harmonization was justifiable. Data harmonization was performed considering multiple factors, such as complete or partial variable matching regarding question asked/responded, the response coded (value level, value definition, data type), the frequency of measurement, the pregnancy time-period of measurement, and missing values. Multiple imputation was used to address missing data resulting from the data harmonization process.
ResultsSeveral variables such as ethnicity, income, parity, gestational age, anxiety, and depression were harmonized using different procedures. If the question asked/answered and the response recorded was the same in both datasets, no variable manipulation was done. If the response recorded was different, the response was re-categorized/re-organized to optimize comparability of data from both datasets. Missing values were created for each resulting unmatched variables and were replaced using multiple imputation if the same construct was measured in both datasets but using different ways/scales. A scale that was used in both datasets was identified as a reference standard. If the variables were measured in multiple times and/or different time-periods, variables were synchronized using pregnancy trimesters data. Finally, harmonized datasets were then combined/pooled into a single dataset (n=5,588).
Conclusion/ImplicationsVariable harmonization is an important aspect of conducting research using multiple datasets. It provides an opportunity to increase study power through maximizing sample size, permitting more sophisticated statistical analyses, and to answer novel research questions that could not be addressed using a single study.

IntroductionFrailty is a geriatric syndrome that is predictive of heightened vulnerability for disability, hospitalization, and mortality. Annually an estimated 250,000 frail Canadians die, and this estimate is expected to double in the next 40 years, as Canadians grow older. Currently there is no single accepted clinical definition of frailty.
Objectives and ApproachThe objective of this study was to develop an operational definition of frailty using machine learning that can be applied to a primary care electronic medical record (EMR) database.
The Canadian Primary Care Sentinel Surveillance Network (CPCSSN) is a pan-Canadian network of primary care practices that collect de-identified patient information (such as encounter diagnoses, health conditions, and laboratory data) from EMRs.
780 patients from CPCSSN have were randomly selected and assessed by physicians using the Rockwood Clinical Frailty Scale (as frail or not frail), and their clinical characteristics from CPCSSN used to develop the definition using machine-learning.
ResultsA total of 8,044 clinical features were extracted from these tables: billing, problem list, encounter diagnosis, labs, medications and referrals. A chi-squared automatic interaction detector (CHAID) approach was selected as the best approach. The bootstrapping process used a cost matrix that prioritized high sensitivity and positive predictive value. 10-fold cross validation was used for validity measures. Key features factored into the algorithm included: diagnosis of dementia (ICD-9 code 290), medications furosemide and vitamins, and use of key word "obstruction" within the billing table. The validation measures with 95% confidence intervals are as follows: sensitivity of 28% (95% CI: 21% to 36%), specificity of 94% (95% CI: 93% to 96%), positive predictive value of 53% (95% CI: 42% to 64%), negative predictive value of 86% (95% CI: 83% to 88%).
Conclusion/ImplicationsNo other primary care specific frailty screening tools have sufficient validity. These results suggest heterogeneous diseases require clearly defined features and potentially more sophisticated algorithms to account for heterogeneity. Further research utilizing continuous features and continuous frailty scores may be more suitable in the creation of a case detection algorithm.

Schizophrenia does not present uniformly among patients and as a result this patient population is characterized by a diversity in the type and amount of healthcare supports needed for daily functioning. Despite this, little work has been completed to understand the heterogeneity that exists among these patients. In this work we used a data-driven approach to identify subgroups of high-cost patients with schizophrenia to identify potentially actionable interventions for the improvement of outcomes and to inform conversations on how to most efficiently allocate resources in an already strained system. Administrative health data was used to conduct a retrospective analysis of "high-cost" adult patients with schizophrenia residing in Alberta, Canada in 2017. Costs were derived from inpatient encounters, outpatient primary care and specialist encounters, emergency department encounters, and drug costs. Latent class analysis was used to group patients based on their unique clinical profiles. Latent class analysis of 1659 patients revealed the following patient groups: (1) young, high-needs males early in their disease course; (2) actively managed middle-aged patients; (3) elderly patients with multiple chronic conditions and polypharmacy; (4) unstably housed males with low treatment rates; (5) unstably housed females with high acute care use and low treatment rates. This taxonomy may be used to inform policy, including the identification of interventions most likely to improve care and reduce health spending for each subgroup.