Suchergebnisse

ABSTRACT
ObjectivesValidation studies compare diagnostic information in linked administrative and reference (i.e., gold standard) data; they are an essential tool to develop accurate case definitions, the rules used to identify individuals in administrative data with a specific health condition. Validation studies often estimate the accuracy of multiple case definitions, in order to identify the data features (e.g., diagnosis codes, type of data source) that influence accuracy estimates. Descriptive analyses are commonly used to select a case definition(s) with the greatest accuracy estimates, but fail to account for uncertainty in accuracy estimates. The objectives were to: (1) compare the performance of regression-based approaches to test for differences in diagnostic accuracy estimates, and (2) demonstrate how to apply and use these models.
ApproachComputer simulation was used to compare three regression models: (a) univariate fixed-effects models applied to estimates of sensitivity and specificity; (b) univariate fixed-effects model for Youden's index, the average of sensitivity and the complement of specificity; and (c) bivariate random-effects joint model of sensitivity and specificity. The simulations varied the means and variances of sensitivity and specificity, the correlation between these parameters, and the number of case definitions. Performance was compared using: (a) bias (i.e., difference between estimated and observed mean), (b) mean squared error (MSE), the sum of the estimated variance and bias squared, and (c) 95% confidence interval (CI) coverage, the proportion of times the population mean is contained in the 95% CI. For objective 2, we applied the models to estimates of diagnostic accuracy from a published rheumatoid arthritis (RA) validation study with 61 case definitions.
ResultsUnivariate models of sensitivity and specificity had lower bias than the bivariate model (e.g., univariate=1.8%, bivariate=2.2%). The bivariate model had a smaller MSE than the univariate models when sample size was large and there was a small correlation between sensitivity and specificity (e.g., univariate=3.4%, bivariate=2.6%). Across all scenarios, the univariate model for Youden's index showed small bias (average=2.4%) and MSE (average=2.1%). For objective 2, the univariate models of sensitivity, specificity, and Youden's index revealed multiple case definition features that were associated with estimates of RA diagnostic accuracy: 1+ diagnosis in hospital records, >1 diagnosis in physician claims, and 1+ diagnoses by a specialist physician.
ConclusionsWe recommend the bivariate model when a validation study contains a large number of case definitions. When the data contain a small number of case definitions, univariate models are recommended.

IntroductionThe validity of chronic disease case definitions for administrative health data may change over time due to changes in data quality. Trend control charts to identify out-of-control (OOC; i.e., unexpected) observations in a time series may indicate where disease estimates are influenced by changes in data quality.
ObjectiveApply and compare trend control charts methods for multiple sclerosis (MS) incidence and prevalence estimates using previously-validated case definitions for Manitoba, Canada.
MethodsEight case definitions were identified from published literature and applied to Manitoba administrative health data from January 1, 1972 to December 31, 2018. Incidence and prevalence trends were modeled using negative binomial and generalized estimating equation models, respectively. Trend control charts were used to plot predicted case counts against observed case counts. Control limits to identify OOC observations were calculated using two methods: predicted case count ±0.8*standard deviation (0.8*SD) and predicted case count ±2*standard deviation (2*SD). Differences in proportion of OOC observations across case definitions was assessed using McNemar's test.
ResultsThe proportion of OOC observations ranged from 0.71 to 0.90 for incidence and 0.72 to 0.98 for prevalence when using the 0.8*SD control limits. A lower proportion of OOC observations (0.46 to 0.74 for incidence; 0.30 to 0.74 for prevalence) was observed for the 2*SD control limits. Neither method resulted in significant differences in OOC observations across case definitions.
ConclusionsThe proportion of OOC observations in trend control charts varied with the control limit method adopted, but statistical significance did not. Trend control charts are a potentially useful tool for developing surveillance methods, but may benefit from disease-specific calibrated control limits.

ObjectivesThe overarching research objective was to examine, culturally adapt, and further evaluate a mental health promotion approach called the PAX within 8 First Nations communities. This presentation describes a research process whereby First Nations community members and researchers worked in collaborative and culturally safe ways to reach their research objectives.
ApproachBuilding on a strong existing relationship between Swampy Cree Tribal Council (SCTC) members from Northern Canada and academic researchers, a team was formed to prepare the research proposal. This team included community members, leaders from First Nations organizations, decision makers, program developers and researchers. This research was guided by two-eyed seeing, a principle developed by a Mi'kmaw Elder, that recognizes both Indigenous and Western ways of knowing, where one worldview does not dominate the other. The research process was compliant with Ownership, Control, Access, and Possession (OCAP) principles that ensure self-determination of First Nations communities over research involving their people.
ResultsA First Nations community liaison was hired as a research team member ensuring that traditional and cultural protocols were adhered to and connections to community members facilitated and sustained. Over the course of the research, the team met monthly to oversee implementation and annually with SCTC community members for guidance and for sharing and interpreting results. All 8 communities were actively engaged and benefitted from their involvement. Seeing the value of examining PAX's effectiveness through linkages to administrative datasets, community members supported engagement of an additional 16 First Nations communities thereby ensuring an adequate sample size for the study. Health and education databases were linked to program data from 20 First Nations communities. Infographics, lay summaries and presentations were prepared for meaningful knowledge exchange.
ConclusionFirst Nations communities deemed it essential to understand what works and for whom regarding mental health promotion. Building relationships with First Nations community members based on trust and respect provided information that was relevant and beneficial to their communities. This relationship-building should be considered when developing research timelines and budgets.

ObjectivesPAX, a mental health promotion approach, has been shown to decrease negative mental health outcomes and improve academic achievement. These effects have yet to be shown among Indigenous children. We evaluated PAX for improving First Nations children's outcomes following a research process wherein community members and researchers work more collaboratively.
ApproachBuilding on a long-term relationship with Swampy Cree Tribal Council, community members, First Nations leaders and researchers worked together through all phases of the project. This cluster randomized controlled trial used population-based health, social services, and education administrative data that allowed de-identified individual-level linkages across all databases through a scrambled health number. Our cohort of 725 children from 20 First Nations schools were randomized to PAX (n=469, 11 schools) or wait-list control (n=256, 9 schools). We used propensity score weighting and multi-level modeling to estimate the differences over time (2011 up to 2020) between children exposed to PAX and those who were not.
ResultsDifferences in baseline characteristics were found between the two groups of children, despite the cluster randomization. After applying propensity score weights, children in the PAX group had significantly greater decreases in conduct problems (β:-1.08, standard error(se):0.2505, p<.0001), hyperactivity (β:-1.13, se:0.3617, p=.0018 ), and peer problems (β:-1.10, se:0.3043, p=.0003) and a greater increase in prosocial scores (β:2.68, se:0.4139, p<.0001) than control group children. The percentage of children in the PAX group who met academic expectations was higher than those in the control group, however, only grade 3 numeracy (odds ratio (OR):4.30, confidence interval (CI):1.34 – 13.77) and grade 8 reading and writing (OR:2.78, CI:1.01 – 7.67) met statistical significance. We found no evidence that PAX was associated with less emotional problems, diagnosed mental disorders or better student engagement.
ConclusionThese findings suggest that PAX was effective in improving First Nations children's mental health and academic outcomes in First Nations communities. Examining what works in Indigenous communities is crucial because approaches that are effective in some populations may not necessarily be culturally appropriate for remote Indigenous communities.