To use research diagnoses to test alternative algorithms for deriving diagnosis from routinely collected data in people with psychosis, research diagnoses were obtained for 319 consenting individuals from NSW sites of the Survey of High Impact Psychosis (SHIP). SHIP records were linked to data from the NSW Health Information Exchange. Diagnoses were derived using four algorithms: (i) any (ii) most recent (iii) modal (“dominant”) and (iv) hierarchical diagnoses (schizophrenia > affective psychosis > brief psychosis etc.). Agreement between derived and research diagnosis was compared for (i) any psychosis diagnosis, (ii) schizophrenia and (iii) affective psychosis, using percentage agreement, Cohen’s Kappa, and prevalence and bias adjusted kappa (PABAK). The influence of setting (inpatient or community) was examined.
Agreement between derived and research diagnoses was poor using unadjusted kappa, but moderate to good after adjusting for prevalence and bias. Modal diagnosis and most recent diagnosis provided the greatest agreement with research diagnoses. Agreement was best for the overall distinction between psychosis and non-psychosis, intermediate for schizophrenia and least for affective psychosis.
Agreement between research and derived diagnoses was similar for inpatient and community settings within NSW administrative data. Modal diagnosis across inpatient and community settings is the preferred approach, producing diagnoses that have reasonable agreement with research diagnoses. Diagnoses derived from routine data are – with appropriate cautions – suitable for use in population research on psychosis.
