New York Statewide Planning and Research Cooperative System (SPARCS) Data
Initially created to collect information on hospital discharges, SPARCS collects patient-level data on patient characteristics, diagnoses and treatments, services, and charges for all hospital inpatient stay and outpatient visits in the state of New York. SPARCS also includes data on all ambulatory surgery visits, emergency department admissions, and extended outpatient treatments from NY licensed centers.
All inpatient and outpatient services provided at licensed hospitals, licensed hospital extension clinics, and diagnostic and treatment centers in New York State.
Access
SPARCS offers three levels of data access: public, limited, and identifiable. Public, de-identified data may be requested without application through Health Data NY. Limited data contains indirect identifiers that are considered identifiable according to the standards outlined by the Health Insurance Portability and Accountability Act (HIPAA). Identifiable data directly pertains to an individual’s facility stay and could expose the patient.
Limited and identifiable data may be obtained by submitting a Data Request Form. Applications for limited data require review and approval from the SPARCS operations staff while identifiable data must also be reviewed and approved by the Data Governance Committee (DGC).
As part of the application process, researchers must provide data security plans. Studies that include confidential patient data, including epidemiology and research studies as defined by SPARCS, must obtain IRB approval. All persons and organizations that will have access to or will help store data for the duration of the research project must enter into a Data Use Agreement (DUA) or organizational DUA. Requests for identifiable data must provide specific justification for each data element requested. Data requests may include up to three years beyond the current year. For a flowchart of the data request process, see page 22 of the SPARCS Operations Guide.
Timeline for Access
Review of data requests occurs every six weeks at DGC meetings. Applications are due three weeks prior to such meetings, and requests are approved or denied within the ten days following the meeting. Approved requests must then be ratified by the NY Commissioner of Health. If ratified, researchers are given an invoice and data are typically delivered within thirty days following payment.
All data requests expire two years after receipt of the final year of data. Requests for extensions should be submitted to [email protected] and may be granted for up to one year at a time. Upon completion of the project, any limited or identifiable SPARCS data must be destroyed, and a Data Destruction Certification letter must be submitted.
Lag Time
Incomplete SPARCS data are available on a one-month lag. Complete data for the previous year are made available typically around mid-August of the year requested. Thus, incomplete February 2018 data are available in March 2018 and complete February 2018 data are available in August 2019.
Because data are reconciled gradually, incomplete data that is delivered later in the subsequent year will be more complete than if supplied early on. Thus incomplete February 2018 data that is delivered in June will be more complete than if delivered in March.
Cost
Cost of accessing SPARCS data depends on the type of data requestor and/or affiliated organization. While there are a number of discounts that researchers may be able to apply towards their initial request, data updates can only be supplied at full cost. For a more detailed cost structure, email [email protected].
Linking
Researchers must perform the link between SPARCS data and external data sources. Data can be delivered based on a description of the desired cohort that can include age, gender, patient residence (by county), hospital county, and data type. In order to receive SPARCS data, researchers must submit approval from every additional data provider allowing the link to SPARCS data. Researchers must indicate on the Data Request Form whether they will require linking SPARCS data to another data source.
Identifiers Available for Linking
- Address
- Date of birth
- Permanent Facility Identifier (PFI)
- Patient Control Number
- Medical Record Number
Linking to Outside Data Sources
SPARCS data can be linked to New York City or New York State State Vital Statistics data. The application for linking to the relevant city office can be found here, and researchers may obtain an application for State Vital Statistics by contacting [email protected].
Data Contents
Until October 2017, SPARCS data were categorized into inpatient and outpatient data and updated on a monthly basis.
Inpatient data included all inpatient discharges from licensed hospitals in New York from 1982 to October 1, 2017. Please find the data dictionary for inpatient data here. Outpatient data included all ambulatory surgery from 1983 to October 1, 2017, outpatient services data from 2011 to the present, and all Emergency department data, which was voluntarily collected beginning in 2003 and became mandatory in 2005. Please find a data dictionary for all outpatient data here.
For data requests after October 1, 2017, the data format consists of 14 relational tables linked by a Claim Transaction ID variable. See the Data Dictionary for more information on how these tables are organized.
All direct and indirect identifiers on HIV/AIDS and abortions records are removed from the data set before being delivered. Some of this redacted information, including some geographic identifiers such as ZIP code, can be given to researchers if they are able to provide justification for the data elements, a letter of support from the Commissioner of Health, and signed consent forms of the patients.
Partial List of Variables
Admission/discharge date, type of admission, source of admission, alternate care days, neonate birth weight, Mother’s medical record number, expected principal reimbursement, patient residential address, source of payment, policy number, payment identification number, covered/non-covered days, type of bill code, accident related code, worker’s compensation amount/No fault amount, blood furnished amount, accommodations information, provider information, ancillary services information, diagnosis code, other diagnoses information, external cause of injury information, method of anesthesia used, total charges, length of stay, non-acute care information.
J-PAL Randomized Evaluations Using this Data Set
Unknown.
Other Research Using this Data Set
Doyle, Joseph J., John Graves, Jonathan Gruber, and Samuel Kleiner. 2015. “Measuring Returns to Hospital Care: Evidence from Ambulance Referral Patterns.” Journal of Political Economy 123(1): 170-214. Doi: 10.1086/677756.
Federman, Alex, Rachel O’Conor, Michael Wolf, and Juan Wisniveski. 2019. “Impact of a Self-management Support Intervention for Older Adults with Asthma on Use of Controller Medications and Emergency Department Visits.” Journal of Allergy and Clinical Immunology 143(2): AB210. Doi: 10.1016/j.jaci.2018.12.639.
Pollack, Craig E., Shawn Du, Amanda L. Blackford, and Bradley Herring. 2019. “Experiment to Decrease Neighborhood Poverty had Limited Effects on Emergency Department Use.” Health Affairs 38(1): 1442-1450. doi: 10.1377/hlthaff.2019.00452.
Wheelock, Sophie, Mark Zezza, and Jessica Athens. 2020. “Complications of Childbirth: Racial & Ethnic Disparities in Severe Maternal Morbidity in New York State.” NYS Health Foundation. Accessed May 2, 2022. (https://nyshealthfoundation.org/wp-content/uploads/2020/08/severe-maternal-morbidity.pdf)