Presentation to Ryan White Grantee Meeting - dataCHATT
Contents
1. The Importance of Data Quality for Grantees Program Management e Internal monitoring and evaluation e Planning e Quality improvement e Grant writing DEVE MO IUE Ni Wm Ore arers aals But e What if it s not timely e What if It s not valid e What if its not complete e Why is good data so important to grantees www datachatt jsi com et atacHATT SO where do you start e o ensure quality data you need to follow a series of steps in the collection reporting and use of your data e hese form a flow from identifying what you need to collect through where you will get it to how you will collect and report your data www datachatt jsi com gt atacHATT Essential Steps of Data Flow from Collection to Reporting and Use Identifying and Defining 5 Data Reporting Data Elements How do you submit the data you What do you want need to have collect c 2S Data 2 Data Sources 6 ommunicating about Data Where can you find what you How do you use the aata you need to collect have to inform our program about 3 Data Collection how you are doing How can you get the data you 7 Using the Data need to collect How do you use the data you 4 Data Validation and Data have to inform our program Quality Procedures decisions arempi you Assessing the Effectiveness of reflects what you are trying to the Current System measure or report How can you improve our d2. www datachatt jsi s dataCHATT e Work back from the submission deadline include time to review address identified gaps etc e More frequent collection allows for more time to review data collected Care and services being provided Missing data Other data problems e Grantees with subcontractors can request data submissions more frequently than reporting requires more than annually www datachatt 7 com 82 datacHATtT e Define and set reasonable timelines e Communicate and stick with timelines e Include a process for reviewing whether data was submitted on time providing feedback and requesting revisions e Consider implementing consequences for lateness and rewards for timeliness www datachatt jsi 3 datacHATT Data are protected from deliberate bias or manipulation for any reasons w datachatt jsi daEACHATT integrity Key Questions e Are there risks that data might be manipulated for any reasons e What systems are in place to minimize such risks www datachatt jsi com HH atacHAtt Confidentiality Clients are assured that their data will be maintained according to organization State and national standards www datachatt jsi com HH atacHATt Confidentiality Key Questions Do you provide routine training e to program staff on the importance of confidentiality and on confidentiality requ
3. Do you provide clear and consistent training across all sites s the instrument always administered by trained staff e Guidance lnstructions Do you provide detailed procedures and instructions to all sites and providers Are all providers trained to ask clients to self identify their ethnicity race and gender Is it possible that some providers make assumptions based on appearance e Consistent tool across all sites and providers e Refer to user manual www datachatt jsi com ms daCACHATT Completeness Complete data do not have any missing elements and are collected on the entire population outlined in the user manual or guidance www datachatt jsi com HH atacHATt Completeness Key Questions e Percent of all fields on data collection form filled in e Percent of all expected reports actually received e Are the data from all sites that are to report included in aggregate data If not which sites are missing e Is there a pattern to the sites that were not included in the aggregation of data www datachatt jsi com ss ACACHATT e Develop a procedure to routinely look for frequency of missing data elements Check for completeness and communicate edits on a routine basis e g monthly e Develop and implement procedures follow up on missing data Volume of missing data often diminishes over time once staff are aware that someone is looking a
4. Ask the hospital I department to reprogram so that external Paps can go into the same field OR the RDR report can look at both fields e STUDY nothing happens as the hospital EMR is a proprietary software and takes significant resources to revise and will take many months www datachatt jsi com pm IET al Baan kal j engan Oo F E UE g E fi o etl el i Act and the Next Cycle ACT Decide to try a different approach for an interim solution PLAN Establish a Log for women getting Paps from external providers and use to manually enter into program database e DO train a nurse and data manager to use an Excel spreadsheet to enter any woman getting a Pap from provider external to the clinic and educate all providers to give the Pap results to the nurse before sending to medical records e STUDY Next RDR rate is only 18 below HIVQUAL data e ACT Continue log and also work with PO to get resources to ultimately automate capture of externally provided Pap tests datachatt jsi com Provide an Overview of HAB Funded Sources of Available TA to Support Data Quality TA Resources to Support Data Qualit Project officer e TARGET Center http www careacttarget or e dataCHATT http www datachatt jsi com e Ryan White HIV AIDS Program Data Report TA http datasupport hab hrsa gov e CAREWare TA http nab hrsa gov careware e National Alliance of State
5. and Territorial AIDS Directors Cooperative Agreement NASTAD htto Wwww nastad org Programs hivcareandtreatment e National Quality Center NQC htto www nationalqualitycenter org e HRSA Information Center http ask hrsa gov www datachatt jsi com 8 datacHATT Get Participant Feedback Delt Water elsvanhy e dataCHATT is developing a series of web based training modules e This Data Academy will include training modules on data collection data quality data reporting and using data e We need your feedback to make sure the information Is presented effectively www datachatt jsi com m atacHATT Feedback Was this content useful Appropriate Did it meet your needs Any suggestions Can we contact you to review future Data Academy modules w datachatt jsi daEACHATT Acknowledgements The HIV AIDS Bureau for support of this Cooperative Agreement e The 101 Grantees who participated in the Request for Information JSI contributing staff Julie Hook Kim Watson Michael Rodriguez and the dataCHA TT team Positive Outcomes Inc www datachatt jsi com it datacHATT For more information Visit the dataCHATT website www datachait si com For copies of today s presentation contact TEIR EIC OEO w datachatt jsi daEaCHATT
6. and make sure everyone has access to It Danni www datachatt jsi com a a pa snama lal aoe ejaj f a TTT w aE Doa ee ie ie b ul el es O ren Ensuring Consistency Standardize forms tools across data collection and reporting efforts e Develop a written protocol you own user guide to document which explains your procedures for data collection quality and reporting Includes clear and consistent definitions of the key elements for data collection Provides the details for each variable data source how you will collect it Defines who will be responsible for what Is clear and easy to understand e Develop data review and data cleaning procedures to be performed at all levels Update tools and protocols regularly www datachatt jsi com Hit datacHATT Key Elements of Data Quality Elements of Data Quality e Validity e Reliability e Completeness e Timeliness e Integrity e Confidentiality Valiaity Valid data are accurate data defined as They measure what they are intended to measure a www datachatt jsi com i D Baad nunen Fr CHATT i i p pa ay a sae O Validity Questions Data Collection e Does the setting and how the questions are being asked potentially compromise their validity For example asking an adolescent about sexual activity in front of their parent www
7. datachatt jsi com dataCHATT Validity Questions Data Collection e How is the primary data collection and entry being done Is there potential for error For example Client fills out a paper form and misunderstands a question Administrative staff enters form into EMR and makes an entry error based on client handwriting Databases are not linked so data must be extracted and then entered hand into HIV program s database opening opportunity for mistakes www datachatt jsi com ss dat aCHATT Validity Questions Data Reporting If you are combining data or calculating rates Are the correct formula and approaches being applied Are they applied consistently e g from site to site over time e Are final numbers reported accurately e g does the total add up www datachatt jsi com inngi BB a 2 dacaCHATT seem ale Validity Steps to Limit Errors e Training Are all staff trained on definitions and how to complete data entry fields Validation Checks Do the data fall within acceptable range Look for outliers e e g age gt 100 e CD4 count gt 4 000 e pregnant men www datachatt jsi com HATT More Steps to Limit Errors e Validation Rules Do you have data validation rules e g can not enter pregnancy if client is male Validation Activities You can do chart extraction to validat
8. ata system in order to effectively accomplish steps 1 7 www datachatt T com 22 datacHAattT Focus on Data Validation and DEVE MO IU lia ea celerslel0 lasts Efforts to measure and improve data need to happen during all of these steps This presentation focuses on Step 4 Data Validation and Data Quality Procedures Key Factors for Ensuring systemic Data Quality Review and use your data Know your data Ihe best way to improve data quality is to review and use the data e Create a system for data quality assessment that is routine comprehensive and reflective Define and follow your data flow steps to collect and report the data www datachatt jsi com it datacHATT involve your stafi Engage your staff and your contracted providers in the efforts to ensure data quality e Define roles and responsibilities at all levels Consider identifying one or more individuals to oversee data quality procedures reviewing definitions protocol development training etc e Conduct routine training to review data related procedures and learn about any changes www datachatt jsi com i n dacacHATT Develop and communicate your requirements and expectations e Provide routine training to internal staff and contracted providers on reporting requirements timelines and expectations through policies procedures contracts or MOUs e Provide written guidance
9. dataCHATT 201 introduction to Data Flow and Data Quality Assessment Mira Levinson JSI Research amp Training Institute Inc Kim Lawton Quality and Information Management Lisa Hirschhorn JSI Research amp Training Institute Inc Data Quality e Having quality data is critical for many program activities Clinical care Quality improvement Planning Reporting But what do you mean by quality how do you measure it and why should you care www datachatt jsi daEACHATT Presentation Overview The Importance of Data Quality for Ryan White Program CICUGER e Essential Steps of Data Flow from Collection to Reporting and Use e Key Factors for Ensuring Systemic Data Quality e Key Elements of Data Quality e Quality Improvement Techniques to Improve Data Quality from Collection through Reporting a really quick tour e Provide an Overview of HAB Funded Sources of Available TA to Support Data Quality Get Participant Feedback se Www datachatt jsi com 7 BB l f i 1 4 The Importance of Data Quality for Ryan White Program Grantees The Importance of Data Quality for Grantees Data Reporting e Grantees need to accurately report HIV services provided and patients served to HRSA HAB e HRSA needs to accurately reports to Congress for ongoing support of the Ryan White Program www datachatt jsi com an nE an
10. e data entereq Double entry usually reserved for research or when data quality is a significant concern or new staff www datachatt jsi com it datacHATT Example Validation Checks In this example Specimen Source cervix endocervix is checked against Gender Male Specimen Source is Cervix which should only be selected tor Female patients qry Cleanin cleaning i specimen Source Matching Sex Select Ls p paced aeo Tournmno IENTNO FACILITY_ID_ SPECIMEI Specimen_Source Text EE E Mz emba Endocerix 0Gst 9909 3371 T eA CenixfEndocenix OBst S9 3371 Male ervixfEndocenix O8st S36 33r 1 7 Male lt Cermix Endocerix Osst 7395 Era Male Cenix fEndocervix www datachatt jsi com 82 datacHATT Reliability Reliable data are measured and collected consistently i e reoeated measurements using the same procedures get the same results Reliability Key Questions e Where are there potential gaps in the data flow which may compromise reliability The same instrument is not used year to year or across sites e Data collected changes without true change in services e One site uses a nurse to extract from a medical record while another uses an non clinically trained data manager www datachatt jsi com i n aLacHATT Procedures to Ensure Reliability Are steps being taken to limit reliability errors e raining
11. irements and procedures to IT staff on the specific issues of HIV confidentiality and electronic information storage and transfer to contracted service providers on procedures for data submission to clients on confidentiality procedures se Www datachatt jsi com 7 BB l f i 1 4 Procedures to Ensure Integrity and Confidentiality e raining rain all staff and contracted providers on confidentiality and privacy protocols e Electronic Data Security Document user access to database Limit user access to database Consider security limitations of laptops handheld devices etc www datachatt jsi com Ennn K E aoe E E E E j E a _S Doa ee ee i b ul el es O ren Procedures to Ensure Integrity and Confidentiality e Security of Paper Data Store paperwork in a secure locked cabinet and or user restricted area e Inform Clients of Confidentiality and Privacy Protocols ew tate chatt jsi a CHATT Discussion How Does This Apply To Me e Validity accuracy e Timeliness there when you need it Reliability consistency e Integrity honesty e Completeness CURUG e Confidential Ensure Quality Through Assessment When to assess program data quality Integrate data quality control mechanisms into standard operating procedures and software Integrate data quality checks into routine s
12. se reported in the RDR are only 40 e Develop a QI Project Goal They want to improve the quality of reported data e Forma leam A team is formed including the program data manager a nurse provider and a case manager e They define the goal as decreasing the difference between reported and actual rates to less than 10 www datachatt 7 com 82 datacHAattT PLAN e Identify the data steps required for a Pap smear to be included in the RDR report Internally internal lab results are automatically entered into the EMR which is then used to download data into a program database for RDR submission Versus HIVQUAL chart review of client sample and entry into HIVQUAL database www datachatt 7 com 3 daCaCHATT PLAN e Pap results for patients seen by external providers are not received 25 of the time e When these results are received they are manually entered into a different field than the one used for Pap results for patients seen by internal providers done via automatic transfer from lab system e For HIVQUAL reviews both fields are manually extracted but the automated RDR report only extracts the data field of the program database of the internally provided Pap tests www datachatt jsi com 2 dat cHATT More Plan Do and Study e PLAN Modify IT systems can be modified so that data sources are the same OR reporting draws from both Pap data sources e DO
13. t it Procedures may be different for data received from contractors versus internally collected data electronic data submission vs paper data submission www datachatt jsi com 2 datacHATtT Look for missing data trends Look for trends in missing data and ask why Are there barriers to capturing or entering the data Meet with your staff and ask for their Insights Use this information for data collection planning www datachatt jsi com HATT Timeliness Timely data are e sufficiently current and frequent to inform management decision making e received by the established deadline e received with adequate time to review for other elements of quality and to address identified gaps www datachatt jsi daEaCHATT Timeliness Key Questions e Is a regular schedule of data collection in place to meet program management needs When are your established deadlines e Does program staff and contractors know and understand the reporting deadline lIs it consistent across all reporting sites www datachatt jsi com Ht atacHATT e Is there adequate time to review data for other aspects of quality and address identified gaps before it is needed for reporting or other use e Are data available on a frequent enough pasis to inform program management decisions e Are data being collected and reported according to your timeline
14. upervisory or contract monitoring visits Conduct periodic formal assessments Provide feedback on submitted data www datachatt jsi com Beni Shae o sss GaACaACHATT TT ee ise Ly Ge Quality Improvement Techniques to Improve DEV me ler liinmicelar Collection through Reporting a really quick tour e he same concepts apply to Improving data quality as they do to improving quality of care Measure the quality Explore steps required for quality data and where gaps may have occurred flow chart Understand the potential causes of the identified gap fishbone or cause and effect www datachatt jsi dat acHaTT Plan Develop a QI Project goal i e what you want to accomplish based on assessment of data quality Decrease missing data improve timeliness Formateam Identify where you think the problem gap may be and develop a potential solution Do Carry out the proposed solution Study Analyze your data summarize what was learned compare with what you wanted to achieve did the solution work Act Determine next steps if worked how to expand if not as successful what to change and then begin Plan to implement Graphic biast from the American Heart Association www datachatt jsi com 22 datacHATT PLAN e identify the problem A hospital based site notices that their Pap rates for HIVQUAL are 75 but tho
Download Pdf Manuals
Related Search