Improving Assessment, Service Coordination, and Evaluation of Older Adults Utilizing Tablet-based Care Planning Tools
Family Service Agency of San Francisco (FSA)
Family Service Agency of San Francisco developed and implemented two mobile screening and assessment tools for structured service planning for non-profit social service organizations to assist low-income, mentally ill older adults. The two screening and assessment tools are the Community Quality of Life Assessment Tool (COQL) and the Assessment Diagnosis Evaluation Planning Tool (ADEPT).
The COQL uses a touch-screen platform to provide social service providers with a quick and simple means of assessing the quality of life of low-income, isolated, frail, and often homeless, older adults. The COQL was successfully used in two implementation sites, the Well Elder Program and the San Francisco Senior Center, in San Francisco, CA. The CQOL tool helped to identify participant needs and direct clients to appropriate services. Insights from focus groups and staff interviews identified ideal staffing models and training needs of staff for use of COQL in comparable organizations.
ADEPT is a touch-screen based tool that staff uses to clinically assess clients, create plans of care, monitor clinical progress, and assess client outcomes. The ADEPT tool was successfully used to assess and support care planning for over 400 severely mentally ill older adults in San Francisco, CA. ADEPT has been translated into Mandarin and Spanish and is now part of the regular practice model of five programs serving chronically mentally ill seniors. Salesforce has recognized FSA’s intervention as a national best practice model and is considering it for distribution both domestically and internationally.
WHO WAS INVOLVED
Family Service Agency of San Francisco (FSA), San Francisco Senior Center (SFSC), WellElder Program, and Salesforce.
WHAT THEY DID
Problem Addressed: Connecting low-income seniors to services can be challenging when not knowing their present needs. Assumptions can be made on the part of the provider; however, by allowing older adults to report directly on their quality of life they can be empowered to identify needs they see as most relevant. This program shows respect for client preferences and puts more control in the hands of older adults. It also provides a more rigorous connection linking needs, services, and outcomes.
Patient Population: Low-income, isolated, frail, and/or severely mentally ill seniors living in San Francisco, CA.
Description of the Program: The program implemented two screening and assessment tools within segmented and often disorganized non-profit social service organizations that provide assistance for low income and severely mentally ill elderly individuals.
The technology components of this project are described below:
- The CQOL Tool (Community Quality of Life) is a touch-screen based, electronic version of the WHOCQOL questionnaire, with an additional 10-15 questions added developed by participating implementation sites. The survey generates graphical results that are reviewed by staff and clients. This assessment tool was designed to provide social service providers with a quick and simple means for assessing quality of life for low-income, isolated, frail and often homeless, older adults.
- The ADEPT Tool (Assessment Diagnosis Evaluation Planning) is a touch-screen based tool that clinicians use to assess clients, create plans of care, monitor clinical progress, and assess client outcomes. The ADEPT Tool consists of three parts: 1) a rigorous diagnostic expert system that assesses older clients for 13 major mental health and substance abuse conditions; 2) a quality of life tool that tracks quality of life across four principal life domains; and 3) an activities of daily living assessment tool. This ADEPT tool is similar to the CQOL Tool, but it is more comprehensive in nature and designed for outcome tracking for severely mentally ill individuals who are in intensive case management programs.
- CIRCE (Cloud Based Electronic Health Record) is a HIPAA compliant, cloud-based electronic health record that serves as the platform for ADEPT. Developed by FSA, CIRCE was designed as a client charting and reporting system to monitor productivity, contract compliance, and financial billing. An evaluation of the implementation of CIRCE identified several benefits: immediate increases in productivity, reduced time spent preparing case notes, and increased time spent with clients. The total amount of staff time spent on paperwork was decreased by half, resulting in 25% more face-to-face time with clients. The invoicing error rate decreased from 7.8% to 0.3%. Productivity per staff person increased by 17%. In partnership with Salesforce.com and UCSF, FSA expanded CIRCE’s utility by adding the Assessment Diagnosis Evaluation Planning Tool (ADEPT) and CQOL.
The CQOL Tool was administered to over 50 older adults at two sites, the San Francisco Senior Center and Well Elder. Sites varied in their staffing models for the project. The SFSC utilized CQOL volunteers who also assist in serving meals for the drop-in lunch program. Well Elder, a residential setting, utilized nursing and care coordination staff to administer the CQOL as part of regular services provided to residents. The CQOL Tool is introduced to clients as a means of identifying relevant resources in the community that the older adult may find beneficial, such as meal support, housing, and socialization. Staff and volunteers generate interest by introducing clients to the iPad, the touch-screen device used to administer the tool. If participants are interested, they begin a CQOL survey. The participants are notified that a results page will be generated in which they can discuss areas in their life they are doing well in, and areas that may need improvement. The identified needs are then compared with resources available in their local community via zip code, using the Silver Tsunami Resource Guide developed by the Coalition of Aging Services for the Elderly (CASE). Three volunteers at the SFSC and three staff at the Well Elder program were trained to administer the CQOL. These trainings covered how to operate hardware and software, interpret results, as well as how to build rapport with seniors and increase their comfort with computers. Staff at the SFSC and Well Elder programs also trained on how to use the results page generated in conjunction with the Silver Tsunami Resource Guide. The volunteers were required to complete one ‘shadow’ shift before they were able to administer the CQOL alone.
ADEPT Program: The ADEPT tool was used in assessing and supporting care planning for over 400 severely mentally ill seniors in the San Francisco Bay Area. Participants are asked to complete the ADEPT assessment using the computer “kiosk” (iPad) prior to each of their visits with a provider. The consumer kiosk is programmed such that at the end of each assessment, two separate reports are created: one for the consumer and one for the provider. The consumer report consists of an easy to read outline that informs consumers of their progress since their first clinic visit. The consumer then has the opportunity to share the report with their provider to aid in the discussion of their progress. The reports generated for the provider give detailed diagnostic information and measures of symptom severity and levels of functional impairment. When collected on reassessment, this information is used in clinical case conferences and in therapy sessions with the consumer to assess the success and focus of the treatment plan. All information gathered through the consumer kiosk is automatically transferred into an electronic client record, thereby eliminating the need for data entry.
CQOL Program: Use of CQOL successfully helped to identify participant needs and direct them to the appropriate resources within both implementation sites. Many older adults are not comfortable verbally discussing if they have problems accessing meals, are socially isolated, or are residing in poor living conditions. Use of the electronic survey allowed participants to disclose this information, initiating the process of connecting them with services to improve their quality of life. Participants had a range of responses to the tool. About one-third had no problem using the touch-screen interface independently; one-third used the interface but required the help of a volunteer or staff member; and one-third was unable to complete the interview using the touch screen.
Staff reported some challenges in implementing the CQOL Tool. Recruitment and follow-up at both sites was challenging and lower than expected. Some staff members administering CQOL at both sites were not confident in their computer skills and how to use CQOL on the iPad. Staff felt this inhibited their ability to recruit participants as they were less capable of demonstrating the tool and speaking to the benefits of the assessment. CQOL volunteer turnover at SF Senior Center led to decreased program enrollment, as volunteers needed to undergo training and spend time connecting with participants. Key informant interviews suggested that staff did not feel that care management was a key element of their job responsibility. However, staff did report that CQOL results could be useful if the programs had provided a comprehensive wrap-around structure to use them. Ultimately, low recruitment and staff computer literacy, as well as high volunteer staff turnover, led to limited program sustainability.
ADEPT Program: The ADEPT tool was successfully used to assess and support care planning for over 400 seriously mentally ill seniors. ADEPT allowed clinicians to quickly screen and assess clients and use results to inform treatment planning. The results of the ADEPT assessment successfully provided a benchmark for client symptoms, quality of life and functionality against which progress could be matched with monthly reassessments. Visually intuitive graphics enabled the client to quickly grasp areas in which they were making progress and to work with providers to adjust their treatment plan to address areas where they were not making progress. In partnership with UCSF and SRI, FSA applied artificial intelligence to interpret case notes and assessment and reassessment trends to make treatment recommendations. With artificial intelligence able to consider thousands of past case notes and every change in assessment results instantly, ADEPT is able to identify trends or trajectories that a clinician could not catch and make treatment recommendations based upon consideration of far more information than a clinician could grasp at once. ADEPT successfully expanded users’ capacity to conduct care planning.
ADEPT has been translated into Mandarin and Spanish. The tool is now part of the regular practice model of five programs serving chronically mental ill seniors. Salesforce has recognized FSA’s intervention as a national best practice model and is considering it for distribution both domestically and internationally.
ADOPTION CONSIDERATIONS AND LESSONS LEARNED
Focus Groups: Focus groups with FSA clients were an instrumental part of developing and improving the ADEPT and CQOL Tools. FSA also convened a monthly Peer and Consumer Task Force in which consumers, volunteers and staff came together to discuss edits and changes needed to the tool. The focus groups helped identify that the quality of life section of the tool was very informative and was a topic most consumers hadn’t considered prior to using the ADEPT Tool.
Getting Started With This Innovation: The essential items required to successfully launch each of these tools is a good working knowledge of the participant population, and a strong relationship with partner agencies. This can be facilitated through a program manager who is able to maintain communications and relations with partner agencies in order to secure buy-in. There must also be a commitment to include participants in every step of planning for the implementation of these tools (e.g., focus groups). Having IT staff who understand the technology and programming needs is also essential for maintaining the technology interface and program database.
Sustaining and Scaling Use of the Technology: Both the CQOL and the ADEPT tools must be part of a program’s standard treatment plan and service delivery model for them to be fully adopted and sustained. For example, ADEPT and its core underlying technology, CIRCE, have been integrated into the practice model of FSA throughout its Senior Division. ADEPT is available through Salesforce.com’s App Exchange. ADEPT can be downloaded and installed by anyone with a Salesforce license. For broader scaling, ADEPT users are able to customize the technology independently, through a third party, or through FSA. ADEPT and CIRCE are now being used in the California Central Valley, Mexico, and China.
Ensure Staff Computer Literacy. Staff administering ADEPT and CQOL tools must be trained and comfortable with technology and computers themselves in order to be able to convey the benefits of completing the survey or assessment in a genuine and relaxed way. For example, while several staff said they felt comfortable with administering the CQOL and using the technology, their actual computer literacy was low, significantly restricting their ability to train clients on how to use the iPad.
Conduct Regular Staff Training and In-services. Many staff members felt they did not have satisfactory training and methods for soliciting clients to use the CQOL. Frequent training, retraining, and check-ins with staff are recommended. Role-playing of CQOL administration and recruitment by administrators, program managers and coordinators can ensure that staff is knowledgeable and comfortable with the technology and the tool.
Build off of Successful Relationships – Utilizing Volunteers and Full-Time Staff. Using volunteers as administrators of CQOL is a very successful method of engaging participants. Volunteers often have more opportunities to engage and build rapport with participants. Once participants become more comfortable with the volunteers they become more receptive to using a technology-based tool. It should be noted however that volunteer turnover can greatly hinder participation. Using program staff members may improve recruitment in that staff members already engaged with clients do not need additional time to build rapport.