Loren Data's SAM Daily™

FBO DAILY ISSUE OF JANUARY 06, 2013 FBO #4061
DOCUMENT

D -- HMP Usability Analytics Platform (UAP)-RFI - Attachment

Notice Date

1/4/2013
 

Notice Type

Attachment
 

NAICS

511210 — Software Publishers
 

Contracting Office

Department of Veterans Affairs;Technology Acquisition Center;260 Industrial Way West;Eatontown NJ 07724
 

ZIP Code

07724
 

Solicitation Number

VA11813I0075
 

Response Due

1/31/2013
 

Archive Date

3/2/2013
 

Point of Contact

Michael Brill
 

E-Mail Address

8-5574<br
 

Small Business Set-Aside

N/A
 

Description

T-16 Health Management Platform (hi2) Usability Analytics Task 2 Market Research for Requirements Development Request for Information (RFI) HMP Usability Analytics Platform (UAP) BACKGROUND: Veterans Health Administration (VHA) Transformational Initiative #16, Transforming Health Care Delivery through Health Informatics Initiative (hi2) will shape the future of VHA clinical information systems through deliberate application of health information technology (IT) and informatics to deliver solutions that transform health delivery to Veterans directly improving quality and accessibility while optimizing value and ensuring VA continues industry leadership in the use of health informatics and health information technology. The Health Informatics Initiative is comprised of sub-initiatives. The effort described herein will support the Health Management Platform (HMP). The HMP is the foundational IT tool that is helping change the future of health care delivery at VA. This technology allows care teams, software developers, technology partners, and all VA staff to participate in truly advanced delivery of patient-centered care. The HMP platform consists of service orientated architecture that comprises three main abstraction layers. The HMP is implemented in Java and utilizes current technologies such as restful services and JavaScript Object Notation (JSON), to implement next generation modular interfaces that are customizable to the user's needs. Central to the HMP architecture is a design allowing for collaborative development, allowing for development partners to add additional functionality or modules to the Health Management Platform. The HMP has three outward facing components: patient-facing, care team-facing and health care system-facing. The HMP is a software platform providing new tools to allow for the transformation of VA health care toward a patient-centered, team based care model. The platform is being designed to provide the flexibility to allow the health care team to interact with information in new ways that better support and manage their patient care workflow. The patient-facing platform will allow Veterans and their delegates to receive and exchange health information and engage in their care in unprecedented ways, through a variety of electronic devices. Additionally, the HMP will enable patients to interact with and contribute to their medical record. The system-facing or population-facing platform harnesses the power of population-level data to monitor and improve the health care system. It will allow for the creation of innovative tools using data from across VA IT systems and patient populations. It will also promote the operationalization of research work, and integrate large-scale system analyses into the workflow of the electronic health record (EHR). In order to create a Health Management Platform that is efficient, relevant, and responsive to care teams, patients, population health workers, and hospital system personnel, the HMP team needs an analytic software platform, tools, and support system to quantitatively and qualitatively capture and profile HMP usability and produce usability-related analytics for continual improvement, data mining and response across the wide diversity of user roles, needs, goals, situations, and actions. Analytics and Usability Analytics: Analytics is a broad umbrella term for a rapidly advancing interdisciplinary field that leverages the large volumes of heterogeneous data being generated and managed in today's IT-driven world. Broadly, the goal of Analytics is to extract useful, actionable knowledge of events, trends or patterns from the "deluge of data" to guide workflow improvement, decision making or automating business or clinical logic. Usability Analytics is a subcategory of (Enterprise) Analytics that focuses on usage and usability patterns of Health IT systems. Usability, as defined by the National Institute of Standards and Technology (NIST), refers to "effectiveness, efficiency and satisfaction with which intended users can achieve their tasks in the intended context of product use." One of the pillars of functionality is to measure how the "system" is performing, leading to intervention or system redesign where necessary. Desired functional capabilities include real-time visibility into the system with transparency across the enterprise. The effort described here focuses on development of an integrated investigative platform to support usability analytics, performance profiling, and business intelligence needs of HMP. This requires developing software systems that can automatically capture, manage and analyze data generated by users interacting with the HMP modules and integrate this data and the knowledge bases for development of HMP Analytics modules. This RFI focuses on steps that precede actual development or procurement, paying particular attention to needs for integrative technologies - especially standards, interoperability, protocols, and middleware - required for large-scale complex system design. To create an HMP that is continually improving, the VA needs to be able to automatically collect and analyze usability data. Currently, collection of data regarding usability relies on time-consuming and ad-hoc manual and semi-automated techniques. Further, off-the-shelf usability software is limited by the need for manual review and screen scraping to generate data useful for usability analysis. Quality assessment techniques should be integrated within the infrastructure so that data can be directly captured throughout the care process. Moreover, HMP team members need access to automated tools to help analyze the data. It is anticipated that there will be diversity in terms of types of data and reports that are needed to support the diverse needs of HMP team members. For instance, administrators may require data on user satisfaction and workflow, researchers may want to study medical errors, and usability experts may be interested in User Interface (UI) efficiency (e.g., task completion rates). The software and tools suggested and/or specified should work seamlessly within actual clinical settings such as a scheduled clinic appointment or an inpatient assessment during attending rounds on a ward with automated data capture during real use of HMP tools. Key functional specifications of the usability analytics platform include: "Interoperability with HMP application stack (e.g., Data Access Layer, Virtual Patient Record, Business Logic, User Interfaces) and data model. "Provision of near-time monitoring capabilities of HMP usage patterns integrating a variety of data sources including databases, Web tracking (e.g., HTTP client-server network traffic, and User Interface events like click-streams or gestures on mobile devices). "Predefined and custom segmentation and aggregation, dashboards and visualization for HMP activity at the level of individual clinicians, irrespective of location or workstation, at the level of care team (e.g. HMP activity during a 5 hour Emergency Room visit related to a patient) and at the level of individual patients and patient populations. "Provide advanced customization of variables, algorithms (e.g. segmentation, aggregation, and temporal data mining), reports, including visualization. "Support the ability to export the data. "Configuration Management (accounts, web properties, profiles, goals, segments). "Collects and integrates user feedback and enables crowd-sourcing activities to aggregate feedback across users in a meaningful way. (Bugzilla, Google +1 votes, Amazon.com reviews with star ratings, feedback on users who provide reviews and helpfulness of the review, and Facebook likes are examples of user feedback solutions for commerce, rather than health care IT solutions.) "Support for plug-ins and third-party solutions. "Provision of online developer guides and documentation of resources. The scope of this effort is to provide information regarding: 1.Types of data needed to perform ongoing HMP usability analysis, including, for example, usage patterns and user interaction with the software. 2.Characteristics of usability analytics platforms best suited for supporting HMP usability analytic methods (e.g., sequential task analysis). Data collected by UAP can be combined with other data (e.g. field observations) to provide context. 3.Types of usability analytics visualization capabilities required for analyses and reporting of data (e.g., heat maps, time-series bubble charts, activity transition graphs). 4.Specifications for an analytics infrastructure that captures and manages usability and usage information from within the HMP to guide continuous quality improvement in IT as well as clinical process improvement efforts 5.Suggestions for existing available software, software layers, platforms and deployment models for enterprise architectures of up to 150+ facilities and 300,000 users. 6.Scalability and Interoperability of HMP Usability Analytics platform should be addressed with current electronic health record standards supporting the continuity of care delivery in a complex healthcare environment. General information "HMP will be accessed through a variety of platforms, including tablet devices, smart phones, and traditional PCs. "The UAP will gather data from but, will not write data to, HMP. The data collection must not slow down HMP performance. "The UAP will likely need to capture a variety of GUI events and store those events as timestamps with a series of related categorical variables, such as process type, specific process, specific event trigger, or other pre-determined categorical variables. These categorical variables are actively being developed. "The UAP will meet federal data security and privacy requirements. "The UAP will need to allow queries and statistical analyses to be programmed into the UAP system as metrics for repeated use. If possible, these queries and analyses should be able to be easily shared across the 150+ VA medical facilities that may be involved in continual maintenance and modification of HMP. "The UAP will run reports on a near-real-time basis to allow for active monitoring of specific metrics for dashboards or statistical process control use cases. "The UAP will enable direct, integrated comparison between observed user behavior and expected user behavior at an arbitrary level of (sub-)task element, where expected user behavior could be based on a variety of sources and models, including (for example) GOMS models, cognitive architectures (e.g. ACT-R), empirical/statistical models of behavior, or simulation models (e.g. MicroSAINT).. This is a request for information (RFI) only. Do not submit a Task Execution Plan (TEP). It is requested that all companies interested in participating in this effort please note their interest and provide indication of their respective capabilities to perform the effort described in the paragraphs below by 2:00 PM EST Thursday January 31, 2012. Your response should include the following: a.Provide a brief summary of your technical approach to meet the requirements. Address each of the six sections above. b.Submit the name, email address and phone number of the appropriate representative of your company. c.Submit your response via email (subject heading: "T-16 Health Management Platform (hi2) Usability Analytics Task 2 Market Research for Requirements Development: RFI RESPONSE: YOUR COMPANY NAME). This should be submitted to Michael Brill, Contract Specialist at michael.brill@va.gov by 2:00 pm (EST) on Thursday, January 31, 2012. d.Mark your response as "Proprietary Information" if the information is considered business sensitive. e.NO MARKETING MATERIALS ARE ALLOWED AS A PART OF THIS RFI. f.All questions shall be submitted in writing by email to Michael Brill Contract Specialist at michael.brill@va.gov by 2:00 pm (EST) on Friday, January 18, 2012. The Government does not guarantee that questions received after January 18, 2012 will be answered. This RFI is for planning purposes only and shall not be considered an Invitation for Bids, Request for Task Execution Plan, Request for Quotations or a Request for Proposal. Additionally, there is no obligation on the part of the Government to acquire any products or services described in this RFI. Your response to this RFI will be treated only as information for the Government to consider. You will not be entitled to payment for direct or indirect costs that you incur in responding to this RFI. This request does not constitute a solicitation for proposals or the authority to enter into negotiations to award a contract. No funds have been authorized, appropriated or received for this effort. The information provided may be used by the VA in developing its acquisition strategy and Performance Work Statement. Interested parties are responsible for adequately marking proprietary, restricted or competition sensitive information contained in their response. The Government does not intend to pay for the information submitted in response to this RFI. APPENDIX 1: CLINICAL USE CASE EXAMPLES The UAP will be used to identify areas of the HMP that may not be meeting the needs of the end-users. Metrics that have been determined to indicate possible poor/good usability will have thresholds to determine when to "red-flag" a specific metric. Management, HMP developers, HMP local facility maintenance, continuous quality improvement (CQI), or other groups will monitor relevant metrics to determine when to deploy human resources to further investigate a potential GUI, workflow or feature shortcoming. A Conceptual Example: Consider a metric which concerns processes that result in binary outcomes, such as "OK" or "Cancel." When prescribing a medication, once the medication ordering process has been initiated, only two outcomes are possible: (1) a medication order is entered or (2) the process is cancelled. This metric is "red-flagged" when the medication ordering process is initiated more than twice without a medication order being entered. "Facility management: Following a change to the medication formulary (medications available for order), facility management notices an increase in red flags associated with medication ordering. Facility management knows that a formulary change was just made and decides to provide some additional reminders to clinicians concerning the formulary change. Facility management continues to monitor medication ordering for the next 90 days to see if more resources should be deployed to finding out why end-users are having difficulty with the medication ordering process. "HMP developers: HMP developers notice a spike in red flags concerning medication ordering across multiple sites. HMP developers contact facility management at a few of the sites and do not uncover any reason for the increase in red flags, so HMP developers reach out to local systems redesign groups to investigate the medication ordering process. The systems redesign groups find that a new set of hypertension guidelines require monitoring certain lab values and family history items during the ordering of most blood thinners. The current HMP module must be cancelled in order to view those values, so HMP developers raise the priority of the medication ordering module and release a new version to allow the end-user to reference the needed laboratory information during the medication ordering process. "HMP local facility maintenance: After pruning a redundant medication delivery method, HMP local facility maintenance notices an increase in red-flags for the medication ordering process. HMP local facility maintenance immediately rolls back to the previous version and temporarily programs the redundant option to be routed as if the standardized option had been chosen. HMP local facility maintenance then provides information to clinicians about the upcoming removal of the redundant option and rolls out the original update 4 weeks later when the end-users are prepared for the change. "CQI: CQI is making changes to the medication ordering process locally and monitors the red-flags on the medication ordering process as various changes are piloted. The red-flags are used as a "voice of the customer" measure along with the other performance measures to determine which process change is optimal. APPENDIX 2 - EXAMPLES OF VISUAL ANALYTICS GENERATED FROM EHR USAGE DATA Below are examples of analyses performed to study actual EHR usage within the VA. These analyses use off the shelf usability software for capturing EHR screen activity and associated mouse clicks that are then manually coded to specific EHR tasks. Visualizations were performed using statistical software. Figure 1. EHR Mouse Activity per Visit. The number of mouse clicks is shown as the area of the disks, with absolute values displayed in the red legend. X Axis is EHR section (Not = Notes; Ord = Orders; Med = Medications; Lab = Labs; Rep = Reports; Con = Consults; Cov = Cover Sheet; Sig = Review/Sign; Oth = Other; Y-axis is Provider numbers (egg, "01," "02," etc.).Figure 2. Sample chromogram displaying sequential tasks analysis. Graphic was generated from EHR mouse clicks that are color coded to EHR activity along the y-axis. The x-axis is time in minutes. Figure 3. Example of EHR Activity Transition Graph. This is an example of an aggregate activity transition graph used to visualize EHR task transitions across multiple clinical visits. Nodes correspond to EHR activities (e.g. placing orders or viewing labs) and arcs correspond to observed transitions. Derived measures based on Transition Graphs, such as weighed node degree (the sum of all arc values in and out of a node) can quantify the "centrality" of a particular node. Another application of the analysis is to inform the identification of commonly co-occurring tasks, identification of abandoned tasks, and also circular transitions that may represent poor usability. ? Provider A (x visits) Provider B (x visits) Figure 4. Heat map of Sequential Pattern Analysis (SPA) example. In Figure 3, we outline an example of a heat map obtained from the mouse-click data. The heat maps show patterns of EHR tasks and transition activity for 8 visits and 2 providers. Each of these 8 heat maps represents a 13 x 13 adjacency matrix. Along the X axis and the Y axis, each small square represents an EHR Task (e.g., F=Notes, E=Orders, D=Meds, J=Labs). Entries along the diagonal correspond to consecutive clicks within the same EHR Task (e.g. Notes'Notes). Off-diagonal entries correspond to transitions between EHR Tasks (e.g. Notes'Orders). The color of each small square represents the number of such transitions (red = >100, white = 0). This information can help identify the most frequent transitions between EHR screens or Task A and the degree of variability of transitions across visits. From the perspective of EHR UI efficiency and design, pairs of Tabs with highest frequency of transitions could be integrated into a single screen, thus minimizing inefficient navigation between screens. ? Meds Labs Consults Reminders Figure 5. Order Entry Click Stream Analysis. These are stem and leaf plots where X-axis shows click stream sequence number (1= first order, 2= second and so forth), and the counts of each sequence denote CPO (Clicks Per Order) and total counts denote CPV (clicks per visit). These analyses can be done for various activities such as EHR order entry tasks for meds, labs, consults, imaging, or for satisfying alerts and reminders. In this example Sequential Task Analysis of EHR mouse activity is applied to quantify the burden of EHR order entry. It relies on capturing and manually coding individual mouse clicks, a process that can be automated by Usability Analytic Platform. Such click stream analyses are more detailed than window level coding, and can straddle activity across windows, and distinguishes several common categories of orders including Meds, Labs, Consults, Imaging, and Reminders. Key derived measures of EHR UI efficiency include Click Per Order (CPO) rate (i.e. number of mouse clicks per order). CPO can be interpreted as UI inefficiency, but also reflects the complexity of EHR workflow of specific orders. Such analysis can inform usability experts and software designers about the inefficiencies for specific order menus and help drive EHR redesign.
 

Web Link

FBO.gov Permalink
(https://www.fbo.gov/notices/876b94ec9df515db13a3cff08a61ac30)
 

Document(s)

Attachment
 

File Name: VA118-13-I-0075 VA118-13-I-0075_1.docx (https://www.vendorportal.ecms.va.gov/FBODocumentServer/DocumentServer.aspx?DocumentId=574086&FileName=VA118-13-I-0075-000.docx)

Link: https://www.vendorportal.ecms.va.gov/FBODocumentServer/DocumentServer.aspx?DocumentId=574086&FileName=VA118-13-I-0075-000.docx

 

Note: If links are broken, refer to Point of Contact above or contact the FBO Help Desk at 877-472-3779.
 

Record

SN02959650-W 20130106/130104234536-876b94ec9df515db13a3cff08a61ac30 (fbodaily.com)
 

Source

FedBizOpps Link to This Notice
(may not be valid after Archive Date)

---

| FSG Index  |  This Issue's Index  |  Today's FBO Daily Index Page |