Website: https://www.jeffdavishospital.org/
Introduction
Jeff Davis Hospital (JDS) is a Hospital System located in Hazlehurst, GA that regionally serves patients affected by cardiovascular conditions. The Healthcare Group’s interest is to improve usability for users that leverage technology to coordinate their healthcare needs as this becomes a further need overtime with integrated networks and higher use of Healthcare Technology. In previous Heuristics Evaluations, the site presented violations around Consistency, Standards, error prevention, and minimalist design (Nielsen, 2020). The focus of user testing in the established representative tasks is to observe navigability on the site as a common problem of heuristics violations.
Persona and User Characteristics
In summary, the Healthcare Group opted for a User Persona representative of previously identified use cases of healthcare system websites. In summary, our persona is named Jennifer, an adult-child intending on transferring her father from a larger hospital to JDS to continue recovery and rehabilitation from a stroke closer to home. Jennifer is using the JDS website for the first time seeking transfer services and elderly-specific outpatient care coordination.
The user tester who volunteered was a 36-year old adult female (she/her) located in Florida. Similar to the persona, this user tester has experience with family members treated for post-operative cardiovascular care. Likewise, she resides regionally close to population areas serviced by JDS. The user tester was not prompted about prior personal or family healthcare history but disclosed the information during orientation with consent.
Testing Methods
Test Conduct and Materials are based on Rubin and Chiswell’s (2008) Methodologies and Guidelines, using “Think Aloud” Technique in observation with minimal tools. Concurrent Think Aloud (CTA) was primarily used with careful attention to apply impartiality with clarifying questions, resulting in qualitative user opinions and feedback of the website as user-generated data (Running a usability test, 2022). This method encourages the user to actively state how she is thinking through the task (Rubin & Chiswell, 2008).
Retrospective Scoping was used to collect quantitative data by form of a Likert Scale response with the following question and any feedback around it:
“On a scale of 1 to 5, how easy or difficult was it to complete this task? 1 being Extremely Easy, 5 being Extremely Difficult, and 3 being Neutral.”
In addition to the likert response, each task was timed to completion. Criterion for completion was to load the correct webpage according to the task and sitemap. Somewhat Difficult-4 or Extremely Difficult -5 scores indicate perceived effort in the task and time to completion provides a baseline to compare when re-designs are tested.
Materials used were the user’s iPad, a browsing application, a teleconferencing application, and a timer. Apple’s FaceTime was used to allow screen sharing of Google Chrome and screensharing virtually. Initially a desktop or laptop was requested, but an iPad was only available and was able to replicate the desktop website. Qualitative and quantitative measures were recorded using a Markdown Editor. Screenshots were collected from the active FaceTime Call or replicated from user actions. Audio or video recordings were not collected.
Orientation was conducted by first informing the user tester that a persona and three tasks would be provided and tested. She was encouraged to actively describe and comment as she completed each task. Measurements were disclosed and the user tester was reassured that results were solely for testing the website. Once orientation was complete, the persona was read as stated in Group Assignment 2 and each task was prompted to initiate testing.
Representative Tasks and Changes
- Find Information about symptoms and recommendations for Stroke as a healthcare topic on the Jeff Davis Hospital website.
- Find Patient Transfer Services for the hospital.
- Find information about Elderly Care Coordination.
Task 1 was expanded to include specific language about seeking information about stroke “symptoms and recommendations… as a healthcare topic on the Jeff Davis Hospital website” This change was recommended feedback by Dr. Paul Marty with Prototype Testing. Shorter task descriptions implied only general research on strokes, which more likely start with a search engine query. This change ensured tasks are logically linked and confined navigation to the tested website. All other tasks were unmodified.
Test Results and Analysis
| Task | Time to Complete | Likert: Easiness-Difficulty | Feedback Highlights |
| 1 – Find Information About Stroke | 5 min 6 sec | 4 | Inconsistent Responsive Design, outdated site, too many external links, poor guides, and distrust with ad revenue |
| 2 – Find Patient Transfer Services | 3 min 55 sec | 5 | “Swing-Bed” Non-standard terminology; lack of requirements and procedure information |
| 3 – Find Elderly Care Coordination | 1 min 30 sec | 5 | Unclear differences between Rehab Options, Uncertainty being referred to another branded facility |
Results from Task 1 – Find Information About Stroke
Completed in 5 minutes 6 seconds, Task 1 was perceived as 4 – Somewhat Difficult, with major issues around navigation and link redundancy. Previously unknown on the desktop browser, the user tester first failed the task by discovering a searchbar. With iPadOS 16.1 Stage Manger, the website opened as a mobile version, shown in Figure 2. Search results prompted advertisement hyperlinks followed by more accurate page results. However, the task was failed by the user not noticing the correct links. The user tester indicated distrust at this point with the perception that hospital sites should not collect ad revenue.
2 minutes into the task after restart, the user tester correctly identified the “Health Research Center” as the appropriate page through the Patient Services Menu and identified error prevention issues navigating disjointed links to the legacy FastHealth site. In addition to the links to “Stroke” and the FastHealth Image, the user tester discovered additional links under the Subsection “Online Health Search Engine” which further broke down more disparate links about stroke into the FastHealth Dictionary Link as a hosted copy of the Merriam-Webster Medical Dictionary, General Health Research Link as another redirect, and “FastNurse” Link which displayed another hyperlink to launch an email client for contacting a nurse; all shown in Figure 3. As predicted, the user tester was not satisfied that the site had portions of information about stroke located in separate pages that encouraged navigation to sites away from JDS.
Results from Task 2 – Find Patient Transfer Services
Completed in 3 minutes 55 seconds, Task 2 was perceived as 5 – Extremely Difficult, with major issues around terminology of the “Swing-Bed Program” and lack of information on the patient transfer process. At first, the user tester failed the task by navigating to the “Patient Information” Submenu Link under the “Patients and Visitors Menu.” The only actionable information identified was a phone number to Admitting/Registration Services.
On second attempt, the user tester resized the window to trigger the search bar and searched for “Patient Transfer Services,” prompting the Swing-Bed Program Link as the only non-advertisement search result from SEO. After guiding the user tester back to the main site menu, the Swing-Bed Program Link was revealed to be the obvious option under the Our Services Submenu. The user-tester read the webpage and observed over-explanation of the “Swing Bed” term to define it as patient transfers. She also cited lacked information on what she needs to collect in paperwork to start her father’s transfer and the maximum distance allowed.
Results from Task 3 – Find Elderly Care Coordination
Completed in 1 minute 30 seconds, Task 3 was perceived as 5 – Extremely Difficult, with major issues around the terminology and poor descriptions of different services offered at JDS. The user tester initially failed the task by identifying “Respite Care” under “Our Services.” After redirecting her to the main menu and clarifying the term “care coordination” as a comprehensive set of services, she opened new tabs for “Hospice” and “Harmony Center” and cross compared the specified services – as well as comparing Respite Care. Time to complete in this task was short despite the difficulty due to this multi-tab navigation behavior. After reviewing these rehab services, the user tester concluded Harmony Center as the best option. However, she indicated that her decision was based on the lack of information from the other web pages. The user tester also indicated an impression that the Harmony Center brand was not part of JDS and thought this was another instance of the website directing her to other healthcare providers.
Design Recommendations
User Testing confirms that re-design of the site navigation will need redrafting of both old and new site content under a clearer menu to reduce error prevention and promote minimalist design. Selecting a new on-brand template and migrating the FastHealth content into its own knowledgebase will help reduce the disjointed interfaces and external links experienced. Then, by mitigating or removing the brands from the webpage content and centering a well-written introduction of services, the user will be able to verify quicker in taking action on their care. Alternatively, clustering services into single webpages may help if they can be ordered by increasing acuity. Considering focus on cardiovascular servicing and expertise, content and navigation can also be centered around this specialty.
As observed with difficulties of the user tester finding Elderly Care Coordination as Harmony Center and Patient Transfer Services as Swing Bed Program, renaming and reorganizing the main site menu terms and structure will improve comprehension and navigability. Renaming the two brand names to their obvious titles or standardizing under a medical terminology schema is preferable, especially to be in parity with competing hospitals. As shown in Figure 4, simply breaking apart the 15 links for “Our Services” and 13 links for “Patients & Visitors” can help alleviate overload and confusion. “Our Services” can potentially be separated as “Departments” and “Special Services” due to half of links being specialty specific and the other half palliative or rehabilitative services. “Patients & Visitors” can break into “Patients” and “Visitors & Policies” to separate personal health versus corporate information.
Finally, if user responsive design continues under the new template with persistent search, better search engine optimization matched to revamped navigation and removal of advertisements will not only promote consistency but regain overall stakeholder trust in the site content. Incorporating knowledge articles into search will reduce completion time from topic to information action. As shown by the user tester, search was always preferable to menu navigation. If search becomes more frequent, advertisement exposure will be more frequent but cause similar critical errors as using the antiquated FastHealth site. If re-branding and re-design is implemented, removing advertisements is a necessary step to adopting changes.
Total Word Count: 1600
References
Jeff davis hospital. (2022). Jeff Davis Hospital. https://www.jeffdavishospital.org/
Jeff Davis Hospital/fasthealth corporation (Hazlehurst, Georgia – jeff davis county). (2022). Jeff Davis Hospital/FastHealth Corporation. http://www.jeffdavisfasthealth.com/
Nielsen, J. (2020, November 15). 10 usability heuristics for user interface design. Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/
Planning a usability test. (2022). Usability.gov. https://www.usability.gov/how-to-and-tools/methods/planning-usability-testing.html
Rubin, J. & Chisnell, D. (2008). Handbook of usability testing: How to plan, design, and conduct effective tests (2nd edition). Wiley.
Running a usability test. (2022). Usability.gov. https://www.usability.gov/how-to-and-tools/methods/planning-usability-testing.html
Backlinks
Group Assignment 2 – User Scenarios and Representative Tasks