Seattle City Light

Breaking Down Barriers to Incentive Program Accessibility

Role

UX Researcher

UX Researcher

UX Researcher

DURATION

9 Weeks

9 Weeks

9 Weeks

TOOL

FigJam

FigJam

FigJam

Method

Usability Testing

Usability Testing

Usability Testing

Project Overview

This study aimed to evaluate the usability and accessibility of the Seattle City Light website in providing information about incentive programs to small and medium-sized business owners. The usability testing specifically focused on users' experiences navigating to the "Small and Medium Business Solutions" page and understanding the associated "Commercial Retrofit Program Requirements Documentation."

We assessed whether the website allowed business owners to easily navigate, access information, and comprehend incentive program requirements. We set goals, developed test plans, recruited participants, conducted usability tests, analyzed data, and delivered findings and recommendations.

Research artifacts

What I worked on

Led the planning and execution of the entire project, including developing the study kit, overseeing usability testing, performing data analysis, and composing the final report and presentation.

Conducted usability testing sessions as the lead interviewer and facilitator. Effectively analyzed the testing data to generate insights and key findings.

Managed client communication, coordinated team activities, and tracked project progress.

What I worked on

Led the planning and execution of the entire project, including developing the study kit, overseeing usability testing, performing data analysis, and composing the final report and presentation.

Conducted usability testing sessions as the lead interviewer and facilitator. Effectively analyzed the testing data to generate insights and key findings.

Managed client communication, coordinated team activities, and tracked project progress.

What I worked on

Led the planning and execution of the entire project, including developing the study kit, overseeing usability testing, performing data analysis, and composing the final report and presentation.

Conducted usability testing sessions as the lead interviewer and facilitator. Effectively analyzed the testing data to generate insights and key findings.

Managed client communication, coordinated team activities, and tracked project progress.

Who I worked with

I collaborated with 3 master's students as part of a usability studies course at the University of Washington. Our team worked as contractors for Seattle City Light and Seattle Office of Economic Development.

The Seattle City staff provided valuable support throughout our research, offering insights and resources that contributed to the successful outcomes of this project.

Who I worked with

I collaborated with 3 master's students as part of a usability studies course at the University of Washington. Our team worked as contractors for Seattle City Light and Seattle Office of Economic Development.

The Seattle City staff provided valuable support throughout our research, offering insights and resources that contributed to the successful outcomes of this project.

Who I worked with

I collaborated with 3 master's students as part of a usability studies course at the University of Washington. Our team worked as contractors for Seattle City Light and Seattle Office of Economic Development.

The Seattle City staff provided valuable support throughout our research, offering insights and resources that contributed to the successful outcomes of this project.

Impact

Identified 6 key usability issues, provided actionable recommendations to improve accessibility and program clarity, and presented to tens of Seattle City staff.

Clarified target user needs and outlined next steps for future research. Streamlined participant recruitment through improved timing and planning strategies.

Implemented data-driven updates to enhance user experience and inspire improvements across multiple energy efficiency programs.

Impact

Identified 6 key usability issues, provided actionable recommendations to improve accessibility and program clarity, and presented to tens of Seattle City staff.

Clarified target user needs and outlined next steps for future research. Streamlined participant recruitment through improved timing and planning strategies.

Implemented data-driven updates to enhance user experience and inspire improvements across multiple energy efficiency programs.

Impact

Identified 6 key usability issues, provided actionable recommendations to improve accessibility and program clarity, and presented to tens of Seattle City staff.

Clarified target user needs and outlined next steps for future research. Streamlined participant recruitment through improved timing and planning strategies.

Implemented data-driven updates to enhance user experience and inspire improvements across multiple energy efficiency programs.

Reflection

Adapting traditional usability testing methods for PDF documents required treating them as long web pages, demonstrating flexibility in research methodology for unconventional formats.

Coordinating scripts, note-taking, and reporting across multiple moderators reinforced that rigorous process design keeps data consistent and collaboration smooth.

Methodological triangulation revealed complementary insights that would have been missed through a single research approach.

Reflection

Adapting traditional usability testing methods for PDF documents required treating them as long web pages, demonstrating flexibility in research methodology for unconventional formats.

Coordinating scripts, note-taking, and reporting across multiple moderators reinforced that rigorous process design keeps data consistent and collaboration smooth.

Methodological triangulation revealed complementary insights that would have been missed through a single research approach.

Reflection

Adapting traditional usability testing methods for PDF documents required treating them as long web pages, demonstrating flexibility in research methodology for unconventional formats.

Coordinating scripts, note-taking, and reporting across multiple moderators reinforced that rigorous process design keeps data consistent and collaboration smooth.

Methodological triangulation revealed complementary insights that would have been missed through a single research approach.

Timeline

Project Motivation

Our clients from Seattle City Light and Seattle Office of Economic Development (OED) collaborated to bridge the gap for small and medium-sized businesses to effectively leverage Seattle City Light's incentive programs and achieve their energy goals.

However, it was found that the existing program materials were not easily understandable by most laypeople with a small business, which highlighted the need of enhancing the accessibility and clarity of these resources.

Turning Client Questions into Research Questions

To guide project planning and scoping, we broke down our clients' broad questions into specific, user-centered inquiries to inform the research process. Their goals centered on two aspects: website navigation (how users discover and access program information) and documentation comprehension (how well users understand the program requirements).

Client Questions

Research Questions

1

"How to make more awareness of our incentive programs for people looking to make a clean energy project and ensure seamless access?"

Website navigation

How effectively are the users able to navigate to the small and medium business solutions page?

What are the existing barriers hindering the users' access to information of incentive programs?

Are users able to access the commercial retrofit program requirements documentation on the website?

2

"How to improve the user experience by making that information easily digestible?"

Documentation comprehension

Are users able to effectively use the requirements documentation to determine their eligibility for the program?

How clearly are lighting and non-lighting product specifications presented, and what factors impact their understanding?

Are users able to effectively use the requirements documentation to decide their further course of action?

Are users able to effectively use the requirements documentation to determine their eligibility for the program?

How clearly are lighting and non-lighting product specifications presented, and what factors impact their understanding?

Are users able to effectively use the requirements documentation to decide their further course of action?

Are users able to effectively use the requirements documentation to determine their eligibility for the program?

How clearly are lighting and non-lighting product specifications presented, and what factors impact their understanding?

Are users able to effectively use the requirements documentation to decide their further course of action?

Are users able to effectively use the requirements documentation to determine their eligibility for the program?

How clearly are lighting and non-lighting product specifications presented, and what factors impact their understanding?

Are users able to effectively use the requirements documentation to decide their further course of action?

Are users able to effectively use the requirements documentation to determine their eligibility for the program?

How clearly are lighting and non-lighting product specifications presented, and what factors impact their understanding?

Are users able to effectively use the requirements documentation to decide their further course of action?

Are users able to effectively use the requirements documentation to determine their eligibility for the program?

How clearly are lighting and non-lighting product specifications presented, and what factors impact their understanding?

Are users able to effectively use the requirements documentation to decide their further course of action?

1

1

1

"How to make more awareness of our incentive programs for people looking to make a clean energy project and ensure seamless access?"

Client Question

Research Questions

Client Goals

Research Questions

Website navigation

Website navigation

Website navigation

How effectively are the users able to navigate to the small and medium business solutions page?


What are the existing barriers hindering the users' access to information of incentive programs?


Are users able to access the commercial retrofit program requirements documentation on the website?

2

2

2

"How to improve the user experience by making that information easily digestible?"

Client Question

Research Questions

Client Goals

Research Questions

Documentation comprehension

Documentation comprehension

Documentation comprehension

Are users able to effectively use the content presented in the commercial retrofit program requirements documentation to determine their eligibility for the program?


How effectively are the specifications about lighting and non-lighting products presented for user comprehension, and what factors contribute to the digestibility?


Are users able to effectively use the content presented in the commercial retrofit program requirements documentation to decide their further course of action?

Methods

Cognitive Walkthrough & Heuristic Evaluation

To explore the potential problem space, we visualized different navigational paths and key screens, and used established design heuristics to evaluate the website.

Interaction map

Demo of heuristic evaluation

Task-based Usability Testing

Participants

Target audience

Seattle-based small or medium business owners with physical commercial spaces

Navigating recruitment challenges

At first, recruiting participants through client connections seemed viable, but we quickly encountered challenges with low response rates and time constraints. To overcome this, we took a proactive approach by visiting the University District, directly engaging business owners to participate in our usability testing. We conducted on-site sessions when possible, scheduled future appointments, or collected contacts for follow-ups. This hands-on strategy not only resolved our recruitment challenges but also reinforced the importance of contingency planning and creative problem-solving.

Participant overview

We recruited 5 local business owners: four from shops in the University District and one via the clients' network. Participants were aged 30-60 and consisted of three males and two females.

Study Design

Testing procedure

We conducted both remote moderated testing and in-person testing, with each session lasting about 30 minutes.

Task 1: Website navigation

The first task was to find relevant incentive programs for the participant’s business on the City of Seattle official website.

navigate to

navigate to

navigate to

Testing goals

Assess how intuitively users can navigate to the incentive information on the website.

Identify any confusing or incorrect navigational choices.

Task 2: Documentation comprehension

The second task was to use the commercial retrofit program requirements documentation to determine the participant’s business eligibility.

Testing goals

Evaluate the clarity of the information in guiding users to their next steps.

Analyze how the document’s layout and language affect user comprehension.

Data Collection & Analysis

Data collection map

We created a table to present the data we gathered and how it aligns with our research questions.

Qualitative data affinity diagram

We utilized affinity diagrams to organize open-ended responses, think-aloud notes, and answers to post-task and post-test questions. This helped us categorize qualitative data into core themes and extract major usability findings.

Quantitative data analysis overview

We analyzed and visualized quantitative data, complementing qualitative insights for deeper interpretation.

Website navigation

Documentation comprehension

Website navigation (number of clicks attempted)

Findings & Recommendations

Severity Rating

We used a rating system developed by Dumas and Redish to evaluate and prioritize the severity of each usability issue, enabling our clients to address the most critical usability barriers first.

  • Level 1 - prevents completion of a task

  • Level 2 - creates significant delay and frustration

  • Level 3 - has a minor effect on usability

  • Level 4 - subtle problem, points to a future enhancement

Key Findings

1

1

1

Unclear Website Navigation and Information Structure

Relevant task

Website navigation

Severity

Level 2

Frequency

· 3 out of 5 participants were unable to find the right entry points

· 2 out of 5 participants reported that the home page menu grouping was unclear

· 3 out of 5 participants did not notice the search bar

Description

Many participants struggled to find the incentive program information on the Seattle City Light website. They suggested that the information and entry points could be more prominent, especially on the homepage and within the navigation menu. Additionally, the search functionality was problematic, with participants noting that the search bar frequently failed to return relevant results.

City of Seattle official website hompage navigation menu

Severity

Level 2

Frequency

  • 3 out of 5 participants were unable to find the right entry points

  • 2 out of 5 participants reported that the homepage menu grouping was unclear

  • 3 out of 5 participants did not notice the search bar

Recommendation

Increase the visibility of the incentive programs by featuring them in the description of a related drop-down menu item in the navigation bar on the Seattle City Light website, as well as on the City of Seattle official website homepage. This would create clearer entry points and make it easier for users to navigate to the incentive program.

Optimize the search functionality to handle typos and provide more relevant results, and make the search bar more noticeable.

2

2

2

Misaligned Audience and Confusing Technical Terms

Relevant task

Documentation comprehension

Description

Participants found the documentation overly technical and difficult to understand, with limited relevance to their expertise as business owners. All five participants indicated they would need a contractor or advisor to review the requirements documentation, regardless of their level of comprehension. Even when they understood some of the material, they still felt it required further clarification.

"As a regular business person, I wouldn't understand all the details in the documentation; I would definitely take this info to consultants or contractors."

Severity

Level 2

Frequency

· 5 out of 5 participants mentioned they would call a contractor or advisor to review this for them

· 4 out of 5 participants said the Commercial Retrofit Program Requirements Documentation was hard to understand

Severity

Level 2

Frequency

  • 5 out of 5 participants mentioned they would need a contractor or advisor to review the documentation for them

  • 4 out of 5 participants said the documentation was hard to understand

Recommendation

Integrate an AI chatbot to assist business owners in navigating the documentation by providing clear, step-by-step explanations of program requirements, eligibility criteria, and next steps.

Since contractors appear to be the primary users of the documentation, further testing with them would help identify areas for improvement and ensure the documentation aligns with its actual users' needs.

3

3

3

Lengthy and Time-Consuming Documentation

Relevant task

Documentation comprehension

Severity

Level 3

Frequency

· 3 out of 5 participants said the Commercial Retrofit Program Requirements Documentation was time-consuming

Description

Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.

Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.

Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.

Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.

Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.

Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.

Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.

Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.

“I didn’t know to look further down to what might qualify.”

Severity

Level 3

Frequency

  • 3 out of 5 participants noted that the documentation was time-consuming

Recommendation

Include a catalog or summary at the start of the documentation, along with a side navigation guide for easier access to key sections. Add links to the lighting and non-lighting requirements, and incorporate a section with bulleted next steps.

Outcomes

Product/External

We identified usability issues and presented evidence-based, actionable recommendations to enhance the accessibility and understandability of the incentive program. These recommendations have driven updates to the Seattle City Light website, resulting in a more favorable user experience and easier access to incentive details.

For example, a table of contents was incorporated into the documentation based on our suggestions to improve navigation.

Before: No table of contents

Current: With table of contents

Process/Internal

The usability study revealed that contractors rather than business owners are the end users, and additional testing is needed to refine the website and documentation further.

Our experience indicated that participant recruitment should begin earlier and be strategically planned to ensure timely involvement for future research.

We showcased effective communication and collaboration practices, setting a strong precedent for how Seattle City can successfully partner with external teams to streamline workflows and improve outcomes.

Reflection

Conducting usability testing with a PDF document required adapting traditional approaches, treating the document as a long web page. This unique scenario challenged me to modify testing methods for unconventional formats.

Collaborating with multiple researchers and interview moderators required consistency in scripting, note-taking, and reporting processes. Composing clear interview scripts became paramount, especially when explaining key terms or guiding participants through various tasks.

Capturing comprehensive user interactions was essential for data analysis and generating accurate insights. I used multiple tools, including screen recorders, voice memos, and Zoom, to record test sessions and minimize the risk of missing key details or encountering technology glitches. Pre-testing devices would ensure smoother test execution and documentation.

A key takeaway was the value of methodological triangulation in uncovering usability issues. Cognitive walkthroughs and heuristic evaluations revealed initial navigation challenges, while usability testing exposed unexpected insights like additional entry points and navigation paths. These findings demonstrated the importance of using diverse research methodologies to holistically examine user experience.