Seattle City Light
Seattle City Light
UX Research - Usability Studies
UX Research - Usability Studies
Breaking Down Barriers to Incentive Program Accessibility
Breaking Down Barriers to Incentive Program Accessibility
Project Scope
Project Scope
Role: UX researcher
Role: UX researcher
Duration: 9 weeks
Duration: 9 weeks
Tools: Figma, FigJam, Google Sheets, Google Meet, Zoom, Voice Memos, Screen Recorder
Tools: Figma, FigJam, Google Sheets, Google Meet, Zoom, Voice Memos, Screen Recorder
Methods: Task-based usability testing, cognitive walkthrough, heuristic evaluation
Methods: Task-based usability testing, cognitive walkthrough, heuristic evaluation
What I Worked on
What I Worked on
Led the planning and execution of the entire project, including developing the study kit, overseeing usability testing, performing data analysis, and composing the final report and presentation.
Conducted one usability testing session as the lead interviewer and facilitated three additional sessions. Effectively analyzed the testing data to generate insights and key findings.
Managed client communication, coordinated team activities, and tracked project progress.
Led the planning and execution of the entire project, including developing the study kit, overseeing usability testing, performing data analysis, and composing the final report and presentation.
Conducted one usability testing session as the lead interviewer and facilitated three additional sessions. Effectively analyzed the testing data to generate insights and key findings.
Managed client communication, coordinated team activities, and tracked project progress.
Who I Worked with
Who I Worked with
I collaborated with three master's students as part of a usability studies course at the University of Washington in Winter 2024. Our team worked as contractors for Seattle City Light and Seattle Office of Economic Development.
The staff from Seattle City Light and Seattle Office of Economic Development provided valuable support throughout our research, offering insights and resources that contributed to the successful outcomes of this project.
I collaborated with three master's students as part of a usability studies course at the University of Washington in Winter 2024. Our team worked as contractors for Seattle City Light and Seattle Office of Economic Development.
The staff from Seattle City Light and Seattle Office of Economic Development provided valuable support throughout our research, offering insights and resources that contributed to the successful outcomes of this project.
Project Overview
Project Overview
This study aimed to evaluate the usability and accessibility of the Seattle City Light website in providing information about incentive programs to small and medium-sized business owners. The usability testing specifically focused on users' experiences navigating to the "Small and Medium Business Solutions" page and understanding the associated "Commercial Retrofit Program Requirements Documentation." We assessed whether the website allowed business owners to easily navigate, access information, and comprehend program requirements when researching incentive offerings. We collaborated with clients, set goals, developed test plans, recruited participants, conducted usability tests, analyzed data, and delivered findings and recommendations.
This study aimed to evaluate the usability and accessibility of the Seattle City Light website in providing information about incentive programs to small and medium-sized business owners. The usability testing specifically focused on users' experiences navigating to the "Small and Medium Business Solutions" page and understanding the associated "Commercial Retrofit Program Requirements Documentation." We assessed whether the website allowed business owners to easily navigate, access information, and comprehend program requirements when researching incentive offerings. We collaborated with clients, set goals, developed test plans, recruited participants, conducted usability tests, analyzed data, and delivered findings and recommendations.
Impact
Impact
Identified six key usability issues on the Seattle City Light website, addressing the accessibility and clarity of the energy incentive program, and delivered actionable recommendations. Presented findings to tens of Seattle City staff, raising awareness of accessibility challenges.
Enhanced our clients' understanding of the target user groups and provided a direction for future research. Streamlined the participant recruitment process for future studies with strategies on optimal timing and effective planning.
Drove evidence-based updates to the website and documentation, improving user experience and inspiring changes for other energy efficiency programs.
Identified six key usability issues on the Seattle City Light website, addressing the accessibility and clarity of the energy incentive program, and delivered actionable recommendations. Presented findings to tens of Seattle City staff, raising awareness of accessibility challenges.
Enhanced our clients' understanding of the target user groups and provided a direction for future research. Streamlined the participant recruitment process for future studies with strategies on optimal timing and effective planning.
Drove evidence-based updates to the website and documentation, improving user experience and inspiring changes for other energy efficiency programs.
Timeline
Timeline
Project Motivation
Project Motivation
Our clients from Seattle City Light and Seattle Office of Economic Development (OED) collaborated to bridge the gap for small and medium-sized businesses to effectively leverage Seattle City Light's incentive programs and achieve their energy goals.
However, it was found that the existing program materials were not easily understandable by most laypeople with a small business, which highlighted the need of enhancing the accessibility and clarity of these resources.
Our clients from Seattle City Light and Seattle Office of Economic Development (OED) collaborated to bridge the gap for small and medium-sized businesses to effectively leverage Seattle City Light's incentive programs and achieve their energy goals.
However, it was found that the existing program materials were not easily understandable by most laypeople with a small business, which highlighted the need of enhancing the accessibility and clarity of these resources.
Transforming Client Questions into Research Questions
Transforming Client Questions into Research Questions
To guide project planning and scoping, we broke down our clients' broad questions into specific, user-centered inquiries to inform the research process. Their goals centered on two aspects: website navigation (how users discover and access program information) and documentation comprehension (how well users understand and interpret the program requirements).
To guide project planning and scoping, we broke down our clients' broad questions into specific, user-centered inquiries to inform the research process. Their goals centered on two aspects: website navigation (how users discover and access program information) and documentation comprehension (how well users understand and interpret the program requirements).
Client Questions
Research Questions
1
"How to make more awareness of our incentive programs for people looking to make a clean energy project and ensure seamless access?"
Website navigation
How effectively are the users able to navigate to the small and medium business solutions page?
What are the existing barriers hindering the users' access to information of incentive programs?
Are users able to access the commercial retrofit program requirements documentation on the website?
2
"How to improve the user experience by making that information easily digestible?"
Documentation comprehension
Are users able to effectively use the content presented in the commercial retrofit program requirements documentation to determine their eligibility for the program?
How effectively are the specifications about lighting and non-lighting products presented for user comprehension, and what factors contribute to the digestibility?
Are users able to effectively use the content presented in the commercial retrofit program requirements documentation to decide their further course of action?
1
1
"How to make more awareness of our incentive programs for people looking to make a clean energy project and ensure seamless access?"
Client Question
Research Questions
Client Goals
Research Questions
Website navigation
Website navigation
How effectively are the users able to navigate to the small and medium business solutions page?
What are the existing barriers hindering the users' access to information of incentive programs?
Are users able to access the commercial retrofit program requirements documentation on the website?
2
2
"How to improve the user experience by making that information easily digestible?"
Client Question
Research Questions
Client Goals
Research Questions
Documentation comprehension
Documentation comprehension
Are users able to effectively use the content presented in the commercial retrofit program requirements documentation to determine their eligibility for the program?
How effectively are the specifications about lighting and non-lighting products presented for user comprehension, and what factors contribute to the digestibility?
Are users able to effectively use the content presented in the commercial retrofit program requirements documentation to decide their further course of action?
Cognitive Walkthrough & Heuristic Evaluation
Cognitive Walkthrough & Heuristic Evaluation
To explore the potential problem space, we visualized different navigational paths and key screens, and used established design heuristics to evaluate the website.
To explore the potential problem space, we visualized different navigational paths and key screens, and used established design heuristics to evaluate the website.
Interaction map
Interaction map
Demo of heuristic evaluation
Demo of heuristic evaluation
Task-based Usability Testing
Task-based Usability Testing
Participants
Participants
Target audience
Target audience
Seattle-based small or medium business owners with physical commercial spaces
Seattle-based small or medium business owners with physical commercial spaces
Navigating recruitment challenges
Navigating recruitment challenges
At first, recruiting participants through client connections seemed viable, but we quickly encountered challenges with low response rates and time constraints. To overcome this, we took a proactive approach by visiting the University District, directly engaging business owners to participate in our usability testing. We conducted on-site sessions when possible, scheduled future appointments, or collected contact details for follow-ups. This hands-on strategy not only resolved our recruitment challenges but also reinforced the importance of contingency planning and creative problem-solving.
At first, recruiting participants through client connections seemed viable, but we quickly encountered challenges with low response rates and time constraints. To overcome this, we took a proactive approach by visiting the University District, directly engaging business owners to participate in our usability testing. We conducted on-site sessions when possible, scheduled future appointments, or collected contact details for follow-ups. This hands-on strategy not only resolved our recruitment challenges but also reinforced the importance of contingency planning and creative problem-solving.
Participant overview
Participant overview
We recruited five local business owners: four from shops in the University District and one via the clients' network. Participants were aged 30-60 and consisted of three males and two females.
We recruited five local business owners: four from shops in the University District and one via the clients' network. Participants were aged 30-60 and consisted of three males and two females.
Study Design
Study Design
Testing procedure
Testing procedure
We conducted both remote moderated testing and in-person testing, with each session lasting about 30 minutes.
We conducted both remote moderated testing and in-person testing, with each session lasting about 30 minutes.
Task 1: Website navigation
Task 1: Website navigation
The first task was to find relevant incentive programs for the participant’s business on the City of Seattle official website.
The first task was to find relevant incentive programs for the participant’s business on the City of Seattle official website.
navigate to
navigate to
Testing goals
Testing goals
Assess how intuitively users can navigate to the incentive information on the website.
Identify any confusing or incorrect navigational choices.
Assess how intuitively users can navigate to the incentive information on the website.
Identify any confusing or incorrect navigational choices.
Task 2: Documentation comprehension
Task 2: Documentation comprehension
The second task was to use the commercial retrofit program requirements documentation to determine the participant’s business eligibility.
The second task was to use the commercial retrofit program requirements documentation to determine the participant’s business eligibility.
Testing goals
Testing goals
Evaluate the clarity of the information in guiding users to their next steps.
Analyze how the document’s layout and language affect user comprehension.
Evaluate the clarity of the information in guiding users to their next steps.
Analyze how the document’s layout and language affect user comprehension.
Data Collection & Analysis
Data Collection & Analysis
Data collection map
Data collection map
We created a table to present the data we gathered and how it aligns with our research questions.
We created a table to present the data we gathered and how it aligns with our research questions.
Qualitative data affinity diagram
Qualitative data affinity diagram
We utilized affinity diagrams to organize open-ended responses, think-aloud notes, and answers to post-task and post-test questions. This helped us categorize qualitative data into core themes and extract major usability findings.
We utilized affinity diagrams to organize open-ended responses, think-aloud notes, and answers to post-task and post-test questions. This helped us categorize qualitative data into core themes and extract major usability findings.
Quantitative data analysis overview
Quantitative data analysis overview
We analyzed and visualized quantitative data, complementing qualitative insights for deeper interpretation.
We analyzed and visualized quantitative data, complementing qualitative insights for deeper interpretation.
Website navigation
Website navigation
Documentation comprehension
Documentation comprehension
Website navigation (number of clicks attempted)
Website navigation (number of clicks attempted)
Findings & Recommendations
Findings & Recommendations
Severity Rating
Severity Rating
We used a rating system developed by Dumas and Redish to evaluate and prioritize the severity of each usability issue, enabling our clients to address the most critical usability barriers first.
· Level 1 - prevents completion of a task
· Level 2 - creates significant delay and frustration
· Level 3 - has a minor effect on usability
· Level 4 - subtle problem, points to a future enhancement
We used a rating system developed by Dumas and Redish to evaluate and prioritize the severity of each usability issue, enabling our clients to address the most critical usability barriers first.
· Level 1 - prevents completion of a task
· Level 2 - creates significant delay and frustration
· Level 3 - has a minor effect on usability
· Level 4 - subtle problem, points to a future enhancement
Key Findings
Key Findings
1
1
Unclear Website Navigation and Information Structure
Unclear Website Navigation and Information Structure
Relevant task
Website navigation
Website navigation
Severity
Level 2
Frequency
· 3 out of 5 participants were unable to find the right entry points
· 2 out of 5 participants reported that the home page menu grouping was unclear
· 3 out of 5 participants did not notice the search bar
Description
Many participants struggled to find the incentive program information on the Seattle City Light website. They suggested that the information and entry points could be more prominent, especially on the homepage and within the navigation menu. Additionally, the search functionality was problematic, with participants noting that the search bar frequently failed to return relevant results.
Many participants struggled to find the incentive program information on the Seattle City Light website. They suggested that the information and entry points could be more prominent, especially on the homepage and within the navigation menu. Additionally, the search functionality was problematic, with participants noting that the search bar frequently failed to return relevant results.
City of Seattle official website hompage navigation menu
City of Seattle official website hompage navigation menu
Severity
Level 2
Frequency
· 3 out of 5 participants were unable to find the right entry points
· 2 out of 5 participants reported that the homepage menu grouping was unclear
· 3 out of 5 participants did not notice the search bar
Recommendation
Increase the visibility of the incentive programs by featuring them in the description of a related drop-down menu item in the navigation bar on the Seattle City Light website, as well as on the City of Seattle official website homepage. This would create clearer entry points and make it easier for users to navigate to the incentive program.
Optimize the search functionality to handle typos and provide more relevant results, and make the search bar more noticeable.
Increase the visibility of the incentive programs by featuring them in the description of a related drop-down menu item in the navigation bar on the Seattle City Light website, as well as on the City of Seattle official website homepage. This would create clearer entry points and make it easier for users to navigate to the incentive program.
Optimize the search functionality to handle typos and provide more relevant results, and make the search bar more noticeable.
2
2
Misaligned Audience and Confusing Technical Terms
Relevant task
Documentation comprehension
Documentation comprehension
Description
Participants found the documentation overly technical and difficult to understand, with limited relevance to their expertise as business owners. All five participants indicated they would need a contractor or advisor to review the requirements documentation, regardless of their level of comprehension. Even when they understood some of the material, they still felt it required further clarification.
"As a regular business person, I wouldn't understand all the details in the documentation; I would definitely take this info to consultants or contractors."
Participants found the documentation overly technical and difficult to understand, with limited relevance to their expertise as business owners. All five participants indicated they would need a contractor or advisor to review the requirements documentation, regardless of their level of comprehension. Even when they understood some of the material, they still felt it required further clarification.
"As a regular business person, I wouldn't understand all the details in the documentation; I would definitely take this info to consultants or contractors."
Severity
Level 2
Frequency
· 5 out of 5 participants mentioned they would call a contractor or advisor to review this for them
· 4 out of 5 participants said the Commercial Retrofit Program Requirements Documentation was hard to understand
Severity
Level 2
Frequency
· 5 out of 5 participants mentioned they would need a contractor or advisor to review the documentation for them
· 4 out of 5 participants said the documentation was hard to understand
Recommendation
Integrate an AI chatbot to assist business owners in navigating the documentation by providing clear, step-by-step explanations of program requirements, eligibility criteria, and next steps.
Since contractors appear to be the primary users of the documentation, further testing with them would help identify areas for improvement and ensure the documentation aligns with its actual users' needs.
Integrate an AI chatbot to assist business owners in navigating the documentation by providing clear, step-by-step explanations of program requirements, eligibility criteria, and next steps.
Since contractors appear to be the primary users of the documentation, further testing with them would help identify areas for improvement and ensure the documentation aligns with its actual users' needs.
3
3
Lengthy and Time-Consuming Documentation
Relevant task
Documentation comprehension
Documentation comprehension
Severity
Level 3
Frequency
· 3 out of 5 participants said the Commercial Retrofit Program Requirements Documentation was time-consuming
Description
Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.
“I didn’t know to look further down to what might qualify.”
Participants emphasized that small businesses often have only one person handling all responsibilities, making it necessary for documentation to be clear and easy to navigate. They found it challenging to sift through lengthy documents and preferred having direct links to different requirements at the beginning.
“I didn’t know to look further down to what might qualify.”
Severity
Level 3
Frequency
· 3 out of 5 participants noted that the documentation was time-consuming
Recommendation
Include a catalog or summary at the start of the documentation, along with a side navigation guide for easier access to key sections. Add links to the lighting and non-lighting requirements, and incorporate a section with bulleted next steps.
Include a catalog or summary at the start of the documentation, along with a side navigation guide for easier access to key sections. Add links to the lighting and non-lighting requirements, and incorporate a section with bulleted next steps.
Outcomes
Outcomes
Product/External
Product/External
We identified usability issues and presented evidence-based, actionable recommendations to enhance the accessibility and understandability of the incentive program. These recommendations have driven updates to the Seattle City Light website, resulting in a more favorable user experience and easier access to incentive details.
For example, a table of contents was incorporated into the documentation based on our suggestions to improve navigation.
We identified usability issues and presented evidence-based, actionable recommendations to enhance the accessibility and understandability of the incentive program. These recommendations have driven updates to the Seattle City Light website, resulting in a more favorable user experience and easier access to incentive details.
For example, a table of contents was incorporated into the documentation based on our suggestions to improve navigation.
Before: No table of contents
Before: No table of contents
Before: No table of contents
Before: No table of contents
Before: No table of contents
Current: With table of contents
Current: With table of contents
Current: With table of contents
Current: With table of contents
Current: With table of contents
Process/Internal
Process/Internal
The usability study revealed that contractors rather than business owners are the end users, and additional testing is needed to refine the website and documentation further.
Our experience indicated that participant recruitment should begin earlier and be strategically planned to ensure timely involvement for future research.
We showcased effective communication and collaboration practices, setting a strong precedent for how Seattle City can successfully partner with external teams to streamline workflows and improve outcomes.
The usability study revealed that contractors rather than business owners are the end users, and additional testing is needed to refine the website and documentation further.
Our experience indicated that participant recruitment should begin earlier and be strategically planned to ensure timely involvement for future research.
We showcased effective communication and collaboration practices, setting a strong precedent for how Seattle City can successfully partner with external teams to streamline workflows and improve outcomes.
Reflection
Reflection
Conducting usability testing with a PDF document required adapting traditional approaches, treating the document as a long web page. This unique scenario challenged me to modify testing methods for unconventional formats.
Collaborating with multiple researchers and interview moderators required consistency in scripting, note-taking, and reporting processes. Composing clear interview scripts became paramount, especially when explaining key terms or guiding participants through various tasks.
Capturing comprehensive user interactions was essential for data analysis and generating accurate insights. I used multiple tools, including screen recorders, voice memos, and Zoom, to record test sessions and minimize the risk of missing key details or encountering technology glitches. Pre-testing devices would ensure smoother test execution and documentation.
A key takeaway was the value of methodological triangulation in uncovering usability issues. Cognitive walkthroughs and heuristic evaluations revealed initial navigation challenges, while usability testing exposed unexpected insights like additional entry points and navigation paths. These findings demonstrated the importance of using diverse research methodologies to holistically examine user experience.
Conducting usability testing with a PDF document required adapting traditional approaches, treating the document as a long web page. This unique scenario challenged me to modify testing methods for unconventional formats.
Collaborating with multiple researchers and interview moderators required consistency in scripting, note-taking, and reporting processes. Composing clear interview scripts became paramount, especially when explaining key terms or guiding participants through various tasks.
Capturing comprehensive user interactions was essential for data analysis and generating accurate insights. I used multiple tools, including screen recorders, voice memos, and Zoom, to record test sessions and minimize the risk of missing key details or encountering technology glitches. Pre-testing devices would ensure smoother test execution and documentation.
A key takeaway was the value of methodological triangulation in uncovering usability issues. Cognitive walkthroughs and heuristic evaluations revealed initial navigation challenges, while usability testing exposed unexpected insights like additional entry points and navigation paths. These findings demonstrated the importance of using diverse research methodologies to holistically examine user experience.