Highlights
- Development of a holistic feedback strategy and mechanism; incorporating various methods for active and passive feedback collection.
- Design and implementation of a recurring feedback survey, generating an insights report to track the state of user experience.
- Implementation of a feedback widget and feedback form in the Help Centre.
- Design of governance processes to manage and respond to feedback, including a feedback repository.
Methodology
Discover and Define
Design and Iterate
Context: What, how and why
The nsw.gov.au runs a weekly development cycle to resolve bugs, add new features, and improve the user experience. It is critical to the product success that the users experience and satisfaction is considered, measured and improved over time. The user base, platform maintenance costs, and the cost to users continues to increase, and with that so do the expectations from the user.
An audit uncovered that there is no mechanism to collect feedback, affecting how we measure satisfaction and success. To combat this, I developed and implemented a holistic feedback strategy, and multiple feedback collection points in order to better understand user needs/pain points, enable user satisfaction measurement, and allow user feedback to inform the prioritisation and development of the nsw.gov.au product.
Problems
An audit revealed risks in feedback and product development, primarily due to user frustration with slow feature delivery and the lack of a formal feedback channel. Feedback was scattered across various channels, leading to unaddressed concerns and increasing user frustration, which threatened program funding.
1
No channels for users to provide feedback contributing to growing user frustration.
2
No processes to govern and manage feedback that is received.
3
No formal mechanism to measure agency sentiment and gather feedback.
4
Product development and prioritisation did not consider user feedback.
Solutions
To resolve and minimise the negative impact of the identified problems; multiple solutions were developed, tested and implemented.
1
Design and implementation of multiple feedback collection points to give a voice to users.
2
Feedback strategy developed to guide internal users on how to consider and manage feedback.
3
Reporting tools and automated surveys to collect feedback at critical points.
4
Guidance to ensure user input is considered during roadmap planning.
Understanding the current state
I conducted generative research with users and stakeholders to better understand needs, pain points, objectives and overall journey experience.
Stakeholder Interviews
Interviews with key stakeholders, including the product team, customer support, and marketing, to understand their goals and expectations for feedback.
User Interviews
To gauge current sentiment towards the program, how users feel about the topic of feedback and gauge sentiment.
Analysis of Current State and Competitors
To identify the most effective and efficient ways to collect and utilise feedback and ensure that the feedback mechanism delivers meaningful insights aligned with business needs and strategic goals.
To help define project objectives
Findings from generative research were collated and synthesised into How Might We statements. This ensured that all design decisions are human-centerd and based on real quantitative/qualitative user data and scenarios.
How might we...
Encourage users to provide feedback at various touchpoints.
Ensure the process is seamless and non-intrusive.
Categorise feedback to make it actionable for the product team.
And guide solution exploration
Findings from generative research were collated and synthesised into actionable insights, ensuring all design decisions are human-centered and based on real quantitative/qualitative user data and scenarios.
User Journey Mapping
Mapped out the user journey to identify the optimal/critical points for soliciting feedback collection.
These included onboarding, post-transaction, and after significant updates.
Feedback Channels
We decided to implement multiple feedback channels to maximise user engagement; In-App Surveys,
Feedback Widgets, and Email Surveys.
Wireframes and Prototypes
Wireframes and interactive prototypes for the feedback touch points were tested to ensure they were intuitive and integrated seamlessly into the user experience.
Setting up for success
Pre-implementation activities were conducted to ensure a successful and smooth launch, this included:
Development Collaboration
Working closely with the development team to integrate the feedback mechanisms into the app: Embedding in-app surveys and feedback widgets; Setting up automated email survey triggers; Ensuring data collected was securely stored and easily accessible for analysis.
Pilot Testing
Before full-scale launch, a pilot test was conducted with a small user group to identify any technical issues and ensure the feedback mechanism was functioning as expected.
Data Collection, Analysis and Reporting
Set up a system for regular reporting to the product team, highlighting user sentiments, common pain points, and suggestions for improvement.
Formal active feedback collection with surveys
A recurring survey was designed to actively collect feedback at critical junctions of the user’s journey.
Audiences were segmented, with skip logic being used to capture relevant responses dependant on user role. For example, CMS editors were asked questions focussed on their CMS experience and support they receive; senior staff were asked about their sentiment towards the program, costs and value.
From these surveys, I was able to generated insights reports highlighting themes, and track sentiment across user groups.
This was built using SurveyMonkey and leveraged Archetype work
Informal passive feedback collection with forms and chat
Creating a safe space to encourage feedback
Passive customer feedback is equally valuable and authentic since it also captures implicit feedback. Alongside the surveys, multiple passive feedback mechanisms were design and implemented to gather relevant and focussed feedback without the risk of inducing survey fatigue.
A widget was deployed within the Help Centre, and Feedback forms were developed with links from the CMS directing users to the form, providing a more complete picture of the customer journey.
This was built using Zendesk Chat
Implementing a robust feedback mechanism proved crucial in understanding and improving the user experience of nsw.gov.au. By continuously gathering and analysing user feedback, we were able to make informed decisions that enhanced user satisfaction, reduced churn, and drove the product’s success.
This case study highlights the importance of integrating user feedback into the product design process to create a user-centric product that evolves based on real user needs and experiences.
Informed Product Roadmap
The feedback provided clear guidance for the product roadmap, ensuring that future developments were aligned with user needs and expectations
Tracking of User Sentiment
The surveys provided us with the ability to analyse and track user sentiment over time, and look into specific user experiences comparatively.
Improved Connection with Users
By connecting with our users, we are signalling that we are open and listening.
Qualitative feedback indicated an improvement in trust and connection with our users. Our user base also continued to provide more feedback, observing a 10% increase in the response rate between surveys.
New insights to the user journey
The surveys and subsequent interviews enabled us with new insight into previously hidden steps of their journey through the migration onto the platform, allowing us to map out the journey more accurately and address points of friction.
Other work
Feedback mechanism
CMS Dashboard design
