This case study is password protected
Enter the password to view.
Data Forwarder
Enabling Users to Forward Data Downstream
The Problem
Due to rate limiting constraints and other shortfalls with APIs, our users needed a more durable solution to pipe massive amounts of security event data out of Carbon Black and into other tools for aggregation and automation, such as SIEM and SOAR tools that help collate information across multiple tools and automate remediation actions.
As this functionality came available, users struggled to set up forwarders, first via API (because that was all that was available), but they also needed a way to create forwarders from within the product itself.
The Goal
We broke out the project into phases to release functionality over several iterations. This allowed us to unblock users sooner, but continue to make it easier and more flexible over time.
- Explain clearly how to set up a forwarder via the existing API in one consolidated quick start guide
- Create an interface so non-developer users can also set up forwarders from within the product
- Provide the ability to reduce the amount of data with granular filtering
- Allow users to forward any type of data that exists in the product
- Let users control when they upgrade to the latest data schema to avoid breaking active pipelines and integrations
- Provide more destination options for relaying the data beyond AWS
My Process
Iteration 1: Consolidated API Instructional Guide
- Led user research activities to identify problems in the current workflow
- Collected feedback via surveys to understand customer complaints
- Performed a heuristic analysis of the end-to-end workflow for setting up a forwarder via the API
- Mapped out the full user journey to identify points of failure in using the API instructional guides
- Too many guides forced users to flip back and forth between different pieces of documentation, making it easy to lose their place
- Dense (and sometimes inconsistent) jargon and terminology
- Lack of examples forcing users to guess how to achieve complicated tasks
- Wrote a new instructional guide, providing one end-to-end guide with an accompanying video
- Consolidated all of the steps into a single guide to avoid forcing users to navigate to external documents
- Outlined the steps up front so users could skip ahead to the pertinent topics
- Aligned the language to the content style guide, eliminating phrases like "a customer should…"
- Provided extra guidance in how to complete certain steps
- Included example inputs based on a sample scenario to show how answers to inputs should be formatted
- Provided screenshots to show what a user should see at certain steps in the process
- Provided a copy-and-paste-able example AWS S3 bucket policy so users didn't have to write it from scratch
User Journey Mapping
Diagrammed all the steps of setting up a data forwarder, noted the disparate locations of all the existing documentation, and highlighted the biggest pain points to explain to and convince engineers and management why we needed to make certain changes.
Documentation Design
Incorporated user feedback in consolidating multiple guides into one document, linking to high-level topics for easier navigation, used predictable and searchable naming conventions, followed a common document template for familiarity and clarity.
Code Snippets
A top request from users in response to the feedback survey was to include more examples and code snippets to not just tell, but show how to complete certain tasks. I turned images into copy-and-paste-able examples and illustrated a scenario of how one might complete a given task.
Iteration 2: Ability to Create a Forwarder In the UI
The biggest barrier to entry for users was that they weren't all comfortable completing the complicated workflow via API. The goal for this phase was to take the existing workflow for creating a forwarder and give it a graphic user interface to perform the same tasks.
- Taking the steps from the API journey map, I plugged the steps into wireframes
- I explored a few different progressive disclosure strategies to break down the complex workflow into achievable steps
- I veered away from a wizard concept to a form that grew as the user made determinations
Quick Wireframes
I started out with low-fidelity wireframes so I could get feedback quickly before going too far down a given direction. I led UX workshops to gather feedback from product management, engineering, and the broader UX team, which helped identify existing form patterns I could leverage.
High-Fidelity Wireframes
I modeled workflows for adding a forwarder, editing or deleting an existing forwarder, reviewing existing forwarders and details about how they are configured, and turning the flow of data on and off — including error and success messaging, links to documentation, tooltips, and helper text.
Design System
My goal was to create a familiar interface out of existing design system components so users would already know how to use the page controls. Using existing components sped up the UI engineering process and helped avoid creating bugs since they were tried-and-true controls.
Iteration 3: Advanced Filtering Capabilities
Once the base functionality was completed, we could focus our efforts on secondary user problems. A major hurdle for our customers was getting the right data downstream. Originally, they could choose to either get a small set of alert data, or a huge amount of more granular event data. The problem with alert data was that threats could be hiding in a user's organization that weren't being detected yet by alerts. The problem with event data was that it could cost tens of millions of dollars to pull all of it into SIEMs like Splunk, and 90% of it might not even be helpful. Our goal with this phase was to provide filtering options so users could pick and choose which data they wanted to include or exclude from their forwarder.
- Worked closely with product management who helped gather user needs from various users on what data they wanted to forward or not forward
- Modeled out various customer scenarios to identify opportunities and guide us to a direction that would meet users' needs
During my wireframing process, I had an idea for how to offer this new granular filtering capability while also maintaining the more basic filtering functionality. My hypothesis was that some users would be excited to write custom queries and tune exactly what data they wanted to include or exclude, but that not all users were comfortable writing queries. The product manager and engineering lead did not agree, and assumed all forwarder users would know how to write queries — they explained that maintaining the basic filtering functionality would increase engineering effort substantially.
- I worked with a user researcher to put together a study to get feedback directly from customers on whether the basic filtering experience would continue to benefit them
- Research proved that offering both a basic and advanced experience was preferred — even advanced users liked the option to sometimes use basic filters, and explained scenarios when more junior colleagues might benefit from the original functionality
- From our research findings, engineers were convinced that we needed both experiences, and agreed to build in the flexibility so users of different skill levels could successfully complete the task
Digital Whiteboarding
This phase of the project began at the beginning of the pandemic, and I led virtual whiteboarding workshops using Miro to collaboratively sketch and ideate. This benefitted me in that I learned a lot from the product managers about what data was important to users, and it helped the product managers learn about my process and see why certain ideas were not going to work.
Criteria Builder
The original "basic" filters allowed users to click and build out their criteria. I researched existing search and filter interfaces, especially from library websites, to put together this concept, and merged in styling from other parts of the product that included criteria builders.
Query Guidance & Repurposing Components
The user research study helped me identify what types of guidance were needed to successfully write queries, and clear up flaws in the design. I repurposed an existing search bar component that validated query logic to inform users when their queries were correct or incorrect and offer suggested terms and syntax — which also sped up engineering since it was complicated logic they didn't have to build from scratch.
Iteration 4: Ability to Forward More Types of Data
One of the benefits of the data forwarder was that it could send larger volumes of data than our APIs, which were restricted by rate limiting and less stable. The problem was that we only offered Alert or Endpoint Event data via the forwarder, but there were many other data streams in our product. This phase unlocked the ability to forward two more streams of data.
- By adding the ability to forward Watchlist Hits, users could create custom detections and send them downstream to their SIEM tools
- The addition of Authentication Event data meant users could forward data out of Windows machines and preserve it longer than 3 days
Extensible Design
Knowing about the full product roadmap up front meant this phase was pretty simple from a design standpoint — I knew we might need to add more data types, so the initial design allowed for the list of data types to grow and evolve over time.
Iteration 5: Let Users Control When They Upgrade Their Schema Version
As our engineering teams added to and improved the backend schema for the data, we found that backend changes were incredibly disruptive to some of our users — sometimes breaking their integration apps and disrupting the flow of data to their downstream solutions. My aim was to provide a way to let these users choose when they received the backend changes so they wouldn't miss indications of cyber threats and attacks.
The risk of adding this capability was that the process was already complicated, and I didn't want to make this additional step painful for all users if their integrations weren't impacted. Another consideration was how to balance technical complexity and clarity in the content — technical users needed exact terminology, but less-technical users might be confused by jargon. I worked closely with a content designer to strike the right balance, and with a user researcher to run a study to validate the direction.
Defaults and Recommendations
To avoid overcomplicating this for users who weren't impacted by backend updates, I applied a default value for the Schema Updates input so the option was there for users who needed it, but didn't increase the cognitive burden for those who didn't. I also included progressive disclosure so users didn't need to choose a specific schema version unless they changed the schema updates setting.
Microcopy and Guidance
Users could select how they wanted to receive their updates. We defaulted users to receive all minor updates automatically, labelled as the recommended setting. If the options in the dropdown weren't clear enough, they could use the question mark icon to get a clearer description of each option.
Communication and Collaboration
I worked as an ambassador between product management, content design, engineering, and users to work through complex content challenges — gathering research on release rings and constraints, talking with engineers about feasibility, and helping a new content designer onboard mid-project. I focused attention on the research findings to bring everyone to a compromise that would help users decide with as little friction as possible.
Iteration 6: Ability to Forward to New Destination Types
While the ability to forward data out of the Carbon Black Cloud console was great for SIEM and SOAR users, a common complaint was that not all of our customers were able to use Amazon's AWS S3 Data Lake as a destination. Some users found it cost prohibitive, and some were competitors of Amazon, so we needed to add more destination options, like Microsoft Azure, to let more customers benefit from our forwarder capabilities.
I added a new option for selecting what type of destination a user wanted, then adjusted what types of inputs showed for each destination type using progressive disclosure. AWS required an S3 bucket address, while Azure required storage blob identifiers.
Appropriate Labelling
While the UI updates to include this change were not huge compared to previous iterations, it was important to focus on the details and get the wording exactly right. I researched Azure Blob Storage to understand how it compared and contrasted from the AWS S3 destination provider and advised on how to portray this new option.
Challenges
- Designing in a highly technical subject matter, understanding all the terminology, and making it clear to users
- Making a very complicated process and form easy to parse and navigate — organizing the form into clear and logical sections, using progressive disclosure to keep the form brief when possible
- Designing in phases, getting de- and re-prioritized, dusting off old wireframes and getting them finalized at a moment's notice
- Pivoting the design on the fly once the backend was built and no longer matched the initial design, plus scope creep creating new challenges that needed fast solutions
- Convincing engineering teams it's worth the effort to build it a harder way in order to support less technically savvy users
- Working with volatile teammates and others who didn't understand or value the UX perspective until hearing directly from users about the added benefit
Reaction from a Skeptical Product Manager — Spark Greatness Award
"Lisa is a tenacious, detail-oriented, collaborative partner to Eng & PM. Lisa continues to pursue effective, elegant and consistent customer experiences — in UI and content both — that presents what the user needs to know and act on, to perform complex tasks such as configuring a Data Forwarder. Lisa's open to feedback and acts to improve on what came before, whether by someone else's hands or her own. And Lisa is collaborative in working with her partners outside UX in a way that gives where necessary and works up new approaches when none of us are satisfied.
I say this in the best possible interpretation: Lisa's a pain in my butt, and that has consistently meant better outcomes for our customers than had I been working without her. We clash at times, but professionally (well, for sure in Lisa's case — I won't overestimate my behaviour) and in ways that keeps the focus on the evidence and good design principles we need to honour. I wouldn't have it any other way."
— Mike