Table of Contents
Welcome to The Founder Loop, a behind-the-scenes interview series where we sit down with Ryan Millner, co-founder and CEO of Unwrap, to explore the thinking, challenges, and technical decisions that shape our product and processes.
From handling customer data to leveraging the latest in AI, we dig into the how and why behind Unwrap.
In this first installment, we’re tackling one of the most critical aspects of working with user data—personally identifiable information (PII) redaction. Ryan discusses how we approach PII, especially in the context of LLMs, and why, with Unwrap, security doesn't have to come at the cost of utility.
How does Unwrap identify and redact PII across customer data?
Ryan Millner: “So Unwrap uses one of the industry-leading PII redaction tool, called AWS Comprehend. It’s incredibly accurate and has best-in-class precision around identifying and removing PII.
Plus, it has both out-of-the-box entity detection and the ability for customization. The first piece redacts known entity types like first name, last name, email, etc. But, you also can suggest your own patterns to look for—like a user ID or an airline rewards number—and redact that as well.”
Can you explain how redaction affects LLM performance, especially when generating insights or customer feedback summaries?
Millner: “I think this depends on the use case, but I can speak to how it affects Unwrap specifically.
Unwrap analyzes customer feedback, categorizes it to very granular levels, and then surfaces insights from that analysis. The good thing is that we don't need PI to actually perform the analysis.
If we know, for example, an email address or a name of someone who submitted the feedback, it doesn't actually impact the output of our results. Say you're trying to figure out how many people are having issues resetting their password, you don't need any PII in there to understand that at an accurate level. We're able to remove PII and not have any adverse effect on the accuracy or the precision of our models. Again, other use cases may differ, but for us, there's no dip in performance.
If desired, keeping PII does allow our customers to more easily chase down specific users who were affected by a specific issue, but each customer can make that choice based on tradeoffs for their use case.”
For a platform like Unwrap, what are the pros and cons of PII redaction?
Millner: “The pros are that typically it may just be a requirement from a company’s InfoSec team. They may say, ‘We can’t have any PII ingested by third-party platforms.’ For a hard requirement that you have to comply with, it’s good we have the processes in place to do so.
As a con, it’s harder to follow up directly with customers. Say you are trying to find all of the people affected by a given issue, because you want to send them a message letting them know it’s resolved. You’d need their email addresses to know who to follow up with—if that information is in Unwrap, it makes it really simple to do that follow up. If not, it’s obviously more tricky.
There are certainly ways around that. For example, could pass us a user ID that we don't know the mappings of names and email addresses too, and then you could do the mapping on your end.”
How does Unwrap handle edge cases, like when redaction removes business critical context?
Millner: “Fortunately because of the service we provide this just doesn't happen. Going back to what I said initially, PII does not affect the analysis work our platform does. Unwrap looks for trends in customer feedback, and you don't need PII to do that effectively.
If you remove a customer’s email address or name, you're still able to understand what their pain point was and why they reached out.”
How do customers know what’s being redacted?
Millner: “It's very clear in the platform. The PII is removed and then it’s replaced with the entity type that was redacted.
Let’s say the original feedback entry was, ‘Hey, I'm Ryan, and I'd like to know more about your student discounts.’ The redacted version in Unwrap would say, ‘Hey, I am <REDACTED FIRST NAME> and I'd like to know more about your student discounts.’ So you'll see it very clearly in text.”
When does PII redaction happen and does Unwrap have access to unredacted information?
Millner: “PII redaction happens before any data is actually ingested into our system. This means, we never gain access to the unredacted information. Customer data goes straight to AWS Comprehend, and then AWS Comprehend passes us the redacted content.”
Unwrap: Is there a scenario where an Unwrap customer may request later on that they want PII redaction to occur? If so, how does Unwrap handle that after the fact?
Millner: “It's a configuration that's easy to adjust. This has actually happened with one of our customers. They started without PII redaction, then a year later said, ‘We actually need it.’
We’re able to simply turn it on and AWS Comprehend scrubs all the PII—retroactively as well. Moving forward, all PII gets redacted on a daily basis.”
