Docusign’s CEO on the dangers of trusting AI to read, and write, your contracts

Today, I’m talking with Allan Thygesen, who is the CEO of Docusign. You know Docusign; it’s the platform where you sign things online. It turns out 7,000 people work there, which is one of those facts you see flying around sometimes that’s always felt like perfect Decoder bait. What are all those people doing? And what kind of product roadmap does a company like Docusign even need?

I always assumed I would never find out, because most enterprise software CEOs do not like being on Decoder. That’s because most enterprise software is bad, and they often don’t actually use their own products, which means they have a hard time answering my questions. So I was pretty happy when Allan agreed to come on — and then told me he had actually used Docusign himself just that morning. 

From there we talked about what Docusign’s platform actually is, how it’s expanding, and, of course, how all of those employees are structured. Allan has been CEO there for just three years, so he had an interesting perspective on where the company was and the changes he wanted to make when he joined Docusign from Google. 

Of course, that brought us to AI. Allan and I spent a long time talking about the idea that Docusign should summarize contracts for people before they sign them and who is responsible if the AI gets that interpretation wrong. We also spent a while talking about how Docusign’s customers actually generate the kinds of documents that get signed and how automating that process with AI does and does not work. You’ll hear Allan point out that a lot of this looks like just a fancy mail merge, which was at least refreshingly down to earth in the context of an AI conversation.

I also had to ask Allan which parts of his enterprise software were bad and how he’d improve them — he actually answered the question, which might be a first for an enterprise software CEO. This is a good one; there’s even a couple dunks on Google in the mix, as you’ll hear.

Okay: Allan Thygesen, CEO of Docusign. Here we go.

This interview has been lightly edited for length and clarity.

Allan Thygesen, you are the CEO of Docusign. Welcome to Decoder.

Thank you, Nilay. Good to be here.

I’m excited to talk to you. I always love it when an enterprise software CEO comes on the show. Most of your competitors in enterprise software are very reticent to show up and answer the first question: When was the last time you used your end product?

This morning.

What did you sign? Can you say?

I signed an agreement for our procurement team, but yesterday I signed your appearance release form.

Very good. I think it’s the most important question for enterprise software CEOs because on the whole, the experience of using enterprise software is, I would say, pretty dicey.

It’s not great. I agree.

And Docusign is a weird one. It’s in the background of everything all the time. Most people don’t use it. I think a lot of lawyers and maybe procurement teams use the product. Most people sort of experience Docusign. How do you think about that?

I mean, it is sort of a two-sided network, right? So, we sell to businesses and other organizations that want to prepare and send documents for signature. That’s historically been our business. Then, consumers or other companies sign documents. They’re the counterparty. But they don’t pay us anything. So, most people have experienced us that way. If you work at a company or organization that uses Docusign, you may have experienced being on the sending side of approvals or other functionality.

That business is really interesting to me. Docusign as a company is interesting, most notably because it’s been an independent company for 20-some years. It’s a public company. From what I experience — again, as somebody who just experiences Docusign — it’s just there. It’s just in the background of almost everything I do. 

But it’s shocking to me that it hasn’t been acquired or people haven’t tried to acquire Docusign. The product range hasn’t dramatically expanded in other ways.

It’s expanding now.

We’ll come to that. And it’s in the way that everyone’s expanding, right? Now the chatbots are going to do all the work for us. But how do you think about the fact that it has been effectively one product experience all this time? What has made it resilient in that way?

You’re right. We are a little over 20 years old. The original idea was to help companies and individuals sign agreements online. At the beginning, no one thought that was a good idea. Not regulators, not companies, not consumers. So, there was a tremendous amount of trust building across those three constituencies that happened over time. But as you said, we became trusted for a variety of transactions over time. Now, it’s in the water across companies of all sizes and functions.

Look, we did come up with a great idea, and when you have a good idea, you want to run with it. I think Docusign has done a really good job of that. With that said, part of my goal joining about three years ago was to say, “Okay, that’s an amazing foundation. We’re privileged that people have positive associations, and we’re used by 1.8 million companies. What do we do for an encore? How do we broaden the platform?”

We’ve had some ideas on that for a while, thinking about the entire agreement journey, but we’d never really put it all together. We’d never gone beyond the marketing side of identifying that problem to actually delivering solutions that could solve it. I think that’s what we’re doing now. So, that’s very exciting. But the signature piece is a foundation. That is the most pivotal moment in the life cycle of an agreement. It’s a very high value, high stress moment for many people. Making that simple and delightful was a very meaningful value proposition. And it’s a lot harder than it looks.

Again, I’ve “signed” Docusigns. My signature in Docusign has no relationship to my actual signature. Docusign just generates some cursive that says my name. What is a signature? In Docusign’s worldview, what is a signature?

A signature to me is where identity and consent commingle. With the identity aspect, there are a lot of things we do to identify you. Obviously, this thing comes to you in your email, via SMS, or WhatsApp. We do all kinds of IP tracing and other things to validate that you are the intended recipient if you’re the signer. That whole trail is then auditable and can be used in a court of law. It’s a perfect substitute for a wet signature that you might otherwise have used in practically all cases. There are a few states in the US that still require you to sign something in person, but we’ve done a pretty good job over time getting to a place where that’s felt super secure. 

From the consent perspective, you’re right. It could be a dot, it could be a checkbox, could be a signature. It’s almost less important. There are some personal expression aspects, but it’s the identity piece that’s the most important, along with the fact that you take a step that indicates consent.

Just to make that as reductive and simple as possible, it sounds like the service that you’re providing as Docusign is, “You know who I am. You know who the other part of the agreement is.” I hit the button and your database says, “This person that we verified hit that button and they say they agree.” Then, if the contract comes into question, you can say, “Well, you definitely signed it. Maybe you disagree on the terms, but you definitely signed the contract. Let’s argue about that.” That implies fundamentally that a product is just a big database of identities and then a marking of consent. In the most reductive way possible, is that how you think about it?

That’s the aspect on the execution side. There’s all the stuff that leads up to preparing a document for execution. We have a lot of products that you as a signer would not see, but that companies use to get documents ready for signing, and to get them approved internally to maybe customize them for you. Let’s say I’m running a big sales team — let’s say the Docusign sales team — and I want to send that agreement to Decoder so you can use Docusign for your releases. I can automatically mass customize a standard template using Docusign’s functionality that’s embedded in Salesforce and other CRMs.

That’s an example of the type of workflows that we do. We do that for hiring, for procurement, for new vendor onboarding, all those tools that lead up to that magical signature moment where it is, I think, about identity and consent.

Even that part where you say, “Okay, we’re going to generate a bunch of documents,” is downstream of the first product, which is signing. The reason I ask about it in that reductive way is because there are a lot of ways you could verify identity and mark consent. You have a lot of competitors there.

We do that.

Is Docusign’s dominance and market leadership based on just network effect? It exists, it’s the one you can use, it’s won over your trust, and because so many people have already used it, it’s the easiest thing to use?

I do think that the network effect is very important. People choose Docusign in part because they know that the recipients — consumers mostly — trust it. Maybe if you’re a giant bank and you have an existing relationship with a customer and you’re sending something to them, that matters a little less. But for most companies in the world and most situations, like all new customer onboarding, that trust component is super important. 

Now, to the other thing that was implied in your question: what about identity and these alternative technologies for establishing identity? We could sort of see that coming, We worked on a federated identity strategy to give you all kinds of identity validation at different levels of risk. We let people do knowledge-based authentication. That’s sort of on its way out, but that’s answering those questions that you would probably remember. Where did you live on the street 10 years ago?

We do biometric-based identification like video or we can link up with the Apple and Google stuff. We do risk-based assessments so we can fork you to validation mechanisms. We use the new digital IDs. So in the US, that’s Clear or ID.me, which is the government one that the IRS uses. In Europe, many governments have a national digital ID, along with many other parts of the world. We go all the way up to a notary solution with online notaries. So whatever your risk assessment, whatever you feel is the appropriate trade-off between convenience and security, we can give you all those solutions turnkey in one platform. Then, at the end, most of the time, you’re also legally required to give a signature.

When you think about that journey, who are your biggest customers? Is it just big banks? Is it mom-and-pops? How does that break down for you?

We’re incredibly diversified. We’re used by, as I mentioned, 1.8 million companies. If you have that many customers, most of them are going to be small. But we are used by over 95 percent of the Fortune 500 and equivalent in other international markets, many mid-sized companies, and then a huge long tail of small companies. And no one company represents any meaningful share of our business. That said, our biggest customers tend to be banks or other large companies that have high volumes of high value agreements. If you have agreements that matter and you’re entering them in large volumes, you’re very likely to be a Docusign customer.

Is that the push for growth? I do want to talk about your expansion into other kinds of products and services. Is part of that that you’re just out of big customers? There’s no more fish in the sea?

We already have most large companies in the world as customers, but of course there’s so much more we can do for them beyond the signing piece. And we’re certainly not done with the signing piece. I mean, you go and audit the large banks who’ve been with us for over a decade on how many of their agreements are actually automated, digitized, and executed electronically, and it’s 20 or 30 percent because people take the biggest high-value workflows and everything else is left to the side. So, we have a lot of opportunity. If we can make that more efficient and easy to do, we can close that gap, as well as provide broader value in all other aspects of the agreement cycle.

Talk to me about that. There’s identity and signing, which I was very curious about. I don’t think most people ever spend time with the notion that a signature represents many complex concepts underneath. That’s the first part.

Then there’s the part where, okay, we’re generating documents for signature. We might as well help draft the documents, prepare them and get them approved for that final step. I want to understand that a little bit. Then, there’s obviously your expansion into AI and other tools and parts of the workflow. At what point do you run into Microsoft Word? Where is the boundary of some of that work, and where are the boundaries of some of those opportunities?

We ran into Microsoft Word the day we got started, right? Microsoft Word is where 90 percent of legal documents get authored. That’s the tool most lawyers use. That’s where documents are crafted and often negotiated, redlined, etc. So, we’ve always been deeply integrated with Word and, of course, with Google as well. Frankly, with all of the tools that people use to do their jobs. I mean, we’ve had a 20-year partnership with Salesforce. We were the first or second vendor on the app exchange and deeply embedded in their flows. We do similar work with Workday, SAP, and so on. 

I don’t think that we or Microsoft have ever thought about competing with each other. I worked at Google for 12 years before taking this job,so a similar scale company, and you take on the biggest, broadest opportunities in your office suites. Yeah, you might add more and more features to your office suite, but you don’t want to add a separate workflow or things that are too specific. You rely on your ecosystem for that. And that’s what Google and Microsoft have done.

They’ll both allow you to render a signature inside of a doc that you author in those packages, but no one thinks that’s a substitute for Docusign for all the reasons we’ve discussed. It literally doesn’t come up with customers. I’ve never heard anybody say, “Well, maybe I could use Microsoft Word or Google.” That has never happened in my three-plus years here.

I’m just thinking about our listeners who, again, I think most people experience Docusign and then I know a bunch of lawyers who —

You get a PDF and that’s how they experience it.

That’s how they experience it. It’s all of the steps leading up to experiencing Docusign that are fascinating to me. 

So when you say, “Help prepare a document for signature,” there’s the drafting, right? There’s Microsoft Word… honestly, Microsoft Word might be the reason I’m not a lawyer anymore. I was like, “I can’t use this software anymore.” There’s the Microsoft Word of it where we’re going to redline a bunch of stuff. We have the document. From that point, what do you mean by “prepare a document for signature”? What are the services Docusign provides?

Some contracts are completely custom crafted, right? I think you’re a lawyer, right? So, most contracts start with a template of some kind. Maybe it’s a template for something really complex, like an M&A agreement, or something really simple, like an NDA, or anything in between. It could be a master service agreement between a company and another company or a license agreement, but they always have a template.

Then, they tailor it, with a sort of legal tailoring, right? I want to have different limitations of liability, I want different payment terms. And some of it is, let’s say, mass customization. I want to get the data about that customer and some things we’ve already negotiated, and I want that to flow automatically from whatever the system of record is. Could be Salesforce, could be Workday, could be SAP, could be whatever. And I want to populate that into the agreement. So this template that I have gets personalized and customized. That’s also what happens in consumer applications. The forms that you get from Chase or Wells Fargo are mass customized that way. That’s essentially a data pool. 

So, that’s what I mean when I say “customized” or “personalized.” Now, there are a lot of things that can flow from that. Let’s say you are a national employer, but you need to take into account employment laws in the 50 states. That would be an example of a customization that’s a bit more complicated. That gets even worse if you’re a global employer and you have to craft offer letters in 180 countries. That starts getting pretty complex. But there are rules. We have a system for applying those kinds of rules so that you can have global standards but still create documents that are tailored to local laws and customs.

This gets me right into the opportunity for AI broadly, which is that you’re starting to do some of the lawyering a bit, right? You’re saying we’re not going to hire the junior associate to figure out the local law. We can actually just programmatically make a contract that–

For that type of application, correct. We’ll come back to this, but I think there are two big use cases for us. One is to make agreement workflows more efficient, sometimes just by automating them and sometimes by re-imagining them. That’s a huge part of what we do. This mass customization is not a new thing. That’s been going on for a while. It’s sort of advanced mail merge, right?

[Laughs] You said it. I didn’t want to say it. You said it.

It is! But look, it’s really good. It saves a lot of time and a lot of companies live with that.

If I could start a business that was charging a premium on mail merge, I would do it too.

Oh my god, yeah. Managing people’s address books is another one that would be very valuable. No one has solved that. 

But anyway, making workflows more efficient and now tackling all of the steps in the agreement process is a part of what we’re doing. So, we’re tackling things like the ingestion of agreements. We have a whole queuing system where a sales or procurement rep can trigger the sending of documents with legal. It can get an automatic first review, then it gets assigned to a lawyer. The lawyer does a quick end edit, and everyone has real-time status. That’s an example of re-imagining legal workflow, which today is totally asynchronous email, unpredictable, and non-transparent. That’s one piece. I think that’s what people have historically focused on with contract management. 

Then, the other piece is to take AI and extract data out of the agreements to run my business better. That’s something that was conceptually possible historically, but it was just too heavy and hard to do with legacy LLMs. Now, we can do that completely automated. So, I can go to people and say, “Hey, you have 5,000 agreements with me. Would you like to know what’s in them? Let me highlight how these agreements deviate from agreements with peers. Company X is coming up for renewal in 90 days. How can I review that? How can I know what I should be renegotiating?”

I can give you that using AI, which is automated right off the shelf. It leverages the fact that you already used Docusign and you’ve already stored your agreements with us. It uses modern AI, and then adds, of course, some workflow and automation on top.

Let me take a beat here and just ask the Decoder question so I have a better sense of the company itself. How is Docusign structured? How many people is it and how are they organized?

Docusign has a little under 7,000 employees, and our structure has gone through some transformation. Historically, I would say Docusign, ignoring the very early days of conceiving the product, evolved into a very sales centered company. Most customers interface with Docusign through sales. We had a very large sales team. That was  the most powerful function of the company. Things flowed from there and were optimized around that.

When I joined, I very quickly realized we needed to completely reimagine our product roadmap. I really wanted to put product at the center of the company and say, “We’re going to articulate a new product roadmap, and things are going to flow from there.” That’s not to say that we’re not taking lots of customer input. After all, we’ve got 20 years of experience working in agreements with millions of customers, but we want the product vision and the overall architecture to be the guiding light for how we go to market, how we spend, etc.

The second area I focused on was marketing. Historically, marketing here worked in the service of sales. It was basically about creating qualified leads for sales. But I was like, “This is an electronic contracting product. They should be able to just come in and do whatever they want to do directly, self-serve to the greatest extent possible.” I came from Google where every advertiser, including people who spent more than $1 billion per year, places their own orders in the system. But we somehow couldn’t do that at Docusign. So, that was another area. It was about taking all the pieces of the company that worked on identifying, sourcing, and bringing new customers on, and combining them in one team. 

Then, there’s the third area that I’m really pushing on. We’ve always felt like we needed to own the ball. When you have an iconic, simple product, you maybe don’t need partners as much, but now with this broader vision for agreement management, we really need partners to help us go to market and install and service the product. So, we’re very focused on working with system integrators like the Deloittes and Accentures of the world, as well as distributors and resellers. Those are the big changes we’ve made.

I have to say, 7,000 people is a shocking number. The last number I heard was 4,000, which was shocking then. How are 7,000 people organized at Docusign? Is it sales and marketing? Is it product?

Sales and marketing is the biggest area. We have about 1,000 account executives who cover everything from SMB accounts all the way up to enterprises. We’re not a small company. We do over $3 billion in revenue. Our revenue per employee is pretty much in the median of the SaaS businesses.

But sales and marketing is the biggest function, and it’s everything from presales to account executives, to customer success people that help with implementation and support. We have about 1,000 people in product and engineering. We’ve had 1,200 in engineering proper, and then there are product managers, designers, and so on.

If you look at the number of countries and the complexity of the regulatory and compliance environments we’re in, that’s a significant effort. Now, pretty much all of the company’s open investment dollars over the last three years have been pivoted into [Docusign] IAM. We’ve gotten much leaner on the sales and marketing side, and we’ve invested more heavily in product and engineering and security–

IAM is the AI, I just want to make it clear for listeners.

That’s right. That’s our broader suite. 

When you think about shifting that investment focus and then something like self-serve… you were at Google for I think 12 years. You’ve been the CEO of Docusign since 2022, so you’ve been there a minute.

Three years.

How did you think about changing Docusign? How was it organized before? Then, when you got there, how did you work through, “Okay, here’s how I’m going to reorganize it for the opportunities I want to attack?”

I think the low-hanging fruit was helping the company pivot from being just a good machine for acquiring new customers but not necessarily an amazing retention machine. So, being focused on customer retention, customer success, and adoption. I think we took that a little for granted. So that was one area. Second was making us much more efficient, as I alluded to, whether it was through building out digital channels or just streamlining our sales and marketing efforts. Those were the big, immediate pushes into the low-hanging fruit areas that were in my control early on. 

But the biggest change was the product vision effort and pivoting the company towards being product innovation led and restarting the innovation engine. COVID was a seminal event in the company’s history and not in a good way. Everybody thinks that Docusign is a COVID darling and that must have been so great because everybody had to use us. Look, there was some of that temporarily, but in the long run, it didn’t change the secular adoption of signing things electronically. 

Everybody literally fell asleep. If you’re a sales rep and your revenue growth rate goes from 25 percent to 60 percent without much effort, you’re in the auto taking business. On the engineering side, we didn’t need to develop that much. We just needed to keep the servers running and scale that. So, I think restarting that and saying, “Hey, we can build something great, something new, and reclaim that innovation mojo.” I think that was the most important transformation for Docusign. 

It’s still ongoing, but the pace of product innovation and release is completely different now. We actually had to throttle the number of our releases because there was so much. It was sort of starting to overwhelm our customers and our sales team. We have so much stuff, and there are so many opportunities and that’s an exciting place to be.

Was that your pitch when you got the job? Very few people are going to ever interview to be the CEO of a multibillion-dollar, publicly traded company. What was the pitch? “I’m going to restart your product innovation?”

It was a dual pitch. I said, “Look, what I know I can do is come in and make things more efficient.” Of course, I used my digital marketing background from Google to make some suggestions for things that I thought we could do.

Did you make just a killer [slide] deck?

[Laughs] I didn’t get time to make a killer deck. Literally, I got a call from headhunter on a Thursday night that the board wanted to meet with me for an hour Saturday morning at 9:00 to hear my thoughts on the future of Docusign. And I was working full time at that time. I had a big job at Google, so I couldn’t just take a day off. I suppose I could have, but I didn’t. I basically had Thursday evening and Friday evening to prepare a short slide deck. The reality in these kinds of situations is that nobody expects the super polished multimedia thing. What they’re really focused on is the thoughts, right? You can use a 10-point Courier font, that’s fine. 

So, what I was really focused on was, “Look, I know I can make the company more efficient. I know there’s gold on the retention side. I believe that the agreement problem is fundamentally unsolved, but Docusign is the best position to capture that opportunity.” I mean, we were big Docusign users at Google. It still is one of our largest customers. Huge eSign and contract lifecycle management were the advanced contract management projects that enterprises use. Yet, I knew we were in the earliest possible phase of transforming how agreements get done. There was so much opportunity. 

I just articulated my confidence in that. I knew enough that I would lose credibility if I was too specific. So that took six to nine months to really say, “That’s fine at a 50,000-foot level. How do you get to the real vision? What’s the singular thing we’re going to do?” As an example of that, it’s easy to talk about all the steps in the journey, and I think people naturally gravitate towards the drafting and negotiating side of agreements because it’s almost like the movies. That’s what lawyers do. But we actually started at the end and said, “The foundational piece of re-imagining agreements is to have an intelligent repository where all your agreements are stored and to be able to apply AI to that so we can tell you what’s in all your agreements, start comparing them, and ultimately close the loop on what happens under those agreements.” So that’s what we built.

Is that what you mean when you say the problem is fundamentally unsolved?

Well, that’s part of it. The way I would describe it is we’ve digitized the asset. We’ve taken what used to be an offline asset, turned it into an electronic document you can move around electronically, mostly via email, and then we hopefully execute them electronically. Other than that, absolutely nothing has changed in agreements over the last 50 years. All other aspects of the workflow are just as inefficient, just as brittle, just as unpredictable. There’s just as much time lost waiting for somebody in the process. You don’t even know who they are and what step they’re on. Once you’ve signed, the agreements go to a deep, dark place. It might as well be a physical filing cabinet in a basement. In fact, they’re probably harder to find now than it used to be when they were in the filing cabinet. Now it’s on some SharePoint drive, in an email inbox, or who knows. 

So, bringing all that together and presenting that information in a way that is intelligent, bite-sized, and appropriate for the persona is incredibly valuable. Every CFO wants to know the top 10 contracts where the company has leverage or there are things they’ve negotiated for that they’re not getting. Every head of sales wants to know the top 10 renewals and the three things their rep should be renegotiating. You can just go down the list. It is an obvious pain point that no one has solved. So, that’s our opportunity. That was my pitch. I think it was a good pitch, and I think the story is much better now than it was three years ago [laughs].

I want to get into that. I want to talk about how you were implementing that as a part of it. I want to talk about whether it’s working, which is interesting. But I want to ask the other Decoder question first. 

You were at Google for a long time. Google’s decision-making… There’s a lot to say about decision-making at Google. What’s your decision-making framework? What did you take from that experience and what do you do at Docusign?

I loved my time at Google. It’s an amazing company, and I’m proud of the time I spent there. It’ll always have a special place for it in my heart. One thing I do not miss is the seven-dimensional matrix of trying to make decisions. I really try to avoid that here and have much greater clarity about who owns the decision, who needs to be consulted, and all those decision frameworks. 

But I’d say the one very positive lesson I took away from Google was that, in my experience — and this is not limited to Google —  the most effective leaders in tech can go super deep when they need to, and they need to be willing to do that. Management is not floating up in the abstract 50,000 feet above the problem just admiring it and thinking you can just operate at that level. I think when hard decisions need to be made on things that are complicated, you need to be able and willing to really dive into the details. I’ve always prided myself on that, and I think that’s particularly important for a transformation moment because it needs more push from the top to get people to change. People naturally want to do things super incrementally. That’s not unique to any one company, that’s just how people are. So, if you’re trying to get more radical change, you need to push. In order to push and not break everything, you need to get deep in the details. 

So, when we were deciding on the early architecture and what the key launch pieces were going to be for Intelligent Agreement Management (IAM), I was in daily whiteboarding sessions to help decide what that was going to be. Once we had set that direction, then I stepped back and just let the team run. Then you maybe get to a decision point where you say, “Well, how are we going to talk about it? How are we going to explain to people this newly reimagined Docusign?” That was a big moment where we got super deep into deciding how we were going to message this to the market, effectively how we were going to relaunch the company. 

And then you have to decide, “How do we take all of this capability and find the singular thing that we need to tell customers about to get them to try and buy it?” Because all software has 10 times the functionality that people use and finding that singular proposition is super important. So, those are examples of things I got much more deeply involved in. I think that is essential to being effective, but then you’ve got to let go.

Let’s talk about the technology here. It’s IAM, or Intelligent Agreement Management. That’s the new system. That’s where the AI focus is going. You’ve talked about it already a little bit. You have this library of documents you’ve signed, you can extract intelligence from them. 

I’ve got to say, I looked at the website just before we started talking to see how you are marketing this thing, and a lot of it involves Docusign telling you what’s in the agreement you’re signing. All of my lawyer red flags just went off. You should not let the robot interpret the document for you. One of the reasons is, well, is Docusign now responsible for the interpretation of this agreement? If I sign this and the AI hallucinated an interpretation of this clause so I’m mad, do I get to sue Docusign?

I think the reason you’re seeing that on the website is that we literally just launched this for consumers last week. It’s the most recent release. It wouldn’t generally be the top thing, but right now it’s newsworthy, and consumer-facing things tend to get more attention, as you know.

Here you are on Decoder.

With all that said, we’ve had agreement summarization for a long time. We provided it internally to companies that prepare and send documents. We waited a long time to launch it for consumers exactly because of the concerns you raised. We wanted to get to a high level of accuracy. We wanted to make sure that we could position this as assistive, that we’re not replacing a lawyer and you still need to get legal advice. People valued that sort of high level summary. All of our very robust testing suggests that people are more comfortable, have greater confidence, and, at the same time, understand that when something is sensitive, they need to get to a lawyer. 

Look, it’s a delicate thing. We want to be the place that people trust the most for receiving and executing agreements. Not providing an AI service isn’t really an option. At the same time, you’ve got to have a ton of guardrails to avoid putting yourself in a position with people who claim they relied on you or that you are acting as a lawyer. It’s a tricky one.

I’m just curious about the dynamic there. I was talking to a colleague of mine just before we started and I said, “They’ve got this new product.” And what she said to me was, “Oh, that makes sense. I already take the agreements and paste them into ChatGPT.”

Exactly.

It’s happening. But in that case, you don’t get to go in front of the judge, file the complaint, or even send the threatening letter being like, “This is what ChatGPT told me the document I was signing says.”

They’re probably worried about that–

But I think there’s that fact pattern that says, “Well, that’s your fault,” right? I pushed the button. I indicated consent after having my identity verified in the database. Next to the summary is a different fact pattern. What are the guardrails that you thought of? What made you comfortable that you could escape… because it’s going to happen, right? Someone’s going to blame you for a mistake they made or that the system made. What are the guardrails that made you comfortable with that or that made you not liable at all?

I think there are two answers to that, sort of what got it comfortable legally. We got very comfortable with all the language, the disclaimers, and how it was done, so it was fine. But I don’t think that’s the big question. To me, it’s more of a moral question. Are you doing the right thing for the customers? Can you feel good about that? We got to a place where there was no question in our mind that this was better for consumers, that to continue to uphold trust in our platform as a place to come and execute documents, we were in the best position to provide that kind of additional advice and context.

In fact, it would be a dereliction of duty not to provide it. We needed to do the best possible job to make sure that people understood that this was context, but if this is something that’s really sensitive for them, they should get a lawyer.

Of course, they don’t today. They just sign [laughs]. I think we’re improving the situation both for consumers and also, by the way, for companies. So, that was our motivation, and it’s impossible to eliminate… People are going to claim things. I think we’re doing the right thing, and we felt like we were taking the necessary precautions not just from a legal perspective but from a company values and walls perspective. This felt right. So, that’s how we got to it.

The reason for me asking about liability and whether the system’s going to be wrong is that LLMs as a core foundational piece of technology still hallucinate. This is the flaw with LLMs. If you talk to any of the big AI CEOs, if you talk to [Google CEO] Sundar Pichai, he’ll say it’s also their feature. The reason that they are creative is because they are effectively hallucinating, and if we box that in — 

That’s a creative repositioning.

You said it, not me. This is the dance, right? You want an LLM that can do a bunch of creative writing, you need hallucinations. You want an LLM that’s going to do legal analysis on a library of documents that is maybe 20 years old, you really don’t want it to hallucinate. How are you constraining the LLM to make sure it does what you want it to do?

So, if we pivot to the folks who use our tools to prepare documents, there are a lot of guardrails that we put in. We have a whole risk scoring thing. We benchmark you versus your existing templates — or playbooks, as they’re often called inside of companies — and highlight things that deviate from that. This is true both for agreements that the company prepares itself and for third-party agreements it receives from others because most companies are takers of terms from other companies, right? So, there’s a lot of workflow there. 

We also don’t fully automate flows. We deliberately put humans in the loop at key decision points, and I think that will persist for a very long time. Companies are very averse to the risk of automation going rogue. We talked about it at Google for a little while, and there were a lot of efforts to automate the commerce process. And this was for buying toothpaste. Both consumers and retailers were very resistant. Even with all the new stuff, I think it’s getting better, but I still think there’ll be a lot of hesitancy, and those are much lower stakes than agreements.

Put that into practice for me with the products you’re talking about today. I am a company. I’ve got 20 years of agreements in Docusign. I want to know how my terms have changed over time. I ask Docusign to generate that intelligence for me. What’s the guarantee that it’s not just hallucinating, that it doesn’t go through the first five years in the rag process and just decide to get bored and make up some contracts that never existed?

I think the stakes and the opportunities for hallucination in an extraction scenario are far lower than in a drafting scenario.

Sure. But it’s going to draft a report, right? You’re not layering — 

Yes, it’ll highlight things. You’ve got to link to the sources. You’ve got to have all of that to show your work and the logic of the decision making. But look, there’s no amount of automated decisioning that’s ever going to be 100 percent perfect. I think we’re getting to an extremely high level of accuracy. We have the largest agreement repository in the world. 

This is a very interesting story. When we got started with IAM, we had petabytes of agreements because of our signed product, but we did not license those agreements specifically for processing and AI because no one had thought of that, and I don’t think people would have consented anyway. So we said, “We’re not going to use any of that data. We’re going to require customers to consent on a one-on-one basis to allow us to process individual agreements, and we’re going to put in product functionality so they can opt out at any time.” That meant that when we started, we only had access to public agreements, which is what the big LLMs have as well.

We had a very good level of accuracy in our extractions as rated by human eval. Then, we turned it loose on actual agreements inside these companies, and the accuracy fell 15 percentage points out of 100. 

Wow.

That turns it from super useful — not perfect but super useful —  first drafts into, “Wow, I feel like I have to redo everything this thing does.” We’ve now mostly worked our way back to our original starting point, and we think we can go much further. That’s because we now have 150 million private consented agreements in our AI, and we’re adding tens of millions every month, which is orders of magnitude bigger than anybody else. And these are all the complex private agreements that generally aren’t publicly available.

So, I think we have a hidden, huge AI advantage and accuracy. No one’s ever going to get to 100 percent, so it doesn’t take away from your point that there could be cases where you need human eval and oversight. But I’m saying we’re going to be in a much better place to get to the real productivity benefits because we can be much more accurate.

When you think about that data set, you’re not training your own models, right? You’re obviously using foundation models from other companies.

Oh yeah. We started with OpenAI and ChatGPT, and we used Azure as our first public cloud provider. Since then we’ve added other frontier models, so we score them against each other and use the best of the best. I mean, it’s an incredible innovation pace, and the capital investment that’s going in and the cost per unit, if you will, has just been dropping precipitously. That’s amazing for a company like us, for whom that’s the cost of goods. I can now go to pretty large-sized customers and say, “Just give us all your agreements. It’s included in your price.”

Tell me about the dynamics of the AI pricing per token because the idea that the prices are falling requires some amount of competition in the market, right? It requires the ability for you to switch. Are you building your system so you can switch from model to model or are the models different or differentiated enough?

We already have. Early on, you have to say that ChatGPT was head and shoulders ahead of other frontier models. So, we started there, plus with our internal models. We had five years of experience building and using AI models internally. Over time, it became clear the frontier LLMs were great, OpenAI in particular. For a little while, it was kind of the only game in town, but now there are a lot of choices. We’ve gotten excellent results with Gemini and other models. So, we have a good range of choices, but even if you’re with one vendor, the cost of processing an individual document has plummeted. 

When we got started in the summer of 2023, I was very worried that we’d need this very complicated pricing structure tied to the number of agreements you were uploading because the cost to process and reprocess them all the time for all the things people wanted to do felt like it was going to be prohibitive. Now, even if you’re a fairly large company it’s just included in your subscription, and it’s a totally different game. I mean, it’s like a factor of a hundred.

So do you think the models are interchangeable? I mean, you’re talking about them all getting good and they’re getting — 

I don’t think they’re perfectly interchangeable, but for our purposes and applications… I mean, obviously the models have different personalities in a consumer context, and there are some models that have specific advantages, like for workflows like coding.  But for document extraction, we’re able to get very good results with multiple models, including multiple top frontier models. And they’re advancing at roughly similar rates.

This leads me to basically the bubble question. If you’re saying we’re delivering a bunch of value, but the cost of the models is so low that I’m just bundling it into my existing subscription cost — unless the customer’s so big that they’re using massive amounts of tokens — how do you expect the model companies to make any money? That’s a race to the bottom. If you’re not charging additional margin, you’re not going to pass any on to them. How do you expect that to reconcile?

Look, I am obviously writing a large check to Google. So it’s not that it’s insignificant. It’s just that the incremental value of an additional — 

You’re getting more usage for the same dollar.

I’m getting more usage for the same dollar. We rate, but they’ve gotten so much more efficient. The models have gotten better, the hardware has gotten better, the amount of CapEx that’s gone in that’s now being applied is incredible. So, I think some of that is already in the water. But to your point, I think they will all add value in different ways. I do think that the LLM model space is highly competitive, and you’ve got to add value above or below that. 

You can see that in the strategies. OpenAI is really doubling down on both its consumer and enterprise efforts. Google has investments in the hardware side and on the cloud stack. It uses its AI for YouTube, ads, and other vendors. Anthropic is incredibly successful with its coding product. So, you’re seeing different strategies play out to leverage a strong position in the foundation model space.

I’m curious about that because you have the perspective. You run a big enterprise software product. ChatGPT announced DocuGPT, and the idea that it’s going to come and attack your moat, which is the database of identities and the flow, doesn’t seem like a big threat. But I look at that big sweep that you just described, and most of those companies think their money is in enterprise use, right? That’s the first big set of customers. It’s big businesses with budgets that need to get more efficient and increase productivity. Maybe that’s happening and maybe it’s not. I think that’s still an open question.

It’s starting to happen.

Something’s happening. Then, you have the big consumer products. So, you see Google and Meta saying, “Well, we already have ad technology. Meta’s going to move all of its stuff to GPUs, and we’re going to serve Reels ads that way.” It doesn’t matter how many GPUs we buy because it’s already being monetized. Google’s the same way. Then you have OpenAI, which announced an advertising product last week. “We’re going to do ads in ChatGPT.” 

I guess the question in relation to Docusign is, do you think that industry is stable enough for you to bet on in the long term as it works through all these machinations? This is the core technology of your growth. Second, you have experience at Google, you have experience running an advertising business. How do you think that’s going to go?

Okay, there were a lot of questions in there. I think from a Docusign perspective, it’s all been good so far, right? It’s dramatically expanded the value we can deliver to customers, the scope of our services, and, in a way, what we could leverage through the R&D of others while preserving a meaningful competitive advantage through our data, workflow, and trust. I think it’s been a huge win.

As you alluded to, I don’t believe that the big LLM providers are going to provide agreement management solutions. I think they’re going to look for others like us to build applications on top of their systems. I suppose it’s possible that there could be some of that capability that seeps into standard products, like how you can do it in a summary. I feel like it’s been amazing in opening up a much broader market opportunity for Docusign, and we are running as fast as we can to capitalize on that. I think the cost dynamics have been super favorable to us so far, and frankly, I expect that to continue. 

In terms of the other part of your question, on the ad side, I think it was always inevitable that OpenAI was going to get into the ads business. You can’t really do a scaled consumer services play without advertising these days. It was just a matter of time before it got there. It’ll be delicate to incorporate ads into an assistive AI experience. But it was delicate for Google to do that with paid search but it did, and it turned out that it unlocked huge value.

So, that prize is very significant, and I actually think it was late in launching it. It should have done it earlier. The transformation of having all the context from the journey you’ve been on and the ability to fully close the transaction potentially takes that kind of intentful activity to the next level. You can literally almost go through what used to be called the funnel on one platform. That’s incredibly valuable, and so it’s a big prize. That’s why I think OpenAI is investing, why Google is fighting hard to keep its position, and why I think Meta is well positioned. It’s why there are so many Meta people at OpenAI.

I think it was always inevitable. We’ll see how it shakes out. I don’t have a moral issue with it. I think it needs to be done well, right? I think you can say a lot of things about Google, but I think it did a pretty good job of keeping those walls pretty separate. I saw that when I was there. You have to maintain the integrity of the results people are getting while incorporating advertising experiences that are accretive to the users. That’s a difficult thing to do.

I was just at CES, and you couldn’t turn the corner without a marketing influencer jumping out of the woodwork and saying, “the funnel is dead” because everyone experiences everything randomly nowadays. That’s a different podcast.

That’s a different podcast.

Let me try to draw a connection between these two ideas. There’s a reason I ask them together. If you are looking at all the big model providers and they are saying, “Our first opportunity is an enterprise and the enterprise customers are going to unlock a bunch of value and build products around them.”

Are they actually saying that?

By and large, when I talk to these folks —

It doesn’t seem like what OpenAI, Meta, and Google are doing, but I mean —

Well, I think Meta is the unique one. Maybe it’s the exception that proves the rule, because it has no enterprise business to speak of, so it has nothing else to say.

Fair enough. I agree there’s a huge focus on enterprise, and we’re very focused on enterprise.

Of course. So you see that. I’m talking to you, you’re building a scaled enterprise product on the back of these models, and you’re saying they’re almost interchangeable and the rates are dropping because I can go get pricing terms I can just switch. That’s one dynamic of pricing that’s happening in the industry. 

Next to it, you have, “Well, what’s the biggest prize in the history of the internet?” It’s search advertising. OpenAI is going to attack that, and it’s going to take some of that share away from Google, and Google’s certainly going to defend its territory. Then, Meta’s going to do whatever Meta’s going to do. That’s where you would decommodify your models, right?

You would say that ChatGPT is so much better and such a better consumer experience that you have no choice because all the users are here, and that might be a zero-sum game. In my view, those things are pulling in wildly opposite directions. You have commodity models for enterprise, which is where all the budgets are, and then you have deeply specialized consumer experiences where you can layer in advertising at high rates.

Can you bring those together? That’s why I asked if the foundation seems stable because eventually, one of those things is going to be more lucrative than the other. You might run out of enterprise model providers, and that industry might collapse to one or two that charge high rates.

I think both are so large that the biggest players are going to go after both but on very different teams. Obviously, we’re very familiar with Google. Google Cloud is a standalone operating unit inside of Google and operates separately, and necessarily so, right? In fact, I don’t think Google Cloud could have become successful if it wasn’t pulled away from Google’s traditional consumer services and ad business. OpenAI is a smaller company, but is going to need to replicate that very different MO and go-to-market that comes with the ads business in the consumer side of the house versus the enterprise side. 

That’s a lot to take on for a young company. It’s an incredibly capable team with a huge amount of IP, but that is a very ambitious undertaking to do both. Anthropic seems to have decided to focus exclusively on the enterprise and executing that incredibly well. We’re using Claude Code and it’s fantastic, so huge kudos to it. Then, there’ll be a host of other players that become either more enterprise or more consumer focused. But I think Google, Meta, and OpenAI are probably the three that are trying to do both. Meta is probably more consumer-y right now, but clearly has ambitions in the enterprise.

Are you structuring your contracts with these vendors to be long-term or short-term? How do you think about that?

The contracts in that space tend to be relatively long-term. We and they want three-plus-year, commitment-type deals. As much as people want to say, “Oh, I can just swap one for the other,” that’s not really how things work if you’re doing enterprise deployment. So, there’s significant investment in pointing to a platform. And the vendor, of course, wants to see the benefit of your growth and the upfront investment they’re making in helping you come onto their platform. And so they’re three-plus-year deals, sometimes longer.

When do you think you’ll have a feature set using these tools that’s so robust that you can actually charge a premium to all of your customers? Or do you think it will always be —

We kind of do that today. We went from being a signed provider. We’re charging a substantial premium from moving to this AI-assisted suite because of the value. I can provide that value instantly, right? You signed with Docusign, and I can turn on and give you AI insights into your agreements on day one. In fact, we deploy our new products as fast as we can deploy. We deploy in under 20 days, which is kind of unheard of. If you follow enterprise software, that’s kind of unheard of. And frankly, a lot of that is just human stuff because the product is ready on day one. 

I think we sign agreements with customers… Our customer agreements right now are around 19 months on average, but they’re longer with big customers and tend to be shorter with smaller customers. That’s a typical pattern. I expect that our agreements will get longer over time, but we already charge a substantial premium to get access to the AI features and customers have been very willing to pay. We have over 25,000 customers live on this new AI platform under 18 months.

I’m always curious about that number, and then I want to wrap up. I’m always curious about that number. I hear these usage numbers from all the companies that deploy AI tools, and underneath it is, well, it just showed up one day and we’re counting that person as an AI user, right? AI overviews are there, everyone loves them and it’s like, “I don’t know about that. They just are in my face whether I want them there.”

Are you actually measuring happy customers using the tools because they want to or because the button is there and everyone just clicks it?

Both. Look, we give you a license for however many users you want. We obviously track the consumption of that license, what are people doing, how often they access their repository, how many searches they do, how many extractions they do, and how many document sendings and document executions they do? We keep a very close eye on that as fundamental health metrics. 

I think product adoption leads to renewal and retention, and we are very focused on that. No, not everyone uses the AI features, but we’re seeing really robust usage. We’re constantly looking for more ways for users to discover what’s possible. This is one of the endemic problems of enterprise software. You build this stuff, but how do people actually figure out what’s possible?

The good news is we have something that has a near universal value prop and can be deployed easily. There are all kinds of magic moments where, “Oh, did you know you could do this? ” We’re making that available. So, as you send a document or sign a document, “Hey, would you like an agreement somewhere? Would you like to know how this agreement pairs? Hey, did you know that we have all of them and you can see all this stuff?” It’s a new world. It’s exciting.

I do love that the future of all software is some combination of ultra smart Clippy and Tooltips. There’s something there, someone should write a book on it.

[Laughs] Hopefully we’re providing more value than that.

You’re an enterprise software CEO. You’ve come and you’ve faced the gauntlet. So, I’m going to ask you the hardest question of all. You use your own tools. You started off by saying you use your own tools. What is your biggest frustration with Docusign and how would you fix it as a product?

I don’t think our mobile experience is good enough. I would expect that everything should automatically flow. We should predict the next steps. It all works. Obviously millions of people sign with Docusign daily and often on their phones, but I think there’s still some room for improvement there. So, I’m pushing the team on that.

Well done. You actually answered the question. Most enterprise software CEOs don’t, so I commend you for that. Allan, this has been great. Thank you so much for being on Decoder.

Thank you, Nilay. I really appreciate you having me.

_{Questions or comments about this episode? Hit us up at decoder@theverge.com. We really do read every email!}