I’m just trying to keep up.

Stacking, not cutting

This week’s question asked how firms are putting AI into the tools they already have. 52% of firms are stacking AI features on top of existing tools, and fewer than 7% said they plan any cuts. Firms are still spending. Amanda asks whether that means AI is solving new problems, or nothing at all.

Keeping up is not a goal

At one of his kids’ events, a managing partner told Rob AI was consuming every day. Asked what problem they were trying to solve, the answer was none: they were just trying to keep up. Amanda heard the same thing at a roundtable at the ALA Executive Leadership Summit. Many firms still want to be fast to add, not fast to look at what they are adding.

Context needs a foundation

Rob spent the morning building his own memory layer, and context was the theme of the NetDocs conference two weeks earlier. You do not have to leave your vendors, but your billing, practice management, and document management need to share data. Building context around a matter takes client names, emails, time entries, and fact patterns in one place, which is why add-on products hit a wall without the foundational systems. Yesterday was the best day to start; today is the second best.

Demos and pricing

Amanda brings two pushbacks from members. Demo performance is burning goodwill: firms that moved to fully cloud-based, AI-inclusive tools found the real product underperformed the demo. And vendor pricing is the structural problem nobody can solve: token-based pricing, SaaS escalators, and research platform licensing push costs up regardless of usefulness. Rob’s answer on demos is to make vendors show you, and to try the product where you can. On pricing, he says AI compute scales linearly in a way SaaS did not, so usage and outcome-based models, with caps, are coming.

From open API to connectors

When Amanda’s firm bought Centerbase seven years ago, the team joked about taking a shot every time a salesperson said open API. Now open APIs matter more than ever, because context is hard to build without them. Rob says firms will stop hearing about APIs and MCPs and start hearing about connectors: an easy button, or at least an endpoint and a key, simple enough to set up without code.

How do you measure the ROI of Word?

Amanda reads a partner’s reply to an ROI question: after forty years of practice, through fax machines, computers, and cell phones, it is too big a transition to measure at the present time. Rob says outcome-based pricing and a clear before-and-after make ROI easier, while a general tool like Claude added to keep up is harder to measure. The old benchmark of 3% to 5% of revenue on technology is giving way to 7% to 9%. Welcome to running a business, not a firm.

What Amanda built with Claude

Amanda replaced a legacy HR system for attorney reviews with Microsoft Forms and SharePoint, with Claude running the browser to build the forms. Associates name who they worked with, and the substantive reviews assign themselves, alongside hours and collections data the old system could never reach. It took a couple of solid days and stays deterministic, no AI summaries of anyone’s review. She also turned recordings of the open enrollment sessions into an FAQ that walks people through choosing a plan.

Transcript

Welcome back

Rob: Amanda, welcome back to Sidebar Live. We took two weeks off.

Amanda: I think I’ve taken three weeks off. Didn’t I miss one?

Rob: You know, before we went live, we reacquainted ourselves as friends.

Amanda: We did. We have not talked to each other in weeks, which is a long time.

Rob: I mean, it’s a good thing. Listen, I need a little break from you every now and then. But yeah, it has been longer than usual. And Debbie’s supposed to be here, so maybe she’ll come in at some point and make an appearance. We always get a Debbie surprise. So let’s see here. We have a lot of great numbers that came out of this week’s Sidebar results, so let’s just jump into it. Are you going to pull it up?

Amanda: I am.

This week: stacking AI, not cutting

Amanda: So the question we asked this week was really about how people are putting AI into existing tools they have. The overarching theme is that 52% of firms are stacking AI features on top of existing tools. So they’re not really making any cuts, because we asked: are you making any cuts? Have you put anything in that you’ve now had to take out? And the response was, well, since half of what we’re doing is stacking AI into things we already own, no, we’re not making any cuts. In fact, fewer than 7% of our members said they are planning any cuts. So it’s really interesting to hear that people are still spending money.

Amanda: And to me, does that say they’re not really solving anything? Or is AI solving new problems they couldn’t solve before, and therefore they’re adding it?

Rob: Well, you’d have to hope.

Just trying to keep up

Rob: I don’t think I shared this; if I did, I’m sorry. I was standing at one of my kids’ events a few weeks back, and a managing partner for a firm in town walked up behind me. I coached one of their kids. We started talking, and they said, AI is just consuming me. I’m spending all day trying to figure out what to do, what to bring in. I said, okay, what are you trying to solve? Let’s start there. And they said, well, I’m not trying to necessarily solve anything. I’m just trying to keep up.

Amanda: I have an identical story, just with a different group. We were at the ALA Executive Leadership Summit and we had a roundtable, and one of the questions was, well, what are you doing for AI? And I had the exact same answer. What problem are you trying to solve? Well, we just need to get AI in. What do you need it in? What is it solving? There are still so many who just want to be fast to add, and not fast to look at what product they’re adding. That hasn’t changed. Nothing’s changing there.

Context and the memory layer

Rob: So I can tell you, I spent part of the morning putting together my memory layer, because I’m like everybody else. I have five different emails for five different things. Everything’s going in different directions. The reason I’m saying this is that you’re hearing vendors talk a lot about context. That was the theme of the NetDocs conference two weeks ago. It’s important, and it speaks to what we’ve talked about before. It doesn’t necessarily mean you have to leave all your vendors. But you have to have that foundation first, where the context can be stored, whether that’s the vendor you have today or another one. Think about your billing system, your practice management if it’s the same thing, your document management. Can you share things between them? That is so important, because the people who aren’t building that now... what’s that saying?

Amanda: Yesterday was the best day to start, but today’s the second best day.

Rob: Yeah. Every day you miss it, you’re not building that context, and things aren’t getting put together.

What a mid-sized firm should do

Amanda: What does that look like, though? If you’re a mid-sized firm, what does that realistically look like? What should I actually have on my to-do list?

Rob: Okay. Well, it depends on your vendor, so I would start there. But let’s talk about it from practice management, because that’s what I know.

Amanda: What, really?

Rob: Yeah, I don’t know it, I just sell it. If you’re trying to build context around a matter, you need certain things. You need a lot of the fields you’re storing on a matter record, because you need to know client names, you need emails. If you’re pulling together context from other systems, you need these data points, this metadata, to help you pull it together. Especially email: you need the actual emails related to it. You need time entries related to the case. You need fact patterns. You need all of this put together in a single place. So whoever your practice management vendor is, they’re the ones who need to start building this memory for you, or one of your core foundational vendors. Because if you start building these memory layers in all these add-on products, they get to a wall. I can tell you, because I’ve been talking to vendors all week. They get to a wall where they’re like, okay, what’s next? We can’t get to any more, because it’s all stored in those foundational products.

Amanda: So they all have to talk to each other?

Rob: They all have to talk to each other. Yes, they do. It’s important.

Amanda: And when they don’t?

Rob: You’re missing the context. And there are vendors out there right now that already have that out, and vendors that are working on it, because it really is the next wave. There are certain places you can grab context from: systems and data that are already out there. But you see a lot of vendors going after conversational context now too, because it’s the one thing you probably can’t pull from anything else.

Pushback: demos that overpromise

Amanda: So can I push back about vendors for one second? Two things came out of Sidebar this week. The first one is that demo performance is burning goodwill. Members told us about moving to fully cloud-based, AI-inclusive tools, only to find the real-world product underperformed what vendors showed in demos. And that experience is fueling the wait-and-see posture elsewhere. The other thing we heard is that vendor pricing is the structural problem nobody can solve. Token-based pricing, SaaS escalators, and, this is a terrible word, monopolistic licensing requirements from major research platforms are all pushing costs up in ways that have nothing to do with whether the AI is useful. Firms can’t cut their way out of this.

Rob: Yeah, so let’s take them one by one. Is there such a thing as a perfect product? No, there’s not. Is AI moving very fast? Yes. We weren’t talking about memory layers last year. Well, some people might have been, but it wasn’t really a thing. All of this is moving fast. What I would say, and I run up against this with AI products, is that some of them are easy to try. Those that are really well integrated are harder to set up trials for. They take a lot of work on the vendor side and a lot of work on the firm side. It’s just a fact of business: sometimes it’s hard to actually set that up and run it.

Amanda: Are you finding, though, as somebody in the market, that people are holding you to the carpet more, for lack of a better term, about selling products that work the way you said they would?

Every time I hear API

Rob: Yeah. But I remember, Amanda, I think it was when you bought Centerbase, when you were going through that evaluation. The fact that I can remember something from this long ago is great too, by the way. I think I scarred you for life. Can you hear things in the background? Okay. So think about this. When you were buying Centerbase, I think you had a line: every time I hear the word API from a salesperson, I’m going to take a shot. Well, I feel like that’s what AI is turning into in some ways. Sometimes salespeople say, well, our AI is able to do that. I think it’s fair for you to say, well, show me. Let me take it for a drive. Some might even let you do trials. But it goes back to what we’ve always preached: don’t take people at their word. Make them show you. And all the faces you’re making, I know that’s you visually making jabs at me.

Amanda: So here’s what’s interesting. This was six years ago that we bought, right? Or seven, maybe. We went live seven years ago this week. At the time, everybody kept saying open API, open API, and we would take a shot. But now that actually is way more important, because context is hard to build without open APIs. At the time they would say it, and it still wasn’t truly open. Do you actually integrate, or am I going to have to spend another $100,000 to integrate with something else?

Rob: And I would say it’s going two different ways right now. I hear Scott on my team say this all the time, and it’s very true: you’re not going to hear about APIs, MCPs, all these technical terms. You’re going to hear about connectors. Like an easy button: connect. And those that don’t have easy buttons have to be easy enough for you or me to just go in without a coding application, without knowing what we’re doing, and say: here’s the endpoint, here’s the key, go. So we’re getting there. That’s hard to build, because everybody wants everything to connect. And there’s a lot of politics that go into open APIs, if we’re being transparent. From a vendor’s standpoint, we’ve got a business to run and protect. We want to give our customers everything they want. How do we balance those? How do we innovate? So as firms are dealing with these dynamics, so are vendors, and they’re trying to figure out the best way to serve their clients too. What was the other part of it?

Vendor pricing nobody can budget for

Amanda: The second part is the vendor pricing. The first was demo performance. The second one: vendor pricing is the structural problem nobody can solve. And I see this from our end, because I’m trying to budget for next year. When you’re budgeting a year in advance, IT is more of a question mark than it’s ever been. You can guess what token usage is going to be. You can guess what your SaaS escalators are going to be, unless you signed a multi-year. But you can get into it and have it wiped out by somebody coming in and requiring a 30% increase, which I’ve had. Or, oh, that has been sunset, now you must be on this. So vendor pricing is a structural problem nobody can solve. Are you hearing that? How are you approaching that?

Rob: Pricing and packaging is changing. Remember when we went from on-prem, with a license cost and a small annual maintenance fee, where you paid more up front and less over time, to the SaaS model nobody could understand? I was talking to a new AI vendor just before this call, and I was telling them that until about 2018 we were still justifying the cloud, and probably SaaS pricing beyond that. I feel like this is another wave. What’s different is that it’s usage based. It’s different for the vendors too, because a traditional SaaS company is a hockey stick curve: a lot of cost up front, but then profit starts to shoot up, and costs don’t scale linearly. With AI, the more power you want, the more things you want to work together, it costs compute, and that costs us. That’s pretty linear. So the usage model, or the outcome model people are moving toward, is what you’re going to see. There’s going to be some kind of blend. You have to put caps on things. And respectfully, I’d throw this in the ring too: there’s a bigger discussion here that we’ve hit on a few times. Yes, the cost structure is changing, but you’ve also got to change the way your firm economics are working.

Firm economics and ROI

Amanda: Can I read you something somebody sent me a copy of, speaking of firm economics? They forwarded an email where partners were asking, how are we going to measure the ROI of this? It doesn’t make any sense. And the administrator was trying to explain that at a midsize firm it’s going to be hard to determine ROI, and here are the things we’re going to do. And the response from the partner was: I’ve been practicing 40 years. I guess this is like the fax machine, computers, and cell phones, which were not part of my practice when I started 40 years ago. Ultimately, it is too big of a transition to measure at the present time. I thought it was really interesting that they’re lumping this transformation in and recognizing there isn’t going to be a direct ROI, maybe not right away.

Rob: I would say it’s something vendors are still figuring out too. I just mentioned outcome-based pricing; outcome-based pricing helps you tie that ROI better. I would also say that if a firm approaches this trying to solve a problem, with a here’s where we are now and here’s where we want to go, out of the gate, it’s easier. Is the problem solved? Are we making more money? Are we able to offer more services? Are we able to do this faster? Whatever that is, you’ve got to define it from the beginning. When you put in something like Claude, that’s more general, and people are doing it because everybody’s doing it. They’re just trying to keep up. It’s going to be harder to measure, because you didn’t put a stake in the ground from the beginning.

Amanda: Well, it’s also like: how do you measure the ROI of Microsoft Word?

Rob: Yes.

Amanda: It’s a fundamental business tool now that you can’t not have. Every dollar matters in every business, but in law firm economics every dollar is counted by multiple people.

Rob: It is. And I saw, too, that maybe two years ago, before AI was really big, the benchmark was what, 3% to 5% of your revenue toward technology? You and I have talked about that. And I’ve seen more recent numbers that say, hey, you’re transforming. Your people costs might go down, something else might go down, but you might need to allocate more to technology. I’ve seen 7% to 9% now. Welcome to running a business, not a firm. Every other type of business deals with this stuff. For some reason, firms just can’t get there.

How much should go to technology?

Amanda: I wonder what percentage of their gross other types of businesses spend on tech. My whole career has been in legal, so I don’t have that benchmark.

Rob: That’s the Claude tool. That ROI Claude tool.

Amanda builds attorney reviews with Claude

Amanda: Can I tell you something fun I built with Claude this week?

Rob: We always like to end with something fun Amanda built with Claude.

Amanda: I think I already mentioned this one, but we are starting attorney reviews. The way our attorneys do reviews, all of our associates do a self-review. Then the partners either do a substantive review of an associate, one they’re directly supervising every day, or something called a citizenship review. That’s basically: they’re at firm activities, I’ve seen them networking, they’ve helped me with one or two projects. A very surface-level review. We previously used a legacy HR system to do this, and it was very difficult. They’d have to remember their passwords, and assigning the different reviews and deciding who can see what was all about settings. So I said, you know what, this is not difficult. I can make this. It did take me a couple of solid days and a couple of iterations, but I built it in Forms using SharePoint, and it built the entire thing out. Everybody got an email today that said, here are all your reviews, and when they click it, it’s the ones they were assigned. And, context: I was also able to point it to files with associate performance numbers. So now when they do the review, they can see where the associate is with hours and where they are with collections. Our HR system doesn’t connect to our time and billing software in any way, so you could never get that performance information out. It was really helpful.

Rob: Can I ask a question? The forms, from my standpoint, and I’m saying this to show how attainable it is: that’s not even using AI, is it?

Amanda: Well, I had Claude run the browser to build the form in Microsoft Forms.

Rob: Ah, okay. Which is good, because what you’re also seeing is that everybody’s shifted to: everything has to be AI. A lot of things can still be deterministic.

Amanda: Yeah. I’m not going to use AI to summarize. I could do a synthesis of the feelings in each of these reviews, but I’m not going to. The way we’re doing it, it’s the actual answers given. But on the associate self-reviews, for example, we ask them who they’ve worked with in the last year. Now, when they answer, I don’t have to know that. It went and assigned the substantive review to all the people the associates identified. I didn’t have to go in and figure out how many hours this person worked with that person. So I built the whole thing out, and I’m pretty proud of it. We’ll see how it goes.

Rob: Over time, if an Excel file like that is your database, you could hook it up to an artifact and see those changes over time.

Amanda: Well, the goal is that the final review is output as a PDF that’s connected to our employee files, which are currently held in our DMS. But the performance over time, how the ratings change: payroll software and HR software will do all of that. It’s just that the programming, and going back and forth with the programmers, has historically been so difficult that you’ve had to accept the limitations. Here’s your two thousand dollar implementation fee, and you have to accept that, for example, each person can only have 16 people assigned to them. That was a problem I had before. We have 17 partners, so which partner gets left out? It was silly.

Rob: I think it’s really cool, especially stuff that’s deterministic like that and attainable to build. It’s a perfect use case for it. And maybe you can tie some of your savings back to your ROI on Claude and Microsoft.

Amanda: Well, that’s a really good point, because I know how much it costs to buy review software. That’s not for legal work, but it is for us.

An open enrollment FAQ

Amanda: We’re also going through open enrollment. Oh, one more thing Amanda built this week.

Rob: This is all just the Amanda show. Welcome to the show.

Amanda: I know, I’m so sorry. We haven’t talked in a couple of weeks.

Rob: Are you missing me?

Amanda: So we’re going through open enrollment right now. We’re rolling out a brand new insurance plan and provider, and it’s very different from anything we’ve had before. We had people come on site to do the presentations, four of them, so people could come in at any time, and some people were online asking questions in the chat while others were in the room. So I turned on my AI note-taking device and recorded all four sessions, plus the sessions where they were pitching me the product, put it all together, and built an FAQ document: how do I decide which plan is for me, what are the costs, all of it. It just built the whole thing out.

Rob: It’s brilliant. Is it an artifact you’re just sending out? The week you missed, Amanda, Debbie and I were talking about how we don’t send editable Word documents around the company anymore. It’s all artifacts.

Amanda: Yeah. The next step will be taking this data and putting it in our SharePoint, so when people chat with the company they can ask a question like, who do I call for insurance? That’s the context. That’s my next step.

Rob: So Amanda, I’ll do a little pitch for you here. Amanda has actually opened her AI consulting business. Feel free to reach out to her. She’s happy to help you and your firm.

Amanda: That is not true. I don’t have a consulting business. I do this because it’s fun.

Rob: It is fun.

ALA Annual and the Saturday newsletter

Amanda: Can we talk about the fact that we were chosen to speak at the ALA Annual Conference coming up?

Rob: We should pitch that on a regular basis.

Amanda: Rob will be teaching everybody about Claude, and I will be there.

Rob: Well, Amanda will be there with her consulting business, and Debbie will be around as well, and I’m sure she’ll be part of us chatting. So, excited to see people there. We are wrapping up, guys. We continue to see growth from not only firm administrators, but managing partners and partners as well. We’re happy to let those in. Sorry, we still don’t let vendors or consultants in. Maybe one day we’ll figure it out. But if you’re going to join, go to gosidebar.ai. Amanda will approve you, hopefully within a day if she’s not distracted, and you’re in.

Amanda: You’re in, and getting all the great context.

Rob: We not only send out our Tuesday email, we’re also sending out a Saturday newsletter now. And although you’re listening to us right now, if you want to listen to us again, it’ll be in the Saturday newsletter moving forward.

Amanda: Which I haven’t even looked at.

Rob: You’re the second person to know that’s happening. It only happens if we have Sidebar Live that week, so this will be our first week.

Amanda: Okay, good.

Rob: Thank you all for joining us. If you have questions, or questions you want us to ask the community, send them to Amanda at [email protected].

Amanda: This week’s question came directly from a member.

Rob: There you go. Just send them to us, guys. We’re here to help. This is a fun project for us. Have a great week, and we’ll see you next week on Sidebar Live. Bye.