In this episode, we talked with Matt Van Itallie about the rapid integration of generative AI (GenAI) in software development and its profound impact on productivity and business practices. We discussed how GenAI is transforming coding processes, the risks involved, and strategies for mitigating those risks. The conversation also covers how organizations can maximize GenAI’s benefits while maintaining security, legal compliance, and developer oversight in high-stakes environments like finance and health tech.

Key Insights:

• GenAI boosts productivity: GenAI can make developers up to 20% more efficient by assisting with routine tasks and enabling faster experimentation and prototyping.

• Security and intellectual property risks: Despite its benefits, GenAI introduces risks, including security vulnerabilities and potential intellectual property issues depending on how the tools are used.

• Importance of human oversight: Blending GenAI with human input is critical to ensure code accuracy, maintainability, and long-term security.

• Enterprise-grade tools: Using enterprise or pro-tier GenAI tools is essential to avoid exposing sensitive company data and ensure compliance with legal standards.

• Structured visibility and management: Organizations should implement clear strategies and tools to track GenAI use, especially in regulated industries, ensuring responsible and secure adoption.

IoT ONE database: https://www.iotone.com/case-studies

Industrial IoT Spotlight podcast is produced by Asia Growth Partners (AGP): https://asiagrowthpartners.com/

Erik: Matt, thanks for joining me on the podcast.

Matt: Erik, thank you so much for having me.

Erik: Yeah, I'm really looking forward to this one. This is a topic that I feel like everybody is trying to learn in warp speed. Unlike a lot of technologies where you have time to really see how it develops and it might be a decade long journey, everybody's kind of jumping right into this one. So thanks. Thanks for making time. Maybe a good starting point is to understand actually where you came into the topic. Because I know you started your company about seven years ago. And I guess, I don't know if this is true for you. But I think for most people, generative AI was really not on the radar at that point. So I would love to understand a bit the origins of your company and then when did you first start to engage with the topic of GenAI.

Matt: Absolutely. So I was a generalist business executive before I started Sema, working for investor-backed software companies. The original inspiration for Sema is 10 years old now, which is that I was so tired of not understanding what was going on in the engineering department. I like to sometimes talk about CRM tools like Salesforce. Those are fantastic tools for salespeople. They're great for sales managers to help coach and guide salespeople, but they also serve what I call 'executive insight into sales,' which is to say summarizing and contextualizing sales and making it relevant and understandable for the rest of the C-suite and the board of directors managing across and up. That's another function of CRM tools like Salesforce.

Engineering has plenty, hundreds of thousands, because engineers build themselves, build their own tools for engineers. There's hundreds, maybe thousands, of great engineering leader tools, engineering manager tools. When it comes to a CTO tool or a place that a board of directors can go to get just a summary of what's going on in engineering, or the CEO, the summary, it doesn't really exist. I've spent a lot of time with data. I love a challenge that matters. And if you can unblock having engineering leaders and finance and sales and boards talking together, you can achieve so much more. That's what I thought then, and it's most certainly been borne out. So that was the vision of starting Sema.

Because you yourself have an Asia presence, I will also share the story of the name. In wanting to understand code and translate code, to make it understandable what's going on in code, we wanted to base the company's name on a translator to give credence and respect to the creators. The engineers are doing the work. We just want to be able to interpret it. So Sema is a shortened and definitely mispronounced version of "Lokakṣema," who was a Buddhist monk who brought Buddhism from India to China. So obviously, two spectacular cultures and bridging that gap. And so I like to tell you that story, of course, especially with someone with an Asia connection. So anyway, great honor to build this business over the last seven years. In fact, it would have been very impressive if we had built a business beginning in 2017 about generative AI. The truth is, we've been working on one product from the beginning, a code scan product. For American listeners, if you buy a house, you get a home inspection. The home inspection looks at the roof and the plumbing and the electricity and all those different parts, and you do it at a time of buying and selling a house. We've built the code scan to do that about software companies and tech-enabled companies at the time of purchase or a sale. So it's used in technical due diligence. We assess everything at once, almost everything — the team, the code quality, the cybersecurity, the intellectual property risk, et cetera.

And so, over these seven years — I'm so honored by our client list — we've done a trillion dollars’ worth of companies for some of the world's best investors. We've developed some pretty deep expertise in how to synthesize and summarize what is going on with code across all those areas. So we've been working on that now for seven years. But a year ago, some of those great global organizations — I will show respect to them by not naming them — said, "Well, you're good at measuring these parts of the so-called house. Here's another one for you to add." That one is understanding how coders are using GenAI to code. And so that was the beginning of our GenAI product, which has now, I mean, achieved significant market position in this short year, all thanks to our engineers and, I mean, to our customers who said you should build it. A friend of mine out there who's built new products knows what a gift it is for customers to trust you enough to tell you what you should build next. That's the slightly longer version of how we got to generative AI in general and really spending time on generative AI use to code in particular.

Erik: Yeah, fascinating. I was actually having coffee this afternoon with the founder of a Chinese startup where their team was initially a foreign team, their engineering team, and now they're transitioning more over to a Chinese team. They've had all sorts of issues with the code base and so forth. It was interesting to me listening. Because this is a team that's maybe 30 people, 20 engineers, and the CEO was really having trouble understanding what was happening in the engineering room. There's a little microcosm where you'd think, in a small company like that, the CEO should kind of know everybody and have a sense of what's going on. These people are leaving. The productivity is going down. We basically have two teams working in parallel, and he's trying to do this from gut. He's trying to get a sense, right? And so it's very clear. Some kind of dashboard. I know that's probably not the big problem that your customers are facing. That's a specific situation. But you can see, even for a startup founder, how frustrating it can be to not have that visibility into your engineering team.

Matt: Couldn't agree more. He or she is not alone. This is definitely not a salesy conversation. But we have a free, absolutely free, anonymous calculator that actually does a pretty good job if you have the data. I know Erik will say it at the end. But you're welcome to go to our website, anyone out there, and just try it. See what your health score is based on 12 simple questions. If you ever want to get a sense of, really, how after 900 scans, all that work, our sort of summary version of how to understand the health. So anyone who's out there is welcome to try it. We don't collect emails. We don't even collect the data. Totally anonymous. Totally free. Totally for listeners if they just want to see what their score would be.

Erik: I'm curious. There's an aspect of this which I'm going to be more familiar with, which is the management aspect, right? There's a set of use cases around that. Then there's the legal aspect. I noticed that you have a J.D. from Harvard, which is quite unusual for the founder of a tech company. And so there's this whole kind of regulatory compliance, legal risk, set of use cases, which I personally actually don't think about that often. I'm just not addressing those. But I imagine for private equity investor, those could be very much top of mind. How much did that play into you founding the company, your legal background?

Matt: Well, first, Erik, absolutely great observation. If you looked at our — we have a summary, a one-pager, half-pager of the results. They actually are organized by product risk. If there are those risks, it might make it less likely to be able to deliver the code predictably and continue the roadmap. There's something we call compliance risks, which I just say things that lawyers get upset about. It could be cyber risk. It could be security risk. It could be intellectual property risk. And so it's actually called compliance. So you're exactly right to think about that bifurcation.

The path from law to this was a bit indirect. I'm so honored to have learned. Really, law school was a great training in breaking down complex topics and structuring them in a logical way, frankly structuring them in an opinionated way, to achieve outcomes. I think training of laws was amazing for that. Second, a confidence that logic should prevail. It was those two general traits that walking into the weekly executive team meeting where the Chief Sales Officer, Chief Revenue Officer, had a system of record, the marketing officer had a system of record. What do you mean that the engineering leader, despite being an amazing human, didn't have a system of record? And so one, the confidence that, well, that's illogical. We're here to fix that illogical problem. And second, the confidence that we can go figure it out, that we can build a logical structure that is defensible to technologists but also high enough abstracted to make sense to others. So that was really the path.

There is a great deal of legal risk in code bases. I would say it most certainly is. Assume it does not give legal advice, what we have done is listen to general counsels and deal teams who say these are the legal risks we face. Now please collect data that can help us inform how big that risk is, and collect that data and give it to the legal teams. So open-source code, for example, of course, helps developers be wildly more productive. In theory, you can just go off and use it. You can just go to a public repository and use that open-source code. In reality, all of that code comes with a license. All code comes with licenses. Some of those licenses are very permissive. There's commercial licenses where if you pay for it, you can use it. But some of those licenses are very restrictive. Copyleft is the most common one. What it says on paper, in bytes, is if you use our code in your product, our code is free. So your code has to be free. You're copying zero cost, zero price code of them, and you're copying it from the left to your product. That is a massive legal risk as part of due diligence is. Our clients told us we need to care about this. You need to track it. So we track the open source risk, and then we turn it over to the lawyers who are very well-qualified to make the final determination of how much risk is actually involved.

Erik: Great. I want to get into the GenAI topic. But just to understand a bit more from the user experience, it sounds like you probably have a set of reports because you have reporting requirements. Then you also have, I imagine, the need for alerts or to flag information. But then you have quite a range of different potential users, everybody, from the CTO who should be relatively familiar with what's going on but maybe wants a summary, all the way to maybe a CMO who might be very non-technical or a legal counsel, again, potentially very non-technical. What does this look like from a user perspective?

Matt: So it is fundamentally alert in a SaaS platform. We might build that someday, but we really got first this set of reports. There's just three, three sets. A summary that is like a credit score. Your credit score went up or down. It's this number. Here's how you fit relative to others. That's called the snapshot. S for short. S for snapshot. H for huge. H for health check. A 100- to 700-page PowerPoint document, all this is automatically-generated. That is for the technologists. It is definitely not for the CEO, or CMO, or whatever. Then detailed worksheets especially on the compliance risk. So everybody looks at the executive summary credit score snapshot. Technologists look at the detailed PowerPoint health check. Then the legal teams actually do look at the Excel worksheets for the legal risk. Security teams look at the security worksheets. So it's all part of one product. But you're absolutely right. I mean, my God. I was the original use case. Is it good or not? Is it heading in the right direction or not? What areas are hotspots, right? That was the place I wanted to start the conversation. And so, other than lawyers, the non-technologists spend almost all their time on the snapshot side.

Erik: Got you. Then you have these two — I don't know. It sounds like they're new, maybe over the last 12 months or so — these two new products or modules, which are the Generative AI Code Transparency and the Generative AI Bill of Materials. I would love to understand more the concepts behind those. So maybe first the code transparency. The goal, as I understand, is to measure how much code is created by GenAI. Why is it that important? Then I guess there's probably more to understand around that, maybe not just the volume but probably some other metrics. So what's actually important to track there?

Matt: Amazing. So you did an absolute great job of capturing it. Let's go back to the open-source example just for a moment. Open-source code is code the team didn't write. It's an incredibly good idea to use it. I mean, if you ask a developer in 2024, do you think you should be using open source? Do you think you're still a developer if you're using open source, they'd look at you like you're nuts. Of course, you're a developer. We're just using the best tools available. Our job is to solve problems, and it would be a waste of time to build something that wasn't available that was already available from open source. So, of course, we should use it. Because developers, it makes them more productive and they like using it, it's a massive boom for productivity.

We've actually looked for how much more productive it is. It's probably 10x. Developers can — not to pick like a provocative word. But because you don't have to build from the beginning entire frameworks, you could go 10 times faster in certain respects than coders of 30 years ago. That said, open-source code – you can see I'm going to make the analogy — it shortly comes with risk. We talked about the legal risk. That's very real, especially in diligence. There's security risk. Open-source code is not fully secure. Of course, no code is fully secure. We're not saying it's bad, but we're saying you have to keep track of it. And so, anytime in a diligence situation but also in ongoing, companies keep track of how much open source they have and make sure it's within range, and the code that they are using, the open-source code they're using, is being used the right way. Literally, you can swap out every single thing every time I said open source for that last three minutes, and replace it with GenAI. Is GenAI a good idea to code? Absolutely, yes. There's a tiny number of situations where a company, an organization, might decide they don't want to use it. We respect that. But all of the meaningful risks are avoidable with good tools and good training. So we really think it is safe to use and all, but you shouldn't be building nuclear arsenal software with it just to be safe. But other than that, it's really safe to use. Almost all developers like it. Not all of them. But almost all like it because it makes them more productive. It's a just in time partner. It's a way to experiment and prototype quickly.

But it comes with risks too. Of course, because it helps their job, it leads to a massive productivity boost. We're only seeing the beginnings of which. It will probably in the end be another 10x improvement just like open source, maybe even more than that, over the years. But it comes with risks. First, GenAI code is less secure. You know, I'm going to say, so is all code. So it doesn't mean you shouldn't use it. It just means you have to be worried about security. GenAI code comes with intellectual property risk. Depending on which tools you use and how you use it, it's a real risk. GenAI code comes with long-term maintainability risk. So, audience, think of a colleague at work who shows up to you with a 10-page paper, a memo that you ask them to write, and you say all right. Steven, did you write this? I put some prompts. I typed some prompts, and this is what came out. Did you edit it after the fact? No. You would be skeptical. You would be skeptical because it probably is right but not completely right. GenAI hallucinates. You'd wonder about this person's ability. Because you're not just asking for a memo. You'd want this person to understand it well enough to be able to act on it.

And so, in that example, in that fake example, an unadulterated memo would be called pure GenAI. So pure GenAI is, it just spits out the results from an LLM. Blending is the act of modifying it, of making it yours. That is the long way of answering your other question. Not only are we looking at how much GenAI and where is it, but how blended is it? Because it is the blending of it that lets you capture all the benefits while truly eliminating all of those risks. That is actually the one difference to open source. Where open source, you keep it safe by keeping those packages updated. So you're not modifying it yourself. You're not blending it. You're just keeping the patches going, keeping the patches updated. Whereas for GenAI, you absolutely need a human in the loop. You need them to modify it, to understand it, to make it contextual, et cetera. And so GenAI code transparency is one part how much GenAI is it and where? But the second part, how blended versus pure is it? It's the combination of those two factors that really help companies, help organizations, take advantage of GenAI while mitigating the risks.

Erik: I'm curious around the — there's a lot of Gen AI tools. Everything from ChatGPT, right? I imagine most developers are not using ChatGPT, but everything from that to GitHub or more specialized tools. I guess they all have different risk profiles. Do you track that? I don't want to call out any companies and kind of put them on the spotlight here. But do you have a very clear understanding that product X has this risk profile and product Y has a lower or a higher risk profile based on the toolkit and the way that it generates code?

Matt: I'm going to give you a half answer. The half of that I'm not going to answer is to say there are a large number of extremely good tools, of GenAI tools. In fact, developers do use ChatGPT as well. Many development teams let their developers use more than one because some of them are fit for purpose for different tasks. So many different companies have extremely good GenAI tools. The risk comes from picking the right tier of those tools. And pro tip, I really hope everyone listening, please go have a conversation with your legal team and your CTO as soon as possible. It's incredibly important that your developers and, frankly, everyone at the company is using a tier of the LLM tool that is business, or pro, or enterprise and does not retrain on what you are putting in. That is fundamentally important to protect the trade secret version of intellectual property of your company. Because otherwise, it is sort of the equivalent of posting your code on Stack Overflow or on the Internet. It could be reused. So I really can't stress this enough. Please give your developers, everybody who wants to use LLMs, at least one tool at the right tier and help them understand. Like, if you wouldn't post it on LinkedIn, whatever you're doing, you should not be posting it. Even questions, right? We're thinking of entering this market. We need to solve this problem, right? Those are business secrets too. Do not use a tier of a product where they guarantee that they're not going to retrain based on your inputs.

Erik: Yeah, I mean, here in China, I mentioned — I don't know. Maybe you don't have so many users here — the cloud adoption is quite a bit lower, and the trust level is quite a bit lower. So we really see that risk aversion quite extreme here. Where even if it's the Microsoft Azure version of OpenAI and their cloud services are all on Microsoft, they might still hesitate about using GenAI through that. Because we trust Microsoft. But how much do we trust them with certain data sets?

Matt: I'd say, for folks, yeah, at that level of skepticism, they're not alone. Military grade applications around the world have that. Then I'd say, first principles, GenAI and the SDLC can make your developers at least 10%, probably 20% more effective today once it's rolled out at scale, and developers are using it, and it's not forced on them but they've gotten the chance to use it, et cetera. For most organizations, if you could get 10% to 20% more productivity, that is so spectacularly valuable that you should act accordingly. Acting accordingly in certain countries might mean just going and buying a copilot license, whatever. Acting accordingly in certain other industries in other countries might mean go buy a private set, set up one for yourself. So certainly, there are private LLMs that can then be purposely built for making coding improvements. I just encourage everyone to do that math, every organizational leader. What is the value of 10% to 20% productivity gain? The cost would be higher if you have to put in a private LLM that is not available to the outside world. But there are — I'm going to make a wild guess — 20,000 companies in China and probably 30,000 organizations for whom the ROI would be clear enough to go put in private models, given how valuable it would be for your developers to have that kind of boost.

Erik: Yeah, and I can imagine this type of tool being really valuable for their conversations. Because I don't think this analysis is coming out of really a deep assessment of the risk. I think it's coming out of a lack of visibility into the risk and just saying we don't understand it. Therefore, let's not act right? And so having a tool that can provide a structured visibility and a way to communicate to the legal team, communicate to headquarters about risk profile, let people know that it's being tracked, I think would really be helpful there.

Matt: I'm sorry. Well, just to say, we've written a bunch of white papers. We could not be more supportive of using GenAI code. You should know our biases. Everyone should know your biases when you're talking to somebody. Our white papers reflect it. We try to be objective, but we are most certainly an independent voice. And so we're certainly happy to share them. They're on the website. People can contact us. We are happy to make introductions to sales leaders at leading LLMs who can explain why their tools are safe. Certainly, every organization has to make its own decisions. But certainly, in the United States, we are — not only would I bet my company. I have bet my company that there are enterprise grade tools that you can trust with your most important secrets and would have no personal concern about recommending them. Obviously, it's a country-by-country decision. But if folks want to read the white papers, reach out. If folks want introductions to the LLM sales teams, you let me know.

(break)

If you're still listening, then you must take technology seriously. I'd like to introduce you to our case study database. Think of it as a roadmap that can help you understand which use cases and technologies are being deployed in the world today. We have catalogued more than 9,000 case studies and are adding 500 per month with detailed tagging by industry and function. Our goal is to help you make better investment decisions that are backed by data. Check it out at the link below. Please share your thoughts. We'd love to hear how we can improve it. Thank you. And back to the show.

(interview)

Erik: One more aspect of the risk profile that I'm really interested in your thoughts on is the topic of data. What data do you use to train an algorithm? So one, this topic of developing code but, also, what databases do we put there? This is certainly relevant for putting anything on the cloud. I think a lot of companies, at least the companies that we're working with, even on premise, there's kind of question marks about merging data sets. Often, data is siloed. Maybe it's an inefficiency. But there's, to some extent, sometimes some logic around siloing it so that the pool of users is smaller. Do you measure the risk related to databases that would be used to train an algorithm?

Matt: That's a really good question. We do not do that today for this product. On the relationship of training data to risks for GenAI coding, so that's an area we know a little bit about. In theory, there is a risk that code of basically copyright infringement. So if I work at Acme Corporation and Acme Corporation was using LLM Inc., from a company called LLM Inc., and they were training on data, in theory, LLM Inc. could face copyright infringement claims from the creators of the training data. We're, in fact, seeing that today with lawsuits against some of the major providers, LLM providers. Not just on code but in other areas.

What I'm about to say is not legal advice, so do not take anything I'm saying without a strong conversation with your counsel. Sema assesses the risk of copyright infringement claims to Acme Co. for using an LLM to code as extremely low. One reason is, it's a very disparate set. There are millions of potential targets. There are millions of organizations who are using LLMs to code. And so the training set provider, it's one thing to go after the LLM, of which there's a handful. But to go after all the users of the LLMs is a very disparate path. Second reason, if you use the right enterprise tier of the LLM, they put in indemnification for proper use. And so that's another big reason. We still think the loss at risk is extremely small. Again, not legal advice. And I'll end it with, the folks who are bringing these claims are open-source community members. I would always, always, always, always recommend that organizations be good contributors back to the open-source community at the level that they can afford. It's a good thing to do, but it also buys you some goodwill. If you're using LLM tools, you're indirectly using open source because it was used in training data. But there's plenty of direct ways too. So whether that's contributions to the open source code that you are using, whether it's financial contributions to those teams to support them, these incredibly hardworking volunteers, be a good citizen of open source, of the open source you know about. That would also serve as a mitigation against organizations, at least to some extent, who might be looking to come after end users.

Erik: Got you. Okay. Well, that's good to know. I mean, I guess there's a bunch of other topics which might end up being extension products for you. We don't need to go into them in detail. But I mean, all sorts of issues. Like we have an algorithm that we trained with our corporate branding, and we want to auto generate all sorts of marketing materials that are based on our branding. How do we make sure that those are all compliant and so forth? So a lot of compliance issues. I think those are not in your portfolio today. Maybe extensions.

Matt: Yeah, although, Erik, I will say in terms of siloing, a story that's really stuck with me, we're deep in this product. But of course, spending time listening about AI case studies. We had a CTO of a 10,000-engineering organization say, "There are limits to how much data I'm going to put into our internal, private LLM." It's not about the retraining. It's just internally. Because if I have 10,000 co-workers, there will be at least one bad actor. Just law of large numbers. That must be mathematically true. Of course, I believe this person. But for sure, right, we presented to a pharmaceutical conference about two months ago. That was just about general GenAI risks and not just about code. One of them is about intellectual property risk, whether it's code or other things. Another is about training data, not for code but for biomolecule information. But also, one is just about data and data protection and privacy and security. It's just like the same but harder. Because now there could be — it was just like sexy and fun. Everyone's thinking about, well, what if we combine all of our data? Well, the bigger the target, the juicier the target, right? I don't think that the rise of GenAI is producing a philosophical difference about the risks of consolidating data from misuse, or misappropriation, or theft. But practically, it sure is because of the volume. And so it ends up being something that folks have to redouble their security measures and be super intentional about.

I'm a historian, right? This is GenAI. It's one more tool in the history of tools. It's a very powerful tool, but they're just tools. And as the tool power increases, you have to adjust your working approaches and your relationship to that technology and the security practices you need to follow. It is different, but it's not. You have a checklist of things to do to keep your company safe. Understand how you should modify that checklist to use GenAI, whether that's GenAI to code, whether it's for marketing materials, whether it's for underwriting or molecule discovery. It's all data being used to solve problems.

Erik: Yeah, very interesting. Yeah, it's a constant battleground, right? I mean, a lot of big numbers, right? Somebody is out there thinking about how they can either manipulate or create an LLM coding tool that will input nefarious code into code bases. Somebody's working on that problem. A couple other products, or maybe these are more features, but I'm curious how you think about them. One is that you have these industry-based benchmarks to contextualize risk. We've already discussed that if you're doing something with nuclear, maybe military applications, okay, there, you have a certainly very high-risk profile. Beyond that, if you just look across normal commercial industries, how significant is the variance in risk profile across industries?

Matt: I would say, from our research and hundreds and hundreds and hundreds of conversations, there is a greater amount of in — other than, say, defense, which I think stands alone, there is more industry variation than across industry variation. Which is to say, we've seen within finance, some folks don't want GenAI in their most important repositories. Some folks are comfortable everywhere. In health tech, some folks don't want GenAI. I mean, there's very few organizations at this point, private sector organizations, who say not family, crowned jewels. So that translates. For medium and lower important code, it all matters. But that code, there's, I think, a general understanding that GenAI is safe. Really, where there are folks who have genuine concerns are, should we treat a subset of the code as off limits to GenAI? And that there aren't industry patterns. It's specific to individual executives and teams' preferences. We expect that will go away over time, that sort of like open source. You might use open source more carefully in fintech than social media. No disrespect to social media. But everybody uses open source. You're just more careful about it. I think we're getting to that stage sooner rather than later. Because you can put in all the security measures you need to keep your highest risk assets safe, just like you can when you're using other engineering tools.

Erik: Got it. Yeah, I guess that's one of the circumstances where there's just not a clear best practice today. So people are kind of relying on their gut a little bit, in their risk sensitivity. Then over time, those best practices will emerge and it'll start to standardize more across the industries. There's another feature that I thought was interesting, which is heat maps that alert where GenAI code is being used in an organization. So I guess, I mean that implies something that we should all assume, which is that Gen AI is being used, whether you know it or not, by coders who are overworked. They're going to maybe use GenAI which might not be from the tools that the company has sanctioned. How do you look at that situation? I guess the ideal situation is that everybody is using a sanction tool and kind of a tool that's in your database. If somebody is not doing that, is your system able to still pick up? Because I imagine it would be, to some extent, quite challenging to know that code was developed by a GenAI if somebody is kind of cutting, or pasting, or somehow merging it into a code base.

Matt: It's a great question. Actually, all of our GenAI detection is prediction. It is not yet feasible to deterministically determine whether or not GenAI as an outsider outside of the LLMs themselves, to determine what code is Gen AI or not. So we have a prediction engine that's now in its 150th iteration. It's a really hard engineering problem. Shout out to our spectacular researchers who are working on this. The upside is, it is tool agnostic, and we can predict regardless of which tools teams are using. Certainly, organizations who what they want is to not use GenAI, either in some places or at all, can get notification that GenAI is being used as a way to mitigate that. That is a pretty rare use case, which was not my hypothesis when we started this product line. My hypothesis would be, organizations who have a formal policy to not use GenAI would like to know where GenAI is not being used. That is not true. That has not been borne out. Instead, I would characterize it as, organizations who don't want GenAI to be used would not like to create an obligation to act on that.

So it is true that almost all developers are using GenAI. It is also true that I think 15%, give or take, of organizations have banned it. So there are some organizations where it's being used, where it is certainly not part of the formal policy. Much more common is using GenAI co-transparency to understand where it should be increased. We're using it, but we want to use it more and to make sure that it's blended enough so that it's preventing those risks. Those are, by far, the earlier term cases for developers, for companies using the tools for ongoing usage. Now, in due diligence and other moments of valuation, the world's best investors are checking whether or not GenAI is there. So it kind of doesn't matter if you say you have a policy or not. You're going to have to explain your usage at that moment. I really think in the next 6 to 12 months, there'll be a smaller and smaller set of companies who don't want to use it. They'll have finished their pilots. They'll understand the use case, the benefit. They'll have built, put in the right tools. And so, almost all of GenAI code transparency will be about supporting responsible adoption.

Erik: Okay. But that's really interesting. I was assuming that you had kind of a plug in that would understand that a developer on their computer was using the GenAI tool and would kind of somehow input itself into that flow and say they're using tool X. I can understand why that would be very challenging, because people are working on different computers and so forth. So you're not doing that. You're working from the code base, and you're using a predictive model. So even if somebody, yeah, no matter what hardware and what platform.

Matt: No matter what tool. Exactly right. Exactly right.

Erik: So last question. There might be some other things that we haven't touched on. But last question from me would be, what's in the pipeline? I think this is a very new domain, right? So I'm sure you have a long list of requests from customers. But if we just look over the next 12 months, what's becoming out of your pipeline in this space?

Matt: Yeah, I feel very lucky to have a product. This product, we serve boards of directors who need to know the bottom line. We serve CTOs who need to know trends. But we also serve developers. So unlike the code scans product, this tool helps developers as they are coding get feedback on how much GenAI is being used. They use it as part of their code review process, just like any other check. There are few things more exciting than building products for engineers. Because they are very smart. They have a lot of choice, and they are really good, really candid about sharing what they don't like. That is a treat. Some users don't share their feedback. I just think of several pieces of feedback we've gotten over the last few weeks about what folks want. It's a gift to meet the standards. The state of engineering tools is such that you just have to be excellent and work towards excellence every day. And that is so fun. It's so fun to do. So just the nuts and bolts of building an absolutely stellar developer experience will keep us very, very busy. I know it's a big part of our roadmap.

Of course, we're then also simultaneously increasing accuracy over time. It's at commercially reasonable levels, but it can always get better. Then one of the things where we're really excited about — any of your listeners who want to weigh in on this, we welcome product feedback — is we're building a productivity model to show how much more productive the team is as a result of using GenAI tools today, and what percentage of the maximum productivity gain is your team currently generating. So you could be 5% more productive right now because of your GenAI usage. Maximum productivity gain, given the specific use case in your company, is only 6%. And so you have captured 5, 6. It's 83% of the possibility. By contrast, if you're 5% more productive and the data suggests you could be 20% more productive, then you've only captured that 25. So being able to capture how much more productive are you today, what is the maximum current state of productivity gain, and giving teams coaching, automated coaching, on how to get there is a near and medium-term challenge that's been a lot of fun to work on.

Erik: Okay. Yeah, fascinating. Well, I'm sure you have competitors. This is always a competitive market. But it seems like you're really in a space where everybody over the next two or three years will end up deploying a tool to address this, right? This is really an incredible ramp up, right?

Matt: I love analogies. That's another thing law school taught. If you look at, anyone who's skeptical, look at how people treat open source. Look at where they monitor it. Look at when developers have to keep track of it. Look at when stakeholders use it, and look at the results when lawyers care about it. Ask yourself what is similar and what is different about GenAI code and open source. If you see it like we see it, that they're really common, you should expect that GenAI compliance world will evolve just like open source. And if that's true, then, absolutely, every company that's medium or up will be using AI composition in the years ahead in an ongoing way to prepare for these high stakes moments where they're going to have to explain the composition of their code.

Erik: Great. Well, Matt, I'm sure there's some of our listeners that would like to use your tool and probably some that would like to also maybe pick the brains of some people on your team. What's the best way for them to get in touch with your team?

Matt: Absolutely. Listeners of this show get direct access to me. So feel free to go to semasoftware.com. If you click on any of the buttons, on Contact Us and ask for Matt, or if you prefer, MVI, my initials, it will definitely get to me. You can also peruse the site and see some of the white papers. Although I'm happy to show you around, folks who'd like to try out AI code composition on their own, we have free trials set up on your own. Give it a shot. Of course, always happy to continue the conversation. So if you've been intrigued by anything you've heard here, please feel free to reach out.

Erik: Awesome. Thank you, Matt.

Matt: Erik, thanks for having me.