Exclusive: Kong launches AI Gateway to help enterprises govern and scale generative AI

Kong Inc., the cloud connectivity firm, has introduced the overall availability of Kong AI Gateway, offering enterprises with an AI-native API gateway to control and safe generative AI workloads throughout any cloud surroundings. The product, first launched in beta in February, has already seen vital adoption as organizations rush to operationalize generative AI expertise.

The Kong AI Gateway presents a set of infrastructure capabilities tailor-made for AI, together with help for a number of giant language fashions (LLMs), semantic caching, semantic routing, semantic firewalling, and mannequin lifecycle administration. Crucially, it integrates seamlessly with Kong’s current API platform, permitting enterprises to handle their AI alongside conventional APIs.

“Organizations and builders are constructing new gen AI use circumstances to create higher consumer experiences and buyer experiences. However like each expertise on the planet, within the early days of beginning one thing new with leading edge expertise, it is vitally exhausting to scale that in manufacturing with out having correct infrastructure,” defined Marco Palladino, CTO and co-founder of Kong, in an unique interview with VentureBeat. 

Based on Palladino, the Kong AI Gateway is “in all probability probably the most succesful AI infrastructure expertise on the planet,” attributable to being constructed on high of Kong’s current gateway options whereas offering deep AI-specific capabilities. He drew parallels to the early days of APIs, noting that APIs solely actually exploded at scale as soon as the appropriate infrastructure emerged to help builders.

Introspecting AI visitors and offering a unified API

A key differentiator is Kong AI Gateway’s capacity to introspect AI visitors and supply a unified API to eat a number of AI suppliers. “Each different API administration platform, they take the LLM, the API that provides you entry to the LLM, as being one API like every other API,” mentioned Palladino. “That’s inadequate. Kong goes to the subsequent degree.”

The gateway offers immediate safety, compliance, governance, templating, and a lifecycle round AI prompts. It additionally presents “L7 AI observability metrics” to provide visibility into supplier efficiency, token utilization, and prices throughout AI visitors — capabilities Palladino says are unmatched, as opponents “cease on the API degree, they don’t go deeper into the AI degree.”

Monetizing fine-tuned AI fashions alongside APIs

Kong’s unified management airplane, Kong Konnect, will allow organizations to monetize their fine-tuned AI fashions alongside conventional APIs. “We expect that the subsequent frontier to API monetization is monetization of the fashions themselves,” Palladino advised VentureBeat. “As a result of when the fashions are being fine-tuned with particular company intelligence that solely your group has, there’s worth in harnessing and monetizing that intelligence.”

The launch comes amid skyrocketing curiosity in generative AI following the viral success of OpenAI’s ChatGPT. However many enterprises are nonetheless grappling with the way to successfully deploy and govern the expertise. Kong goals to simplify that with its purpose-built AI gateway.

“Each enterprise group on the planet was attempting to construct the identical factor that we constructed,” mentioned Palladino, noting the product was accelerated to basic availability attributable to overwhelming demand. “As soon as we introduced that there was a beta functionality, all of them needed to study extra. We have already got enterprise prospects in manufacturing with this functionality, a few of them really went in manufacturing with the beta model.”

Initially based below the identify Mashape in 2009 and spun out as Kong in 2017, the corporate has raised over $170 million in enterprise funding thus far. The corporate powers trillions of API transactions for over 500 organizations globally. With its prescient guess on AI-native infrastructure, Kong is strongly positioned to allow the subsequent wave of generative AI adoption within the enterprise.

“With a purpose to unlock GenAI at scale, it is necessary that we deploy the appropriate AI infrastructure,” argued Palladino. “Kong AI Gateway is strictly that.”