Replicate wants to take the pain out of running and hosting ML models

Replicate, a startup that runs machine learning models in the cloud, today launched out of stealth with $17.8 million in venture capital backing; $12.5 million of the total came from a Series A led by Andreessen Horowitz with participation from Y Combinator, Sequoia and angel investors including Figma CEO Dylan Field and Vercel’s Guillermo Rauch, while the rest was from a previously undisclosed seed round.

The company was co-founded by Ben Firshman, who led open source product efforts at Docker, and Andreas Jansson, previously a machine learning engineer at Spotify. The way Firshman tells it, he and Jansson came to the mutual realization that AI was accelerating at an “absurd” pace, but that technical barriers were standing in the way of mass adoption.

Enter Replicate, which offers a library of open source models that software developers can run with a few lines of code. The platform can automatically generate an API server for custom machine learning models, deployed on a large cluster of GPUs.

“If you get a ton of traffic, we scale up to handle the demand. If you don’t get any traffic, we scale down to zero and don’t charge a thing,” Firshman explained. “We only bill you for how long your code is running. The alternative is usually deploying models yourself on Amazon Web Services. Typically, you’d have to battle with servers, Kubernetes, GPUs, API servers, auto-scaling and more.”

Core to Replicate is Cog, an open source tool that lets developers package machine learning models in a standard, production-ready container format. Firshman and Jansson developed Cog, which runs on any newer macOS, Linux or Windows 11 machine.

“AI is currently too hard to use for software engineers and you have to be a machine learning engineer to use it,” Firshman said. “Companies and the industry as a whole is being held back by the lack of machine learning experts. We’re making it possible for software engineers to use machine learning with zero experience, with just a few lines of code, so they can build products with AI and apply it to business problems.”

Replicate hosts thousands of ready-to-use models, including text-to-image and image-to-text models (à la Stable Diffusion). Image Credits: Replicate

Replicate isn’t the only one doing this. The startup competes with vendors including Hugging Face and OctoML (and to an extent Runway ML), which collectively have raised hundreds of millions in venture capital. Google, Amazon and Microsoft could be considered rivals, as well — offering their own solutions for developing, launching and maintaining machine leaning models in the cloud. (See SageMaker, AutoML and Azure’s no-code ML tools).

So what sets Replicate apart? Firshman claims the developer experience is “much better,” which of course remains to be seen — after all, Replicate is brand-spanking new. One clear point of differentiation, though, is the expansiveness of Replicate’s AI library. The platform offers diffusion models including Stable Diffusion, models for creating and editing videos, upscaling models for images and various image-to-text and text-to-image models.

Swift, painless deployment is the focus. Replicate’s website promises: “With Replicate and tools like Next.js and Vercel, you can wake up with an idea and watch it hit the front page of Hacker News by the time you go to bed.”

The marketing appears to be resonating with the developer community, which has enthusiastically embraced Replicate over the past few months — at least according to Firshman. He says that the platform’s seen 149% month-over-month growth in active users and 125% growth in API calls since the middle of last year. Enterprise customers include Character.ai, Labelbox and Unsplash.

“We’ve effectively been indexing the growth in generative AI,” Firshman said. “Founders are building tons of new products, investors are investing in it and users are clamoring for all these new things.”

Leaning into generative AI is certainly a wise decision on Replicate’s part. The segment — under which technologies like ChatGPT and Stable Diffusion fall — has seen a massive uptick in investment over the past several years. PitchBook (via Bezinga) reports that VCs funneled 425% more dollars into generative AI in 2022 compared to 2020, with the space reaching $2.1 billion total capital pledged in 2022.

Firshman sees the growth continuing — and Replicate benefitting.

“It hasn’t yet entered the enterprise’s consciousness how much generative AI is going to upend so many parts of their business: customer support, marketing, sales, content creation, and probably other things we haven’t anticipated yet,” he said. “Very soon, customer support will be mostly automated and extremely good — not the terrible chatbots of the past. Creating assets for marketing will be mostly automated. Most of the ads you see will be automatically generated and personalized. Creating assets for video games will be mostly automated. And this is with the technology we have today.”

Replicate wants to take the pain out of running and hosting ML models by Kyle Wiggers originally published on TechCrunch