Artificial intelligence is transforming the way we work, but it is also forcing companies to rethink their technology infrastructure. Running AI models, especially generative ones, requires a great deal of computing power, usually backed by expensive GPUs that consume resources even when they are not in use.
This is where the serverless paradigm comes into play. This approach, which has already transformed software development in the cloud, is establishing itself as a strategic option for deploying AI models on demand, with a direct impact on costs, efficiency and agility.
What does "serverless" mean in the context of AI?
Although the name can be misleading, "serverless" does not mean there are no servers. It means the company does not manage the infrastructure directly: it does not need to provision, configure or maintain virtual machines or clusters.
Instead, the cloud provider automatically allocates the resources needed to run the requested function, and the company only pays for actual usage time. As Google Cloud's official documentation explains, serverless applications still run on real servers; it is simply the cloud provider that takes care of provisioning, managing and scaling them.
Applied to AI, this means that models do not have to be active all day. They can remain idle while there are no requests, activate in seconds or minutes when they receive a query and scale automatically if demand suddenly grows.
In other words: AI on demand, without idle servers.
Benefits of serverless architectures applied to AI
Automatic elasticity. One of the main problems in AI is sizing the infrastructure. How many GPUs do you need? What happens if demand triples tomorrow? With serverless, that problem shrinks considerably. The cloud creates as many instances as needed in real time. When demand falls, the instances disappear and the cost drops to zero.
Cost savings. Keeping a GPU cluster running 24 hours a day is prohibitive for many companies, especially if actual usage is intermittent. Serverless eliminates the cost of idle time. For non-continuous use cases such as corporate chatbots, document analysis or occasional technical queries, the difference can be substantial: consultancies specialising in cloud cost optimisation, such as Wring, estimate that for workloads with fewer than 1,000 requests per hour the saving compared with an always-on dedicated endpoint can be between 40 and 80 %. It is not a figure published by AWS, but it is consistent with the model's pay-per-use logic.
Operational simplicity. The technical team does not need to deal with load balancing, hardware upgrades, availability or redundancy. All of that is managed by the cloud provider. This frees up time for what matters: optimising the model and its application to the business.
Speed of innovation. Testing a new model in a serverless environment is a matter of minutes: you upload the function, expose an API and it is ready to use. Ideal for innovation teams who want to iterate quickly without getting bogged down in server management.
Technological sustainability. By reducing unnecessary GPU use, serverless lowers energy consumption and the carbon footprint. This makes AI a more efficient technology aligned with ESG objectives, at a time when European regulations on reporting the environmental impact of digital operations are increasing.
Cases where serverless makes the difference
Not every scenario requires AI running permanently. These are examples where serverless delivers tangible value.
Chatbots and internal assistants. A corporate assistant for employees does not receive constant queries. It can be idle for most of the day and activate only when someone asks a question.
On-demand document processing. A company that analyses contracts, invoices or reports can do so occasionally, as new documents arrive. It makes no sense to keep GPUs active 24 hours a day.
Unpredictable load peaks. During marketing campaigns, key retail dates or accounting closes, demand soars. Serverless scales in seconds or minutes and prevents service outages.
Edge plus hybrid cloud. Part of the processing is done locally, for example on industrial IoT devices, and only when more capacity is needed are models scaled in the cloud using serverless.
Experimentation and prototyping. R&D teams can test models in production without having to invest in clusters that will be underused. At LurNova we support this type of project within our AI agent development and intelligent automation services, where serverless is frequently the starting architecture.
Limitations of the serverless approach
It is not all advantages. There are scenarios in which a dedicated deployment is still necessary.
Large models and the need for GPU. Here it pays to be precise: Amazon SageMaker Serverless Inference, AWS's "pure" serverless option for AI models, runs only on CPU, with a maximum of 6 GB of RAM, according to its official documentation, which expressly excludes GPU support. For GPU inference with consumption that adjusts to demand, since late 2024 AWS has offered the Scale to Zero feature for real-time endpoints, which allows GPU instances to be reduced to zero when there is no traffic, although technically it is not SageMaker Serverless Inference. Specialised platforms such as Modal Labs or RunPod do offer GPU with a native serverless model.
Latency-critical applications. Serverless can add delays on cold start. AWS does not publish a fixed figure for this time, as it depends on the model size, download time and container start-up, and recommends monitoring it with the CloudWatch OverheadLatency metric. Estimates from specialist consultancies, such as the one cited above, put that cold start at between one and five seconds for moderately sized models. For cases such as algorithmic trading or real-time medical systems, that can be a problem, although provisioned concurrency mechanisms exist to mitigate it.
Continuous or streaming processes. If the AI must analyse data without interruption, for example in real-time cybersecurity, keeping a dedicated server can be more efficient than managing constant activations.
Serverless providers and technologies for AI
The main cloud providers offer well-established serverless options for AI, although with important differences between them.
AWS Lambda plus SageMaker Serverless Inference. Lambda handles routing and business logic; SageMaker Serverless Inference runs the machine learning model with scale to zero and pay-per-inference, but it is limited to CPU and a maximum of 6 GB of RAM. For workloads that need GPU with serverless-like behaviour, the alternative within AWS is a real-time endpoint with Scale to Zero, not Serverless Inference.
Azure Functions plus Azure ML. Allows models to be run in serverless functions connected to Microsoft data pipelines.
Google Cloud Run plus Vertex AI. Combines serverless containers with managed models and automatic scaling.
Specialised platforms. Modal Labs, RunPod Serverless and Replicate are platforms built specifically for AI inference with scale to zero, optimised start-up times and per-second pricing for actual usage, including GPU. They are gaining adoption among teams that prioritise development speed and control over inference costs.
Open source frameworks on Kubernetes. KServe is the de facto standard for serverless inference on Kubernetes. The project was formally accepted by the CNCF as an incubating project at the end of September 2025, an announcement made public in November of that year. It offers scale to zero through integration with Knative, support for the main frameworks such as PyTorch, TensorFlow or vLLM, and deployment in hybrid environments. Seldon Core is a suitable alternative when inference pipelines with advanced logic, A/B testing or canary deployments are needed.
The future: AI on demand and hybrid architectures
The trend is clear: more and more models will run on demand, optimising costs and simplifying operations. But the future will be hybrid.
Small and medium models, with almost entirely serverless deployment, fast activation and pay-per-use cost. Large models, with a combination of dedicated instances for critical latency and scale-to-zero mechanisms to absorb demand peaks. Platforms with intelligent orchestration that automatically select the most efficient option, serverless or dedicated, depending on the type of query.
In this scenario, companies will not have to choose a single architecture, but will be able to combine the best of each option.
At LurNova we help you innovate efficiently
At LurNova we believe the future of corporate AI lies in flexible, hybrid and on-demand architectures. We design solutions that combine serverless for occasional queries or demand peaks, dedicated clusters for critical models and intelligent orchestration that chooses the most efficient infrastructure in real time.
This is how we ensure your company can innovate quickly, scale without fear and control costs, without compromising security or service continuity. If you want to assess which architecture best fits your use case, you can explore our AI consultancy and private AI models services, or contact our team directly.
Co-founder and CTO of LurNova, with more than 20 years' experience in developing digital technologies. Specialised in designing secure, private and governed artificial intelligence architectures and solutions for organisations.
