• Privacy Policy
Monday, February 6, 2023
Data Centre & Network News
ADVERTISEMENT
  • Data Centres
  • Networking
  • Infrastructure
  • Data
  • Media Kit
  • Events
  • SUBSCRIBE
  • Contact
No Result
View All Result
  • Data Centres
  • Networking
  • Infrastructure
  • Data
  • Media Kit
  • Events
  • SUBSCRIBE
  • Contact
No Result
View All Result
Data Centre & Network News
No Result
View All Result

The best practices for building an AI Serving Engine

Beatrice by Beatrice
March 19, 2021
in Artificial Intelligence, Data, Machine Learning
7 0
0
Share on FacebookShare on Twitter

By Yiftach Schoolman, Redis Labs Co-founder and CTO

One of the most critical steps in any operational machine learning (ML) pipeline is artificial intelligence (AI) serving, a task usually performed by an AI serving engine. AI serving engines evaluate and interpret data in the knowledgebase, handle model deployment, and monitor performance. They represent a whole new world in which applications will be able to leverage AI technologies to improve operational efficiencies and solve significant business problems.

AI Serving Engine for Real Time: Best Practices

I have been working with Redis Labs customers to better understand their challenges in taking AI to production and how they need to architect their AI serving engines. To help, we’ve developed a list of best practices:

Fast end-to-end serving

If you are supporting real-time apps, you should ensure that adding AI functionality in your stack will have little to no effect on application performance.

No downtime

As every transaction potentially includes some AI processing, you need to maintain a consistent standard SLA, preferably at least five-nines (99.999%) for mission-critical applications, using proven mechanisms such as replication, data persistence, multi availability zone/rack, Active-Active geo- distribution, periodic backups, and auto-cluster recovery.

Scalability

Driven by user behaviour, many applications are built to serve peak use cases, from Black Friday to the big game. You need the flexibility to scale-out or scale-in the AI serving engine based on your expected and current loads.

Support for multiple platforms

Your AI serving engine should be able to serve deep-learning models trained by state-of-the-art platforms like TensorFlow or PyTorch. In addition, machine-learning models like random-forest and linear-regression still provide good predictability for many use cases and should be supported by your AI serving engine.

Easy to deploy new models

Most companies want the option to frequently update their models according to market trends or to exploit new opportunities. Updating a model should be as transparent as possible and should not affect application performance.

Performance monitoring and retraining

Everyone wants to know how well the model they trained is executing and be able to tune it according to how well it performs in the real world. Make sure to require that the AI serving engine support A/B testing to compare the model against a default model. The system should also provide tools to rank the AI execution of your applications.

Deploy everywhere

In most cases it’s best to build and train in the cloud and be able to serve wherever you need to, for example: in a vendor’s cloud, across multiple clouds, on-premises, in hybrid clouds, or at the edge. The AI serving engine should be platform agnostic, based on open source technology, and have a well-known deployment model that can run on CPUs, state-of-the-art GPUs, high- engines, and even Raspberry Pi device.

Tags: AIdatamachine learningRedis LabsServing engine
Share3Tweet2Share

Related Posts

NAKIVO releases v10.8 with vSphere 8 support

NAKIVO releases v10.8 with vSphere 8 support

January 24, 2023
49
Aerospike connect

Aerospike releases Aerospike Connect for Elasticsearch

January 18, 2023
47
Yorkshire Water accelerates data innovation

Yorkshire Water accelerates data innovation

January 5, 2023
72
Kester Capital acquires majority stake in DC Byte

Kester Capital acquires majority stake in DC Byte

December 20, 2022
100
What can data professionals expect in 2023?

What can data professionals expect in 2023?

December 13, 2022
48
UK finalises deal with South Korea to help unlock millions in growth

UK finalises deal with South Korea to help unlock millions in growth

November 24, 2022
54
Next Post
Acronis DeviceLock DLP 9.0 delivers data loss protection for endpoints

Acronis DeviceLock DLP 9.0 delivers data loss protection for endpoints

ITN Productions Industry News announces new programme

ITN Productions Industry News announces new programme

ADVERTISEMENT
Data centres face three key challenges in 2023
Data Centres

Data centres face three key challenges in 2023

February 3, 2023
52
Instant migration made possible with WhiteSpider
Cloud

Instant migration made possible with WhiteSpider

February 3, 2023
41

Head office & Accounts:
Suite 14, 6-8 Revenge Road, Lordswood
Kent ME5 8UD
T: +44 (0)1634 673163
F: +44 (0)1634 673173

Data Centres

Data centres face three key challenges in 2023

February 3, 2023
52
Cloud

Instant migration made possible with WhiteSpider

February 3, 2023
41
  • Privacy Policy

© 2023 All Things Media Ltd.

No Result
View All Result
  • Data Centres
  • Networking
  • Infrastructure
  • Data
  • Media Kit
  • Events
  • SUBSCRIBE
  • Contact

© 2023 All Things Media Ltd.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
Cleantalk Pixel
This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy and Cookie Policy.