# This Company Just Unleashed a NEW Form Of AI (Liquid Foundation Models)

## Метаданные

- **Канал:** TheAIGRID
- **YouTube:** https://www.youtube.com/watch?v=FsFvDHSgG_s
- **Дата:** 24.10.2024
- **Длительность:** 55:20
- **Просмотры:** 39,106

## Описание

Prepare for AGI with me - https://www.skool.com/postagiprepardness 
🐤 Follow Me on Twitter https://twitter.com/TheAiGrid
🌐 Checkout My website - https://theaigrid.com/


Links From Todays Video:
https://www.liquid.ai/liquid-foundation-models

0:00 Foundation Models
0:28 Efficiency Features
0:59 Company Vision
1:27 Liquid Engine
1:58 Real Applications
2:25 Current Limitations
2:44 Company Mission
3:16 Performance Focus
3:45 Model Release
4:30 Neural Networks
5:08 Memory Usage
5:37 Model Performance
6:00 Model Recognition
6:46 Model Updates
7:18 Future Products
7:41 Product Library
8:10 Company Position
8:34 Innovation Areas
9:07 Quality Analysis
9:29 Evaluation Methods
9:58 Quality Aspects
10:27 Efficiency Focus
10:56 Framework Design
11:34 Design Approach
11:52 Development Process
12:20 Evaluation Examples
12:42 Training Progress
13:04 Quality Comparison
13:31 Memory Performance
13:57 Quality Results
14:46 Training Process
15:09 Model Creation
15:44 Training Results
16:26 Japanese Model
16:55 Language Forms
17:29 Model Types
17:48 Biological Applications
18:28 Bio Performance
19:22 Physical World
19:59 Drive Form
20:35 Transaction Analysis
21:41 Fraud Detection
22:28 Multi-Modal Features
22:57 Time Series
23:24 Financial Analysis
24:27 Developer Kit
25:03 Model Building
25:45 Transaction Monitoring
26:41 Model Training
27:32 Model Accuracy
28:35 Interface Examples
29:24 Time Analysis
30:34 Model Capabilities
31:03 Development Tools
31:40 Architecture Details
32:25 Pipeline Example
32:51 Building Process
33:17 Operator Level
34:16 Block Structure
34:58 Development Kit
35:35 Training Pipeline
36:28 Model Distribution
37:04 Model Explanation
37:35 Explain Function
38:10 Vision Model
38:56 Model Output
39:22 Output Analysis
40:35 Model Release
41:03 Edge Deployment
42:10 Future Applications
42:35 Use Cases
43:33 Scale Overview
44:02 Device Support
44:58 Model Demo
45:39 Offline Features
46:33 Voice Assistant
47:24 Model Performance
47:53 Speech Model
48:40 Device Testing
49:31 Creative Writing
50:45 Model Example
51:43 Support Agent
52:31 Speech Input
53:12 Deployment Demo
53:59 Final Overview

Welcome to my channel where i bring you the latest breakthroughs in AI. From deep learning to robotics, i cover it all. My videos offer valuable insights and perspectives that will expand your knowledge and understanding of this rapidly evolving field. Be sure to subscribe and stay updated on my latest videos.

Was there anything i missed?

(For Business Enquiries)  contact@theaigrid.com

#LLM #Largelanguagemodel #chatgpt
#AI
#ArtificialIntelligence
#MachineLearning
#DeepLearning
#NeuralNetworks
#Robotics
#DataScience

## Содержание

### [0:00](https://www.youtube.com/watch?v=FsFvDHSgG_s) Foundation Models

we're building the most capable the most efficient AI systems that they you haven't seen before at every scale liquid foundation models are a new generation of generative AI models developed by liquid AI these models are designed to offer improved efficiency and performance compared to traditional AI models such as those based on the Transformer architectures now there are some key features of liquid foundation models for

### [0:28](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=28s) Efficiency Features

example the efficiency and performance is state-of-the-art while maintaining a smaller memory footprint this makes them suitable for on device applications and reduces Reliance on cloud services which can lower costs and reduce energy consumption and unlike traditional models lfms use a novel architecture that is more fluid and adaptable allowing them to process information dynamically and efficiently making them more capable of handling various tasks such as natural language audio analysis and video recognition overall there's a

### [0:59](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=59s) Company Vision

ton of stuff going on with liquid foundation models and in this video I'll dive into their recent webcast where they explain the future of AI and liquid foundation models and this kind of AI that is truly disrupted to make the leap from what's possible with AI today to push Beyond limitations like privacy computational power and the confines of the cloud we've built something revolutionary something Dynamic fluid liquid the liquid engine is a true innov

### [1:27](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=87s) Liquid Engine

ation that will allow every company to own their own intelligence it allows every step of model design and training to be tailored to the specific needs of your company whatever they are today or in the future models that are more memory efficient more explainable and higher quality giving you the power to scale up or down at an affordable cost making what has up to now been impossible possible like enabling a

### [1:58](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=118s) Real Applications

autonomous drones to find brushfires before they become wildfires analyzing a patient's entire medical history against their unique genome or detecting manufacturing anomalies using multiple modes of sensor data whether you need a model that fits entirely in the palm of your hand or models that will solve the world's biggest problems the future of AI should not be limited it should be

### [2:25](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=145s) Current Limitations

liquid so let's take a look at the first part of the webcast there was a 5 minute segment that I trimmed out because he basically explained all of the current limitations of AI things like the carbon footprint things like the scaling laws and how they're going to continue and then he reaches a point where he says look this is why we've developed a

### [2:44](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=164s) Company Mission

liquid Ai and this continues here we built liquid AI to actually enable us to get access to generative AI in a smarter way we want to enjoy the benefits we built the company with the mission to build very capable and very efficient AI systems that are general purpose at every scale so if you look at this Vision that we have we want to get smarter about scaling properties of this Smalls this

### [3:16](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=196s) Performance Focus

trend is there is not stopping but what we are trying to do is that can we do more with less that means if we are in smaller parameter regime can we unlock better capabilities with AI if you think about it from first principles and we did just that we built something that we call liquid foundation models we released three weeks ago the

### [3:45](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=225s) Model Release

first patch of language lfms these models actually elevated the scaling laws that we've seen before into regimes that higher quality models are now possible at smaller scale at every scale we release a 1. 3 billion parameter model we released the tri billion parameter language model and a 40 billion mixture of expert system that allows you to have this technology at every scale enabling general purpose Computing be it a chat applications be it a reasoning or a mathematical kind of question or if you want to put it in production for performing a task for you

### [4:30](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=270s) Neural Networks

liquid foundation models are neural networks they're large neural networks that their computational units are rooted in control theory and in U signal processing and this is a completely new way to look at AI systems with lfms we want to change the base of AI so another property of lfms is that we not just looking at increasing the quality of models but we want to also be better and more efficient at the same time so usually when you use a

### [5:08](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=308s) Memory Usage

foundation model you are giving it a couple of like context and then the model generates for you more kind of tokens or outputs and then that tokens are increasing as those tokens as you use the model the amount of kind of information that you actually extract from the model you use increases now that would lead to a much higher kind of memory capacity requirement on device that's why we need

### [5:37](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=337s) Model Performance

higher and higher or better and better Hardware but at liquid we enabled something that up to a million tokens you can fit the data that you're using and you're feeding inside the network to map on 16 GB of memory this was something that was not possible before while achieving the quality of the models that we get

### [6:00](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=360s) Model Recognition

we released these models as I mentioned three weeks ago and we became the models became trending on um you know we we're not a consumer facing company we we're serving Enterprises but I mean it seems like the consumer also like love playing with models like right next to Gemini Flash and llama 3. 1 we have had a trending model so I guess today actually if you go to open router as well you would be able to see like our API uh is still in I think and um that's a testimony for us that yes so the work is actually paying off lower energy lower cost the best quality you can unlock from the AI systems I'm very excited to today tell

### [6:46](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=406s) Model Updates

you that we're releasing more we're releasing multimodal elephants we're releasing an audio lfm and a vision lfm the a lfm is just short of a 4 billion parameter model that can fit on a device that can give you seamless audio kind of generation it can do speech to speech and it can perform you know speech to text kind of functionality the vision lfm is going to

### [7:18](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=438s) Future Products

process text and image and can produce context on top of that kind of image there are a lot of applications that we can un lock on the edge and on Prem using this New Foundation models we're going to talk about this today and the team is going to tell you a lot more but there is also more so we're

### [7:41](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=461s) Product Library

going to release a library of elephants we have been applying our technology to solving multimodal data of any nature any sequential nature this data could be biological DNA sequences finan time series this could be um you know in B in any kind of for example in autonomous driving it could be signals that you're collecting for driving we're building a

### [8:10](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=490s) Company Position

full Library as products of liquid and we're going to talk about this the team in a second but before getting there just want to position also the company itself so you should not think about liquid as an AI architecture company we've innovated an AI architecture for sure but architecture is one element of a foundation model company so we are a

### [8:34](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=514s) Innovation Areas

foundation model company we are putting together a full ecosystem of model development we have innovated across not just architecture but on learning algorithms how to train how to pre-train how to post Trin how to evaluate the models we have thought about this deeply from first principles to get into these qualities and this shift that you see in a scaling laws next this is where we take a look

### [9:07](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=547s) Quality Analysis

at Jimmy Smith a founding scientist that talks about the quality and the efficiency of these models so if you want to know just how good they are this is the section to watch from a technology perspective much of what ramine has just shared with you is the result of us building a way to develop highquality and efficient models that do not require you to have to choose between the two so I'd like to take a

### [9:29](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=569s) Evaluation Methods

few minutes to tell you more about what we mean by this and how we think about the problem by quality we mean the capabilities of the model and the usefulness of the outputs it can produce for various tasks we consider many aspects of quality with a few important examples listed here the first is knowledge capacity or given a specific model size for a specific regime we want to deploy in how much relevant knowledge across a variety of domains can be packed into the model weights from the training data this is

### [9:58](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=598s) Quality Aspects

important to ensure the model has a strong base of factual knowledge to be used for various quaries another important aspect of quality that we consider is multi-step reasoning this ensures that the model is capable of breaking down complex tasks into simpler problems that are easier to solve we also prioritize Lan context capabilities which ensures that the model can effectively use information from throughout its context to solve a task this was particularly important for

### [10:27](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=627s) Efficiency Focus

us to focus on because note that just because a model can efficiently process aone context does not mean it can necessarily effectively make use of information throughout that context when we talk about efficiency we think about both inference efficiency which determines the cost and resources required to deploy the model as well as training efficiency which determines the cost and resources required to develop the model and it's important to note here

### [10:56](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=656s) Framework Design

that there have been many prior attempts at developed efficient Transformer Alternatives in fact many of the members of our team have performed foundational research in this area however prior attempts at scaling these prior methods both by many others and ourselves has always led to the obser observation that you have to sacrifice on at least one of these aspects of quality if not many and this is one of the challenges we have been solving at liquid and we have developed a way to deliver highquality and efficient models that do not require you to have to choose between the two and we have accomplished this by

### [11:34](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=694s) Design Approach

developing a holistic framework for high quality and efficient model design given a particular deployment scenario we do not only consider the model architecture which we do and is important but also how everything interacts together

### [11:52](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=712s) Development Process

together but also how everything interacts together from the data the training algorithms and the post-training procedures and how all these interact with the architecture and the scenario we're trying to deploy in and this view has allowed us to innovate in each of these areas and a key driver of this innovation has been our careful design of fine grained internal evaluations which allow us to pinpoint the strengths and weaknesses of different approaches to architecture data and training designs so I'd like to walk through just

### [12:20](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=740s) Evaluation Examples

a concrete example to give you a better sense of how we think about this and let's consider one of our internally developed Lan context capability evaluations which measures the model's ability to use information from throughout its context in a complex way to solve a task and I'll will start by plotting the performance of a strong Transformer Baseline on this metric throughout a training run the y- AIS represents a

### [12:42](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=762s) Training Progress

quality score and the x-axis represents the number of tokens seen during training and we'll note that at the end of training the Transformer achieves what we would consider to be a high quality score and we found that in practice when combined with high scores on our other evaluations this is predictive of strong performance on real world applications that we care about now on the other side we'll also

### [13:04](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=784s) Quality Comparison

plot the memory footprint that the Transformer requires to solve this task and we'll note that of course the Transformer requires what we consider to be a large memory footprint which makes it expensive to deploy but now we can also plot the performance of two previously proposed efficient Transformer Alternatives one a modern input dependent State space model variant and the other a modern gat linear attention variant and we see that

### [13:31](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=811s) Memory Performance

while on the on your right the memory footprint of these efficient Alternatives is very favorable compared to the Transformer Baseline we also notice a large quality gap with the between the performance of the Transformer and these efficient Alternatives and we found in practice this Gap was significant and predictive of very important failure modes when trying to apply these previously proposed efficient alternatives on Downstream task and of course this is just one of

### [13:57](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=837s) Quality Results

our evaluations that has allowed us to identify other important failure modes not previously recognized and this created clear paths for innovation of course the first release of our lfm models are a very early result of these Innovations focusing in on the performance of the one second lfm versus a Transformer we see that the lfm actually improves upon the quality score while maintaining the very favorable memory footprint allowing us to deliver on the cost savings these are the types of challenges we are solving at liquid and this is just a very brief preview to give you a sense of how we are delivering on the promise of highquality and efficient models that do not require you to have to choose between the two now another important core component

### [14:46](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=886s) Training Process

of our framework that I have not spoke much about are our post training procedures so this is where we get onto the posttraining methods which is where they teach the AI a bunch of specific skills after they've done the initial training okay thanks Jimmy Hi everyone so here I'm going to talk a bit about uh the Ping work that we do at liquid AI so

### [15:09](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=909s) Model Creation

during pre-training models are trained on a lot of data to learn General capabilities but we need to tune them to turn them into useful assistance that can answer questions and that can follow instructions this is the goal of post training by creating uh high quality examples we challenge our models to learn how to answer complex questions with multiple steps and with Advanced knowledge but this is an iterative

### [15:44](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=944s) Training Results

process and to monitor our progress we need the best evaluations so in addition to the internal evales that Jimmy talked about we have an end entire stack of evaluation at every stage of the training process these evaluations can guide the pre-training and posttraining process to know when we achieve the right level of performance public benchmarks allow us to confirm our internal scores and also compare our models with our competitors but this is not enough to

### [16:26](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=986s) Japanese Model

make great models we also need to evaluate their chat capabilities how they interact with users to do this I'm really excited to talk about the liquid Arena that we created so in this Arena we have two Anonymous models and they provide an answer to the same instruction then we ask a human to choose the best answer we can repeat

### [16:55](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1015s) Language Forms

this process with a lot of instructions and a lot of answers to understand where models are good and when they underperform but to scale it to millions of instructions and answers we created an automated version of this Arena where we replaced humans with llms this gives us A fine grain understanding of the capabilities of our models

### [17:29](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1049s) Model Types

models all these evaluations provide feedback on areas of improvement for us so for example if we see that our models are not great at summarizing text in German for example we can create better data retrain our model and fix this issue so

### [17:48](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1068s) Biological Applications

to do this we have a custom preference algorithm that can process and learn from Human preferences very efficiently in addition liquid straining recipe also includes distillation and model merging so with distillation we have a big teacher model and a small student model the student directly learns from the outputs of the teacher this effectively compresses the higher performance of the big model in

### [18:28](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1108s) Bio Performance

to a much smaller one which is both cheaper and faster to run then there's model merging this one I'm the most excited about uh so during model merging the idea is that we merge the parameters of different versions of the same model and this creates a single unified high quality model this combines high performance across different tasks and this also allows us to um upcycle our experiments into better models so all of these techniques in the post training World they allow us to create and deliver best-in-class models we also have a strong focus on

### [19:22](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1162s) Physical World

evaluations to Target fine grain capabilities in our models finally for end users this delivers better customer ability to adapt lfms to particular task and domains of Interest a good example of this is the Japanese model we co-created with CTC here we fune lfm 3B on Japanese data and we created the best Japanese Edge model on the market this is

### [19:59](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1199s) Drive Form

confirmed by both public benchmarks and also human preferences we got from the liquid Arena now this is the most interesting part of this webcast that I think personally exists because this is where he discusses the liquid foundation models being used in autonomous driving being used in finance and of course in biology and I think this is one of those talks where you truly get to see the kind of future that we heading towards with Advanced models that are going to be coming and being utilized everywhere in the future thank you Maxim hi everyone this

### [20:35](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1235s) Transaction Analysis

is so exciting because what you've all seen so far today is how liquid is building best inclass language lfms that are state-of-the-art in both efficiency as well as accuracy but today we all know that large language models like llms are extremely overhyped and liquid AI is not a language modeling company liquid AI is a foundation model company we build lfms for all types of sequences and language well language is just one sequence in reality sequences are everywhere in the world and in order for AI to be everywhere in the world we need lfms to be highly capable on all sequences and that's exactly what we're doing at liquid AI today we unveil a suite of lfms that are going to revolutionize

### [21:41](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1301s) Fraud Detection

across domains and industries let me tell you a bit more about how liquid foundation models are revolutionizing biological and therapeutic design we've already seen so far today how lfms have learned the language of our natural language but actually lfms can learn the language of biology as well here you can see bio lfm bfm learns the language of proteins learns how to generate brand new proteins from looking at the entire existence of all of these proteins that exist in the natural world

### [22:28](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1348s) Multi-Modal Features

bfm is highly capable and highly efficient to generate brand new proteins that have never been created before but we can now create them in the lab bfm is so efficient and scalable that we can do this not just for one protein not just for five proteins but at Large Scale across biology and accelerate biological discovery

### [22:57](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1377s) Time Series

and design it doesn't stop with just efficiency bfm scales to capability and accuracy as well bfm is state-ofthe-art best-in-class accurate protein Foundation model at this scale and we are still scaling and improving this model to bigger scale and moving these digital Generations into

### [23:24](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1404s) Financial Analysis

the actual lab Now liquid Ai and lfms not only enable everything from understanding the biological world but we extend to also understanding our physical world as well this is critical because our physical world is so complex it's so Dynamic and everchanging that in order to deploy autonomous systems into the physical world we need to be able to understand and simulate our physical world this is so challenging because of all of these complexities that the physical world entails we built drive lfm to understand our physical world so that we can test and simulate new environments of our real world for these autonomous systems

### [24:27](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1467s) Developer Kit

to be stress test and verified before they go out into reality Drive lfm learns from our world how the world Works to understand the world using a sequence of past video here we look at driving as a particular impactful example to understand the world and then learn how to predict the future learn how to generate brand new futures for every single scene every single environmental condition time of day weather condition and so on bio lfm excuse me drive lfm extends

### [25:03](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1503s) Model Building

to all of these different scenarios and again it doesn't stop here lfms enable us to understand our physical world but also understand our own interactions and behaviors this is so important because when we think about how we interact in our world it's also so personal and so complex every single day each one of you is transacting is interacting with other individuals through financial transactions as well as other businesses and in reality if we look at

### [25:45](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1545s) Transaction Monitoring

this C of transactions identifying and isolating a single fraudulent transaction a single mistake in this massive see of transactions that occur is a classic needle in the haystack problem it's a huge undertaking a huge challenge but identifying this one fraud is one of the most critical things to ensuring and safeguarding our personal finances and the finances of all Enterprises in fact if we zoom into these transactions what we can really see when we zoom into the sea of transactions we can actually see that every individual transaction is nothing more than a composition of what makes that transaction a transaction how you what you purchased how much you spent when you made the purchase where you were all

### [26:41](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1601s) Model Training

of these go into that single transaction but of course we all know that transactions don't exist in isolation these this one transaction was actually part of a much larger sequence of transactions that you participated in understanding this sequence and predicting what comes next in the sequence is the core problem that we solve with transaction lfm is able to take as input all of your past sequences or excuse me past transactions and be able to predict what comes next which enables us to predict when there was a potentially fraudulent transaction in the mix so that we can identify and safeguard every individual's and every business's

### [27:32](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1652s) Model Accuracy

financial transaction lfm is able to scale towards large scale transaction histories which means that our fraud detection models are very accurate because it learns from all of your personal past history of transactions using the liquid technology because these models are so efficient we can scale to these regimes but also it means that they are highly accurate now what we have seen is that lfms scale to language and far beyond to all of these different sequences but in reality when we look at this landscape we can actually see that lfms are not only efficient and accurate they are highly multimodal which means that we can combine these different sequences together and learn how to talk to sequences we can leverage

### [28:35](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1715s) Interface Examples

language on one side and go into an lfm and also communicate and talk with other sequences here for example I'm going to show you a demonstration of how we've built a time lfm a Time series lfm that you can communicate with directly with natural language time LF is able to ingest both language as well as time series which is a highly structured and very complex data type let me show you an example time lfm comes with a very simple interface because it's rooted in natural language but has a backend of Time series you can directly describe what you want to search for or identify

### [29:24](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1764s) Time Analysis

with time lfm for example here identifying key financial Trends using the language model to query in a multimodal fashion all of the underlying underpinnings of our time series model it doesn't stop there we can identify these Trends but we can also use time lfm to analyze key and very precise time and metadata information but of course the most important part is because this is not just a wrapper on top of a language model this is a multimodal Time series combined with language model it means we can actually understand the Dynamics the behaviors of what's happening in the time series things are not things that are not only located in the metadata description of the time series that you could get from a Google search but simp but actually going deep into the time series and understanding the behaviors of then interfacing with this through natural language for example here looking at a funny use case of trying to uncover cryptocurrencies that have had Market manipulation all of these are examples

### [30:34](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1834s) Model Capabilities

of how time lfm are revolutionizing the ability to not just have language models that you can communicate in and out with language but can communicate through all of these different sequences that we've been talking about today and this is just an example of how we've been scaling lfms to achieve highly efficient highly capable and accurate modeling that is fundamentally

### [31:03](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1863s) Development Tools

multimodal now how did we do all of this lfms we built lfms using what we call the liquid devkit is a developer package that allows us to build and deploy lfms for any domain and any scale let me tell you a bit about the liquid devkit the devkit sits on top of P torch as well as optimized kernels that we have created at liquid for all of these efficient and

### [31:40](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1900s) Architecture Details

accurate operations it is comprised of a variety of different abstraction levels that allow our internal engineers and scientists to easily build and scale these mod model across a variety of different levels starting first from the lowest level of operators and then moving up to blocks and backbones from there our engineers and scientists can leverage devkit in a very easy to use way so that we can achieve all of these incredible lfms that we're releasing to the world today in just a single line instantiating and building these lfms

### [32:25](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1945s) Pipeline Example

and I'll actually walk you through an example right now of how we can have the entire pipeline of building state-of-the-art examples with the devkit scaling them to large scale highly capable systems and explaining the outputs every single output every single prediction that comes from this model so that we can certify the model for safety let me walk you through an

### [32:51](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1971s) Building Process

example of how we build with liquid devkit the devkit remember I told you is a composition of layers that make it very easy to scale these systems across all of these levels of complexity I'll start first with the lowest level which is the operator level operators allow us to take as input a sequence and learn how to transform that sequence now

### [33:17](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=1997s) Operator Level

operators typically are extremely complex operations they take a lot of very low-level optimized code and kernels that's very hard to not only write but write efficiently the liquid dev kit takes all of this complexity and removes it abstracts away and moves it into easy to use Liquid operators that can be easily instantiated for your application we do this for a variety not just one operator but a variety of different liquid operators that we package together in the dev kit this includes both operators which we have created and invented at liquid as well as uh leading open source highly efficient alternative architectures that exist in the world we can bring this up a level from operators now to blocks allow us

### [34:16](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2056s) Block Structure

to elevate the level of complexity that our models can handle and they do this by a very straightforward gating me mechanisms that are abstracted also into the dev kit in the same way and finally going up one more level Beyond this we can combine these blocks into backbones and the unique thing here is that devkit is able to handle these levels of abstraction and scale them up into the backbone level in ways that goes beyond vanilla Transformer sequence insequence backbones that we see in the world today the liquid devkit allows us to

### [34:58](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2098s) Development Kit

build backbones that have hybridization or feature sharing or depth sharing allowing us to move beyond the State ofth art today this is how we build lfms at liquid and how we work with our partners to build lfms but it doesn't stop there the liquid devkit enables us to not just build and instantiate an lfm but we now need to train it and to scale that model across Hardware so that we can achieve the incredible results that you're seeing today this is done by actually

### [35:35](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2135s) Training Pipeline

enabling highly scalable and distributed training pipelines with the liquid dev kit you can see an example here on the left and right side of building that model that you saw on the previous slides we built an lfm here a very small lfm I'm going to show an lfm that's only about 150 million parameters but now let's ask a question of okay what if I wanted to build a bigger lfm scale this into a multi-billion parameter lfm well this requires me to basically scale the number of blocks because all the complexity is actually abstracted away into the operators and the blocks I can just scale the blocks to scale my lfm so let me increase the number of blocks from 2 to 12 this makes my lfm about 12 billion parameters now

### [36:28](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2188s) Model Distribution

but what happens is actually this lfm if I put 12 blocks in it no longer fits in my GPU so now with devkit we make it very easy to add one extra line using a Dev kit. distribute line and take those 12 blocks and split them to distribute them across our Hardware so that we can scale to multiple gpus multiple nodes and have all of those nodes seamlessly communicate with each other to handle this much bigger

### [37:04](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2224s) Model Explanation

model finally I'll tell you about how we're able to also take these models that we train with the devkit and explain every single answer that comes out of their uh predictions that comes with their predictions this is because liquid neural networks were invented to be highly explainable systems we achieve this explainability through the devkit and we package the using what we call model. exlain model. expain

### [37:35](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2255s) Explain Function

is a very easy to use function that comes with every single model that is built with devkit so any actually comes with this very special function the do explain function do explain takes as input the input of the model but unlike other neural networks other AI systems that just take the input and predict an output explain explains how this output was generated with respect to the input and model it can do this with

### [38:10](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2290s) Vision Model

respect to the model it can do this with respect to going inside of the model understanding what in the internals of the model made this prediction predict what in the data did it see to make this prediction get outputed right and all across the board looking at all of these different axes of interaction that our model has with itself and with its data let me show you one example of how we've used model. exlain with one of our lfms so we built a vision language model this is a model that takes as input both an image as well as a textual description or a textual question about that image and then model is supposed to respond in text about this question so

### [38:56](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2336s) Model Output

we can ask a question to the model about this image and you can see the image on the top right here describe this image in one sentence you can see a couple things in this image the cat is very obvious but if you look a little bit closer you'll see that the cat is about to play with something in front of it there's a small insect very hard to see let's see what happens when we feed this image to our vision language model

### [39:22](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2362s) Output Analysis

lfm here we can see we feed it to the lfm instantly responds and it's able to say and correctly predict that in this image we can see a cat and an insect on the ground okay this makes sense but let's look inside the model let's say what happens when we use model. expain to explain how we got to this prediction for each output that the model creates we can ask model. expain to explain that output starting first with the very first token moving on to the image once it says in this image we can see that the model starts to expand its focus a little bit and then it talks about the cat it gets to the cat and you can see that now the focus has shifted it's now looking at the cat it's not looking at other places it is explaining why it's predicting the word cat it's because it's looking right there and then you can see it shifts its focus yet again now looking at the insect which was so small in the image and finally relocalizing itself back into its context understanding that all of these things happened while on the ground and shifting its focus yet again from the cat to the insect now to the ground this is just one example of how

### [40:35](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2435s) Model Release

we are using model. exlain but model. exlain is deeply embedded in all of our models that we build because they come with the liquid devkit now again now today you have seen how we've built these lfms that are highly capable highly accurate highly efficient and across the board multimodal for all of these different domains and industries today we release

### [41:03](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2463s) Edge Deployment

this Suite of lfms to the world and also we're going to show case how actually this Suite of lfms one of the most empowering things I think about liquid AI is not only that we build these lfms for the world but we deploy them to the world at all scales and one of the most powerful aspects of our lfm library that you can see here is this part about the edge lfms today and I'll pass to Matias who's my co-founder and CTO of liquid AI who will tell you more about how we're actually deploying this library of lfms directly onto the edge in a fully private fully capable manner so like he said I think this is going to be the Future Part because this is where they actually dive into how in the future you're going to want your own private offline models that work for specific use cases that are all over the place

### [42:10](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2530s) Future Applications

imagine having a really good 2B or 8B model that is lightweight effective and can perform a variety of different tasks for your specific business this is going to really enhance many customer service businesses and there's a short video that gives you a few examples of where this camp be applied what if to maximize ai's potential we need to solve for its limitations what if a major Bank could

### [42:35](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2555s) Use Cases

accurately analyze billions of transactions for fraud without letting a single bit of data leave their firewall on Prem or a leading Quick Service food company could power automated drive-thru windows in rural areas without relying on the internet what if a chip manufacturer could build a microchip with the power of AI in it what if a regional Telecom company could process and filter their data without sending all of it to the cloud whatever your use case is liquid AI can develop and deploy solutions that Target and overcome the specific limitations of your company with efficient scalable models that turn your wha ifs into what next with liquid AI it's not if it's when just seen in this video why gen AI

### [43:33](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2613s) Scale Overview

on the edge matters and I'm going to tell you a bit our progress that we have made recall the slide from before where we showed that we train uh lfms at every scale uh our currently smallest model at around 1. 3 billion parameters uh 3B model and also a 4 B uh model are currently most capable and largest

### [44:02](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2642s) Device Support

variant and the numbers the parameter size of each of these models was not chosen uh randomly but with a specific Target uh end device to run on in mind for instance the 1. 3 billion model is able to run on embedded uh Hardware the 3B is optimized for phone use and the largest variant to run in the cloud or on Prem uh on a single GPU so at liquid we don't just train at every scale but we also deploy at every scale in the particular we deploy where no one else can so here we have a video of a Raspberry Pi a Rasberry Pi is a $60 cheap uh low power fully fledged computer with the compute capability of around a 10year old laptop

### [44:58](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2698s) Model Demo

and we ported our smallest model the 1. 3 billion parameter model uh to run on even such small uh and low power devices particular asking about the Isotopes of hydrogen was the wrong video before but yeah you we see that the model is responsive and can make use of even like the low power and like the small and cheap Hardware of the respirate so this is the smallest model like which is nice but let's take a step and look at the other side of the spectrum of our currently largest uh model the 40b

### [45:39](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2739s) Offline Features

particular one thing that Jimmy mentioned is that we also look into making sure that our models can make of long context so they are memory efficient but we also make sure that they are making use of all of the context uh that they provided and in particular we can then feed additional data and information that is not in a training data into our model in particular here we can add the web search functionality and augment the model with post training like data that's not in a training uh with set particular look up for my co-founder Alexander am uh look up his profile using the web search which is uh quite nice so these are all uh so far text models like text in text out but

### [46:33](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2793s) Voice Assistant

Alexander mentioned to you that our models are natively multimodal meaning that uh we can have uh conversations with our model uh hello what's your name hello my name is liquid Edge I'm a realtime Voice Assistant here to help you with questions and tasks uh can you tell me one thing to do in Boston except the freedom Trail absolutely one great thing to do in Boston aside from the Freedom Trail is to visit the Boston Common it's a beautiful Park in the Heart of the City where you can relax have a picnic or even play a game of catch it's a great place to enjoy the outdoors and take in the city's vibrant atmosphere uh can you give me one short tip for networking at a tech launch event absolutely one tip for networking

### [47:24](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2844s) Model Performance

at a tech launch event is to be genuine and approachable start by smiling and making eye contact when you meet someone ask open-ended questions to show your interest in their work and to encourage a conversation remember to listen more than you talk and be prepared to share your own experiences and insights don't be afraid to follow up with a LinkedIn connection or an email after the event to maintain the connection uh thank

### [47:53](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2873s) Speech Model

you you're welcome how can I assist you further so this was our three 3B uh 3 billion parameter speech model um and due to the native multimodal support provides low latency with uh after the user provided some um instructions for the model uh today's uh large language models are all running in a cloud um connected through an API but what if um we don't have a reliable internet connection and want to make use of geni or what if um we have

### [48:40](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2920s) Device Testing

privacy and security concerns that we don't want to share our data uh with a potential uh API provider we at liquid um deploy where the device is and not where the connections are instead of telling you about it let me show it to you so here we have our app where we can chat with our offline model let's also turn off the Wi-Fi and ask it something Halloween is coming up soon so let's test the creative writing abilities of our model

### [49:31](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=2971s) Creative Writing

give me three uh AI sorry yeah it seemed Halloween costumes so this is our liquid yeah neural network Necromancer as deep learning Droid and generative adversarial night that's nice quite creative here let me uh like write a scary story about what was the first one neural network neoman the title the neural The

### [50:45](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=3045s) Model Example

necromancers Awakening and yeah I'm not going to read it out but I think you get a point so this is our 3B language model running fully offline on an iPhone that's yeah one of the things we have achieved at liquid uh so far we have seen chat models so basically open-ended uh conversations uh with the model through speech through text and so on but what if we have a consumer a customer application uh where we want actually the model to be tailored to a specific use case and we

### [51:43](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=3103s) Support Agent

created a hypothetical use case uh where we want to use our 3B speech model and we tailored it using a fine-tuning approach to uh a tech support a customer agent the custom this this agent takes unstructured speech input irrespectively of format and translate it into a structured format that is machine processible um and readable by a machine it uh the video also let me just customers reporting that their TV is not turning on and uh they're suspecting this is due to a recent software update and uh customers tried to turn it on and off and unplug in and plugging uh customer's full name is

### [52:31](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=3151s) Speech Input

Robert wmart and could you summarize the issue and give me a short classification so we have seen our my colleague Mark um basically using this demo um taking speech input translating into structured format using the uh the classification abilities of the language model as well as the summarization abilities um but as I mentioned before we at liquid bring the intelligence the geni to where the devices is where the devices are so let's uh run it again

### [53:12](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=3192s) Deployment Demo

but making sure and showcasing that it runs fully name is lily last name bazak uh correct that to b s z a k um customer is reporting uh potential malware infections so whenever they turn on the computer essentially there's only a red screen and uh basically the computer is unusable

### [53:59](https://www.youtube.com/watch?v=FsFvDHSgG_s&t=3239s) Final Overview

and could you provide with a classification please and can you summarize this issue awesome so we have seen now like a potential commercial application of our speech lfm fine tuned on the specific task uh running offline we at liquid uh unlock the power of generative AI on devices that were once Out Of Reach in a fully private and secure manner we have seen that we train lfms and our foundation models at every scale we also deploy them at every scale in particular we have seen it as small as a raspberry we have seen that our models are multimodal taking speech and vision input we have also seen that our models can be fine tuned and tailored to customer applications very easily

---
*Источник: https://ekstraktznaniy.ru/video/13932*