Check out our interview with Vivek Natarajan, a member of South Park Commons and coauthor of the recent paper on Med-PaLM, an adaption of large language models to the medical domain.
Topics:
From India to UT Austin to FAIR to Google
Integration of AI into products
Organizing research orgs in large companies
Applications of AI to medicine
Med-PaLM and the limitations of LLMs
Risks and rewards of AI driven products
Links:
Med-PaLM paper: https://arxiv.org/abs/2212.13138
Follow Vivek on Twitter here: https://twitter.com/vivnat
Follow your hosts:
John: https://twitter.com/johnvmcdonnell
Bryan: https://twitter.com/GilbertGravis
Interview Transcript
[00:00:00] hi, I'm Bryan and I'm John. And we are hosting the Pioneer Park Podcast where we bring you in-depth conversations with some of the most innovative and forward-thinking creators, technologists, and intellectuals. We're here to share our passion for exploring the cutting edge of creativity and technology. And we're excited to bring you along on the journey. Tune in for thought-provoking conversations with some of the brightest minds of Silicon Valley and beyond.
Bryan: Welcome to today's episode. I'm Bryan, and this is John. And today we're joined by Vivek,
an AI researcher at Google who has been working on translating AI and adapting AI to usage for clinical medicine.
He's the co-author on a recent a paper from Google about Med Palm, which is an adaption of the Palm model from Google to the domain of medicine.
We're looking forward to talking to Vivek about all the ways that these models are powerful and useful in select domains like medicine and also their limitations. So we're looking forward to [00:01:00] talking about Spoonerisms, confabulation and hallucination and how all of these words apply for the purposes of AI.
Vivek: Vivek Welcome.
Hi Bryan. Hi, John. Excited to be here. And yeah, talk all things AI and medicine.
Bryan: Cool.
Yeah. Welcome. So, Vivek, just to sort of ground your background you did your undergrad in India, then you went to UT Austin, and then you came out to the Bay Area after finishing your master's degree. Is that correct?
Vivek: Yeah, that's right.
Bryan: Cool. I think we may have overlapped in Austin. I lived there for a number of years. I miss Austin a lot of the time. But I curious to hear about your own sort of migration gradually as you made your way west to California.
Vivek: Yeah. I think Austin's a beautiful little city and I think Bryan you wouldn't disagree with me if I say that. I think the school UT Austin adds to the charm as well. And for me it was like coming from India, which is a warm weather place moving straight to Texas and Austin, which was equally warm, was good.
And yeah I enjoyed the scenery over there. It was a very welcoming environment I would say for graduate students. And I [00:02:00] was also transitioning my major from electrical and electronics, more hardware after, more computer science than AI. And UT at that time felt like a very good place to be in.
Had a number of good professors who were doing some amazing research and natural language processing, computer vision, graphical models and robotics as well. So yeah, I really enjoyed my time over there.
Bryan: Awesome. And you found yourself now working at the absolute frontier, I think, of artificial intelligence at Google.
Tell us a little bit about how your experience, how did you find your, the application of interest or this domain of medicine? What kind of drew you to it?
Vivek: Yeah, it's a funny story because even before deep learning, like when I was doing my undergrad back in 20 20, 20 11, in the final year of my undergrad, we were asked to do like thesis projects or pitch ideas, and the idea that I pitched together with my team was actually an AI doctor. And at that point of time, the planning wasn't a common term, it wasn't invented. So my presentation decks all had, support vector machines and all those kind of things. But I still believed in the potential of the technology because it was very clear that if it did not have tech [00:03:00] and AI scaling of medicine, we are not going to be able to scale world class healthcare to everyone. It was quite obvious to me even back then, and especially coming from a place like India where, the medical facilities aren't the greatest. It's, it's getting there. There's been massive improvements in over the last decade, but reaching, the remote villages and towns is still a huge challenge, I would say.
And it felt very natural to me that tech and AI would be the place to be. And so at the back of my mind, I think that was always a place that I wanted to work in. And obviously I had a huge interest in machine learning and AI back then. I remember back in undergrad we didn't have the best of internet connections, so whatever bandwidth I could secure, I would download these courses from the KelTech professor Yaser Abu Mustafa and learn about machine learning.
And it wasn't taught in our curriculum back then. So it was all on the sidelines, but that grew, that drew me into the field. And so when I came for my masters, it was, I wanted to take as many machine learning courses as possible. And when I joined the industry fulltime again, I wanted to do machine learning.
And I got really fortunate that as soon as I came out of grad [00:04:00] school and went over to Facebook, it was when Facebook AI research was started and I got this incredible opportunity to work. At the intersection of research and product. So I had this nice role where I could take the latest and greatest models from fair and put it into production.
I learned a ton over there. So it involved like learning all these machine learning frameworks at that point of time, torch and Cafe. Not easy to use by any means. But it was fun and, getting them into, products with like millions of users. That was incredible learning. And like when you work at that low level, you learn all the details of these models, both at training time, both at influence time, and, learning about optimizing.
And so that was, it was lucky for me in the sense that I got this opportunity as someone who was a relative. Nobody just had taken a few courses. I did not have a PhD, but that just set me up back in 20 14, 20 15. And since then yeah, I would say just have been very fortunate to work at the frontier of AI and deep learning since then.
Bryan: That's awesome.
John: Yeah. And so you have to work at both Facebook and Google. How are the cultures of those two companies [00:05:00] different and how would you describe them both?
Vivek: Yeah, it's a great question. I think the answer that I would give with respect to Facebook is also kind of outdated because when I was at Facebook, it was still like, you know, a few thousand employees fair was just getting started and fair was, maybe you could even think about it as like a research lab or a startup within this big company.
And My experience over there is, yeah, it was incredible because there are all these stories about like, you know, mark Zuckerberg having the AI team sit right next to him. Those are all two. We used to sit right where like the exact team was there and we would be observers to all the meetings and everything that was going on over there.
And he, at the end of the day, sometimes he would just pop over and ask questions. And maybe, and he was also doing some projects at that point of time. These site projects or like projects were, I think back in 15, his project was like an Ironman kind of thing. A speech recognition system or voice command system that you can, could do like house task for him.
And so it was kind of, fun that the systems that he was using at that point of time to build this voice control assistant or whatever, was actually the speech recognition systems that we were building at Fair and [00:06:00] Facebook AI at that point of time. And so, yeah, high visibility, I would say.
But at the same time I think Facebook is one of those very interesting companies, at least back when I was there, where it did not feel like a big company. It felt like a startup. The culture of a startup . The leadership did an exceptional job in scaling it. And even until 20 17, 20 18 there weren't like, you know, as many processes or bureaucracy in terms of getting things done.
Yeah, the goal was always to like, you know, just shift things. If you have an argument don't waste time quantification, rather just build and ship and show me that things work, and then, yeah, that's it. And so I really enjoyed that culture where it was all about shipping. So I would say that was probably the best part about Facebook.
If I were to maybe say, Okay. What wasn't great, it was that sometimes you get so much lost into the weeds and details that maybe you don't zoom out and look at the big picture. You maybe micro optimizing for certain metrics and you just keep on moving fast where you're doing okay. Like, consider all the potential issues that might crop up as you're making advancements on certain [00:07:00] technologies or building certain things.
And so that has obviously manifested itself in different ways since then. But yeah I think that environment was really awesome for builders. The period that I was there, and especially for AI researchers as well, were really well taken care of, provided with all the resources. I remember there's one particular interaction between the CTO of Facebook and Ross , who's a very famous computer vision researcher, like the builder of object detection, some of the greatest object detection systems.
And it was at one of these gatherings Shep came up and he basically said, I would literally sweep the floor for you to, do whatever you want. And so that was the level of privilege or access to resources that you had as AI researchers had at Facebook, at least when I was there.
Bryan: Is there, oh, is there a difference in the way that I guess AI you feel has been integrated into the products at Facebook versus at Google? It sounds like perhaps Google and I realize, that you're still there, so we can't just, dive into the weeds, but I'm curious, just at a high level, do you feel that one has been a little bit more strategic or meticulous [00:08:00] about the choices of when to adopt systems, of course, and Facebook maybe shooting a little bit more from the hip.
What would you say maybe is a thematic difference in the way that those companies have integrated AI into their products?
Vivek: Yeah, it's super interesting. For a long period of time while I was at Facebook, we did. So the FAIR Computer vision team was probably among the best.
It still is the best, but there were many other areas where it was feeling like we were playing catch up to Google. So I was at Facebook when the, when TensorFlow was released, and I remember one of the most amazing, machine learning frameworks, hacker just saying that, oh, this is something I wanted to build.
But it would've taken me like another year. And if I were to like ask a fairy or a genie to bring me something and put it in front of my house, this is what I would want. And there were like similar reactions when, for example, the transformer paper came out. We were like all shocked, oh my God, how well does this work?
And at that point of time, I don't think people realized how important this transformer paper was going to be. But still it was quite obvious that this was. Going to change things. So for a long period of time at Facebook, it almost felt like [00:09:00] we were like catching up to Google.
Like all the inventions were mostly happening there except maybe in computer vision. I think that has maybe changed a little bit now. FAIR has it's own amazing research in a few areas. think some of the work on proteins that has happened is really awesome. And also the work on embodied agents and habitat, the environment.
I think that's all really cool. With respect to product integration. Yeah, I think one of the cool things was By there was this explicit goal of moving as fast as possible from research to production. So I remember like one of these conversations where I think it was probably Sumit Genal someone who when say, I want to take the model from here and put that into production in two weeks, and actually went out and executed by, enabled that.
So when some of the Mascar CNN models came out I think in three to four weeks, it was in one of the internal build demos. So that was like really, really cool. And so Facebook had, at least on the computer vision side, built up this mechanism or like way to like productionize things really, really fast.
And the, and I [00:10:00] think Pieto and Cafe both played a huge role. Pieto more recently. But cafe's role should not be cafe's role should not be underestimated. Then. Who there at along Andrew? They were all like incredible and I. I think Facebook was visionary on that front. TensorFlow is great but I think the life cycle from research to production is probably longer than say with Pito, at least the version of PTO that I'm talking about back in 20 17, 20 18.
And with the switch to computer vision, Facebook had that incredible advantage where they could like, you know, immediately shift things to the app straight from whatever, like timing here, or Ross or other people at Fairway cooking up . And I think that has slowly caught on in other views as well with speech, with nlp.
But that I thought was incredible. That was visionary.
John: Yeah. That's really cool. Cause actually a question I was gonna ask, and maybe that partly answers it, but it's a challenge at these big companies when you have a kind of skunkworks team that's like doing very cool research and then you have like a completely different product team that's okay, I'm in charge of like serving up a great newsfeed or something.
And then how do you move the insights and innovation from the skunkworks [00:11:00] to that production side team. how did that actually work? I mean, and so obviously like from a technical perspective, it sounds like PI Torch made it easier. There's still, it's still like it's not, it's easier, but it's not free.
And then there's also like kind of a communication challenge cuz it's like, oh well, like what should the PMs be building? And if you're in a research team you might not exactly know what the product really needs and kinda vice versa. People on the product side might not realize like the impact they could have with research.
I'm curious about how that's been solved at places you've worked. And I'm also kind of curious about maybe your perspective on how it should work.
Vivek: Um, I think it depends on the goals of the research organization and the wide area AI organization as well. Right. So, if the charter of the AI organization is like to actually serve the bottom line of the company and be integrated into products as soon as possible Then Yeah, for sure.
I think you need to organizationally be aligned with this goal and build up everything like the communication systems, the infrastructure, everything to ensure that you can rapidly deploy and get feedback and improve the models as much as possible. And I wouldn't say this was uniformly happening throughout at Facebook, but the computer vision team was, I think very unique in that [00:12:00] sense because there were very good relationships between the fair researchers and the applied ML teams and as well as downstream customers.
And they were all like, putting together in the same direction in the sense that they all wanted to have the latest and greatest advances in know, shipped in the apps as soon as possible, but in maybe some other areas like, you know, new Street ranking. I think it took a few years, for example, to transition over from some of the logistic regression models or even for ads to like deep neural nets.
Obviously they were like huge wins, huge lifts and metrics, but it wasn't like two weeks, it was more like two years. And I think that is both a mixture of How the organization is set up, as well as maybe some of these areas you are a little bit more reticent to try good stuff because there's risk associated with it.
Whereas some of the computer vision stuff, for example, were like more playful features or features where if you go wrong, it's okay. Right? Whereas I think if you go wrong with your heads that hurts your bottom line. So you probably don't wanna screw that up and see, we really wanna be medicist.
So at the end of the day, I think if you really care about getting your innovation to people as soon as possible, then I think at all levels of your organization, you need to be aligned. And one [00:13:00] thing that really helped was I think that leadership was great.
I think they finally also re rehabbed so that Research and the applied ML teams were all like reporting to the same V page around. And so I think that also really helped. So yeah, I think you have to be very intentional about your organization if you want to like move fast and deploy.
And I think Facebook got that right for a long period of time.
Bryan: And on that topic I'm curious, we're talking about an article where the application is medicine and obviously what's at stake when you're giving people medical advice is it's, you know, equivalent if not far more serious than anything having to do with the company's revenue.
And what is the kind of ambitious context that maybe caused the research group at Google to pursue the application of medicine? What do you think was the sort of pie in the sky goal?
Vivek: Sure. think Google has always been at the forefront of medical ai. Along with say, the transformer paper and 10 flow.
One of the papers that I was really, really inspired by was this computer vision paper from like my current teammates which showed that you could detect diabetic retinopathy from fundus images as well as [00:14:00] like eye care specialists. And so that I would say was a very big personal moment for me as well in the sense that, okay, that really it, it showed me that, with ai we can do some amazing things in medicine.
And it goes back to the same story where it is very obvious that healthcare systems worldwide have like different sets of challenges. But one of the key solutions to these challenges is tech and AI in particular. So. In developing countries there is just a shortage of like specialists and care providers and probably the best way for us to be able to like scale world plus healthcare to everyone is through ai.
And in places like the UK and the US it's more that we do have providers, but their time is occupied in not providing care, but in everything else around it. And they are experiencing levels of burnout never seen before. And again, AI is the solution to help them have a much better experience in providing care.
So yeah at Google one of the great things is like the [00:15:00] investment in medical care has stayed consistent or increased over a period of time. Different efforts have been made. And that is really inspiring for me. And I would even go out and say that probably the most important application of AI is medicine.
And in the next decade, we are going to have a transformative impact using AI in medicine.
John: I mean, one thing I was curious about is, you know, there's that challenge within a company of like, okay, how do you get the research moved in to like production in medicine it's actually a way more complicated, right? I mean, so you have this diverse group of healthcare providers, you have certain companies that are investing deeply in, in medical ai or researchers that are investing in medical ai. And I'm curious like you've kind of got this front row seat. Like, what does that process look like? Like how does the research turn into clinical.
Vivek: It's definitely an interdisciplinary process right. I think you can't just have engineers and, machine learning scientists working on this. So if you look at [00:16:00] our team, we have expertise. We have some of the best clinicions in the world who have worked, and it's not just from the US but also UK and Australia and a bunch of other places.
We have people who have expertise with respect to regulation, who have worked in FDA or like equal bodies elsewhere. We have like legal folks and everything. And so you need all those perspectives to come in just because of the nature of the field that we are in. And I think it's a mix of what are the most interesting things that you can solve in this place as well as, okay we have this technology where we have this unique advantage or this superpower.
And how do. Make best use of that. And so at the intersection, like the magic, and that's why we kind of focus on, okay, find out what are the most interesting problems that you can solve with this technology that we have access to? And that's how we generally end up like picking the problems to solve or whatever projects that we work on, and generally you are looking for the biggest impact that you can make.
And so the kind of diseases that you go after, if you look at it, they're like, you know, diabetes or cancer or neurological diseases, [00:17:00] which probably have the highest footprint across the world. So if you make a dent over there, then the quality impact, the quality of Jesus life is that you can improve by that's significant then.
So that's how we end up choosing our projects or the work that we do.
John: Yeah, well actually, here's another way to put it. Like for example, if you think about. The work that's already been done in AI with medical applications what are some of the big wins so far and like how did those get into clinical practice?
Vivek: It's a great question. I don't think I would be wrong if I say that actually the promise of AI and medicine has not really translated into real world applications. There's been tons of research papers. I think there's 150 x increase in the number of research papers here in the US since 2016. Yeah.
In the medical AI field in particular. But if you look at, say, the number of clinical trials that's lagging behind. More recently there have been quite a few FBA approvals, especially in radiology for using AI applications. But I would say with respect to the research and the promise and the hype, the [00:18:00] translation hasn't necessarily been there.
The ones maybe that are most prominent so far have been in medical imaging. And I think that's probably due to the paradigm of AI and d planning that we've been using so far to build medical AI systems, which are still based on like, supervised learning, acquiring large amounts of data and computer vision at least.
Still you know, GPT three came out was probably the most advanced field of AI and medical imaging had this nice cleaned up... I wouldn't, okay, not necessarily cleaned up by natural image standards, but generally you had data numbering in the millions from different hospitals, probably easy to like homogenize and clean up.
And so it was very well suited to the supervised learning paradigm. And so that's why you saw a lot of activity and momentum and applications in the medical imaging slash computer vision phase. And so, I would say that's probably the most advanced. We've seen applications in radiology you know, there are different startups doing like breast cancer detection models.
There are and lung cancer detection. And then other modalities like the ophthalmology modality that I talked about, like [00:19:00] diabetic retinopathy, a bunch of other eye diseases that you can predict from kind images. Dermatology. I think there's a lot of startups and who are like building these apps that can diagnosis skin conditions from smartphone images.
Yeah. Ultrasound is another important modality that's becoming prominent just because of the cost effective nature of the sensor. And so you can do a lot of interesting things with it. For example recently from our team at Google, we showed that you could predict gestational age from ultrasound and you can do it very accurately.
And so this is cool because it's a cheap sensor and it's a cheap model that you can put on the edge and give it to community health workers and you can like empower them so they don't need it to have access to an expert Iens yeah. Overall I would say medical imaging has probably been the one field in AI and medical AI that has probably had the most set up advances with re respect to the research that has been done, the number of papers and also the number of products that maybe are going through or have gone through FDA approval.
So that is there. I think EHR is another modality where people have been trying to come up with operative insights from your EHR data. Typically in, hospital like ICU settings. [00:20:00] But one of the challenges is if you work at a typical icu, like we have this recording at Google where we just have something which shows like, what does it feel like to be in an ICU setting?
And so they're like thousands of buzzes, right? And like every minute you're bombarded with notifications and everything. It's really, really challenging. And so if you have an AI system you don't want it to add to the noise, rather it should give you a very unique insight. And that I think is still challenging.
So I would say the applications on that front, like using EHR or to like, predict test or medications or predict some interesting stuff like sepsis monitoring from records or something like that. I think that hasn't been successful or that successful just because of the nature of the problem and also the workflow.
So, It's important that you not only consider how good of a model that you can build, right? But it, I think the key aspect is also to consider the workflow that where the model will sit in. And so you can have a very amazing model but if it is inappropriate in the workflow, then it's [00:21:00] not gonna be helpful at all.
And I think that's the real challenge. Like, if the research is done without you know, accounting for perspectives of doctors or people who are actually on the ground, then you're gonna miss out on this insight. A lot of research that has been done today has probably missed out on this insight.
And that includes things like, for example, selecting the operating point of your model. You want to ensure that you send in the right amount of alerts or notifications or do the right amount of recalls because anything less or more, you're adding more burden to the system rather than actually helping out.
Bryan: You know, we think about the application of models and one thing that's a bit of takeaway for me is we often need a fairly risk tolerant or a fault tolerance setting because we need to be able to, you know, ascertain when the models are making mistakes and we need to be able to offer points of intervention and confirmation from professionals who are practicing.
I'm curious if we're thinking about the balance of kind of opportunity to improve people's health versus risk of making wrong decisions, we often specifically for medicine have a very conservative threshold [00:22:00] and a conservative approach to this, where we are very, very risk averse and not very opportunity seeking.
And that might make sense in an environment like the United States where these, as you mentioned before, the established system functions more or less in that people are able to get healthcare. And that's probably true in much of the industrialized world. But I'm curious if you think that in countries where the medical infrastructure is less established, if there's a benefit to being more opportunity seeking, even if that does potentially raise risks or the risk of making mistakes is sometimes higher.
Vivek: Yeah. Great question. It's a hard one as well, right? I am all for more medical ai but you want to be responsible that it's researchers. And so if for example, you built a model only using, data from Western institutions and you're gonna put that in, say a place like Africa or India, it's pretty obvious it's not going to work.
And that's I responsible on your part. So if you've done the legwork where you've actually built a model like used, [00:23:00] sourced diverse training data and actually validated in the appropriate settings, and you've seen that it works, then yeah, for sure. We should, I think maybe dial down our risk tolerance a little bit more and be more proactive in terms of deploying these technologies.
Yeah, with every opportunity comes responsibility. And there's no free pass. I think you still have to do a good job at validating, but I'm with you. I think there is I would say like there, there's a lot more opportunity maybe beyond the US or places with more established healthcare systems in terms of deploying these systems ahead of time and getting feedback data.
And it's probably possible that you might actually see these countries adopt AI faster and make actually, and have a leapfrog in how the care is delivered in these countries. It's the same with for example the financial infrastructure, right? So 10 to 15 years back, I would say China and India were like lagging behind the US and credit cards were dominating in the us but now I feel like US is further behind.
I haven't been to China, but have heard stories and in India We don't have credit cards, but it's all digital. And the ease of [00:24:00] doing transactions with micro transactions and micro transactions is order of magnitudes higher than in the us. And so it might, this is an opportunity for these countries as well, I feel like by adopting AI to have a improvement in the healthcare systems.
And maybe they go even above what's available in Western countries. And I can totally see that happening.
Bryan: I'm realizing that we've gotten this far in track conversation without describing what MedPaLM actually does. So maybe for listeners out there, we should what exactly is MedPalm doing? How do you interact with it? How does a researcher interact with it, given that it's not open to the public?
Vivek: So I will start off by giving the motivation for this work. Obviously large language models have been the rich in the wide area community. And medical AI and particular tool data. If you look at a lot of the models that have been developed, those are all like narrow single task supervised systems.
But on the other hand, medicine is an inherently humane endeavor where language is at the center facilitating interactions between patients, between clinicians, between researchers. [00:25:00] And generally if you ask like clinicians or patients who interact with medical AI in different settings, one of the chief complaints or concerns would be, oh, I wanted to better understand the model one to more interact with it.
But all this model gives me is a prediction with a property and I don't understand why that model is giving me this prediction. Right? So that needs to be solved if you want broader uptake of medical ai and that can be solved through language-based interactions, and that's what language models helps us to do.
So that was one of the, I would say the chief motivation of this work along with the fact that obviously there is a school technology and we have access to these models. And if you look at the work in general, we have considered a broad variety of medical question answering tasks. These include like medical exam tasks medical research questions, and also consumer medical question answering systems.
And we wanted to benchmark and see, okay, how effective are these models and these different potential end user applications? And so the target user could be a medical student, could be a medical researcher, could be actually a consumer who has a medical information need.[00:26:00]
So that's where we started. That was a motivation. And we had access to this model called Palm, which is amazing. It's not open sourced. I don't think it's going to get open sourced. But Yeah, the paper is a way for us to like communicate and get feedback as to what we are doing. And I would say the results that we report with respect to like performance and certain data sets that is not maybe as important as say the evaluation benchmark that we are setting up, or the different axis that we propose to evaluate the answers. And I think this is an iterative process which involves multiple rounds of dialogue between not just AI researchers, but also like clinicians social scientists, ethicists, because I think medicine and even patients and because ultimately at the end of the day, I think you require participation from all these folks if you want to really advance and accelerate the adoption of these technologies and models and medicine.
And if you even leave out one community then that's going to come back and bite us out. And so we wanna ensure that, ok, this paper is not meant, you have this fancy model that's more like, you know, we have this model and this, these models are going to come now let's build them out in the right way so that it's applicable to all.
John: [00:27:00] I find it even interesting kind of the approach of training a model on medical data as opposed to say, like, you give PaLM access to a bunch of information that it would read and use to answer questions. And something that's kind of interesting there, is kind of like distinction between approaches that involve embedding a lot of information in the model parameters versus having the information be external in some way.
And how do you feel like that's gonna end up coming together to make systems that are really useful and robust?
Vivek: I think it's always going to be a mix of both. It's this classic system. One was a system two thinking of the debate that goes on, right? I don't think we can. Or it's one versus the other, rather it's mix up Both.
Large language models are more of the system one kind but I think over the next few months, over the next year, what you're going to see is more like retrieval style models, which are going to allow you to do more system two style thinking and inference. I think with these models, obviously when you are training on internet corpus internet text, there's obviously gonna be medical content in there in different flavors.
Some of them may be accurate, some of them may not be. But the model has seen this. So that's [00:28:00] good because outta the box we do see that these models can answer, but they do understand medical terminology. So if you ask like a model Okay, can you explain this condition? Yeah, it does a decent response but the challenge is really medicine is an evolving field.
And so there's always new research being published, a new guidelines being published, and see we want to feed in that context information into the model, and then teach the model how to use that context information or additional information and integrate that with what it already knows which is included in the parameters of the model, and then come up with the appropriate responses. And so, yeah, it's gonna be mixed up with that. I don't think it's one versus the other.
John: Yeah, I'm just wondering if there's like a way of thinking about it. Like, is it like kind of like, one way you think about it could be like, oh, like should I think about it like the vocabulary? like, okay, I really need the model to have the right vocabulary and I can't just like teach it vocabulary in context very well. And so that's what I'm getting out of tuning it on the domain or is there more to it? Is there like different types of reasoning that happen in a medical domain?
I mean, I know people have had this theory that like, oh, maybe chain of thought comes from code and so like training your model on code is important for that. You know, I'm just kinda curious like [00:29:00] what sorts of things you feel like the model's really getting from the fine tuning that you wouldn't be able to do from say, context?
Vivek: So I wanna clarify that the amount of fine tuning that we do with the model over here, the MET model is actually not that big. We're using on the order of a few hundred examples and we are doing prompt tuning. So it's not even the end-to-end model that's fine tuned. It's just these additional soft prompt parameters that we learn and.
Our hope was that doing this would help condition the model. The one of the assumptions that we have is a lot of the medical data is already encoded within the parameters of the model. But then at test time, we want to do two things. One is actually point the model to use that information. So this is like looking for a needle in the haystack.
And so the model knows about science, it knows about, you know, random stuff on the internet, but you want to condition the model into that, for these set of questions. Use your medical information, user clinical knowledge. Yeah. Right. And so that's one of the things that this helps with.
And the second thing is in the medical domain, there is a very unique way of [00:30:00] answering things. There's a very unique way of reasoning about things, and we also want to encode that information as much as possible in these soft, prompt vectors. And I don't think we've. Yeah, it's possible that there's a limit to what you can achieve with these soft pro factors, because at the end of the day, it's still like a few hundred token millions of parameters.
But at least the impression that I get is you can get the model to understand the stylist technique of the domain. So if you look at the responses of the model generates, it's not overconfident rather it's more subdued. It clearly says this is what I know.
Anything beyond this, you should probably go and seek specialist care. And it also learns to trim down the length of its responses because it knows that anything extra, which it's uncertain about it could be incorrect and that could have downstream consequences. So those are the kind of stylistic natures of the domain that you can also encode.
I think that's what is probably happening more, it's not knowledge that's encoded in, it's more like conditioning to work well within the domain.
Bryan: I'm curious are there any techniques that you are really excited about [00:31:00] for grounding things in sort of external information? So for instance, teaching these things to basically not rely on their system one thinking, but to kind of know, oh, I do need to go fetch this.
I do need to go look this up and verify that. Are there any approaches to that that you are particularly excited about or you think are gonna make progress on these problems?
Vivek: Sure. I think there's been a class of models which point to this direction, web GPT, retro and a few others. And being demos from like a few startups, Neva and publicity as well, which are going towards getting, like using search and using that as additional context to answer questions and then also citing and attributing the sources.
And so, there are a few different approaches, but I think overall they're kind of all the same, retrieve the right information, feed that into the model, and let that model integrate that information with whatever it's already encoded in the parameters already, right? I think the cool part is it seems to me that teaching this kind of behavior to these LLMs is probably quite data efficient.
You don't require a lot of examples. It [00:32:00] seems like even with like maybe a few hundred or a few thousand examples, you can tease the model to learn this generalized behavior. So that seems pretty cool. And so this goes down into tool user, right? And search and retrieval is one of the tools that's in the in the models repertoire.
But you can also imagine this being generalized to say any expert in the loop. And that expert could be a human in the loop or it could be another ai or it could be anything else. It could be a calculator, for example. For me, the most exciting part is it feels like this kind of behavior is learnable without a lot of examples and it also generalizes, but I think, yeah, we need some research papers or maybe someone at Google or Open, yeah, I'll publish this.
But I feel like that's one of the cool things that's coming up right now.
John: When you're choosing your instruction examples, are there domains where you expect this to be deployed that you're disproportionately representing or choosing from?
So it's like we don't literally need the model to take the mcat, right? I mean, it's impressive if it's good at the mcat, but we kind of wanna potentially deploy this in a clinical setting. I mean, are you imagining like doctor, is like wanting to double check their [00:33:00] understanding of a certain condition and so they're they're going to potentially ask the model " I was wondering if this medication interacts with that medication or like, I'm doing a diagnosis here, but I have a kind of strange combination of symptoms. Like, what do you think? Like, what specifically are you imagining are our clinical application dialogues?
Vivek: Yeah. I think there are fair few intended potential applications over here. Probably the ones that we'll see the earliest are more like educational aids to like researchers and students and trainees.
I think we are already seeing evidence of charge being used for like educational purposes. And I've actually learned a few topics just by interacting with it. And you can imagine this happening in the medical domain quite a lot especially with a model that's specialized to that.
So I feel like those sort of use cases where, which are non-diagnostic and that means also not safety critical, are going to be the first that we'll see and probably that'll happen in a few months.
The second set of application is around like aiding researchers and scientists with respect to information retrieval and citations and similar stuff.
I think that that could also be incredibly [00:34:00] powerful. Like if I am writing a paper and if a model could retrieve like 10 more related papers with respect to this paragraph, that'll make my job really easy. Because right now, like I think with every time before a paper deadline, the match scramble is to get your references right and it takes a few hours. I think a model that can do that and summarize it will be amazing. I think that there'll be a game changer for researchers and not just medical researchers, but like all kinds of researchers and scientists.
I think the final set of applications you would see are more in clinical settings. And again, I think this is gonna be different. They're gonna be certain applications and clinical workflows, which might maybe involve like extracting information from notes or different documents in clinical settings and summarizing them either to patients or maybe to the doctor centers or people who are working in those settings and like just giving them a very simple intuitive interface.
Two, the data under the hood and not like the clunky systems that they have right now. Again, that I feel could happen to your timeline. You could say a lot of different applications. I think we are already seeing a lot of interest from healthcare companies in using these models to do such things.
And also there's a lot of [00:35:00] documentation that clinicians generate that they're all like fairly templatized all non-diagnostic. But those can also be automated and have an LM generated summary or like a prescription or like a medication authorization letter or referral letter.
So again, those sort of applications is completely non diagnostic totally possible that those things happen within, again, a two year timeframe, if not less. Diagnostic is further down the line. And I think first set of applications we would see would be where there is a, like a human in the loop of clinician in the loop. And it would more be like an information aid system for them where they like have a chat interface to like a database. Similar to Google search, but like a more interactive conversational system where they can ask about interactions of medicine or like an interface to EHR records. And I think those are the kind of applications we would see first.
And I think ultimately down the line we'll have more diagnostic systems where it's going to be like, an AI and maybe a clinician or an AI alone coming up with diagnosis based on all the context of information. But that I feel is further down the line. And we have all this research.
My hope is this all gets translated very soon, but I feel like that's probably a few years [00:36:00] just cause of the number of challenges that we have solve before we get there.
Bryan: Back in 2020. I'm pulling back from your tweet history here. You stated in an opinion, you shared an opinion that some of these applications, the publicly accessible LLMs might be doing more harm than good.
And I think they probably, at the time you had in mind the potential to be a source of misinformation, be a source of deep fakes, all that sort of stuff. I'm curious if you're thinking about these things and obviously since then ChatGPT has completely exploded, there's a brand new generation of interest in the applications of these models. Are there Do you still feel like the, that sort of sense of caution? I mean obviously Google has been very cautious about releasing any of its LLMs to the public. There's obviously a very storied history of Microsoft and Facebook having to kind of take models offline because of how quickly they become negative.
What do you think about the potential of these things to be open in the public as the APIs?
Vivek: It's interesting and I'm glad you pulled that out. I would say I was a bit naive with that I was assuming the [00:37:00] worst and maybe that hasn't necessarily happened. Stable diffusion is a very good example of that a model that's out there openly and people are using it mostly for creative applications.
I haven't heard, horror stories or anything about that. And so that does point to a future when maybe these models can be open. And honestly, I would love for these models to be open and democratized, but it's. It would be nice to assume everything is good and everyone has good inventions and just answers because that's not true.
And so it's very important that you consider what can go wrong, and different organizations have different levels of risk tolerance. And maybe if you're a startup, you don't worry about that so much because you're not gonna be a legal target. But if you're a big tech company, obviously you have to worry about it a lot.
So yeah, I've been very pleasantly surprised by how stable diffusion has gone and how GPT three has also been put to use. But maybe that also has got to do with the fact that these models, yeah, I mean, it's all over your Twitter feed, my Twitter feed, but that's a very small fraction of the people who, interact with the tech or or on the internet, it's still [00:38:00] probably like 0.1% or even less than that.
So it's not a mass adoption or a mass feature just yet. Yeah, it's hard for me to know to predict how exactly someone in India or like in some other part of the world who is maybe five years behind what we are how would they use these technologies, and it's very likely that people are all going to put this to like amazing use cases. And I hope that is the case but we need to also at the same time be aware of what can go wrong and build tools and systems to ensure that that happens as little as possible so that we can be more open and democratic about these systems because these are amazing.
There are more people we can get them like these into the hands of. That's amazing. Actually just maybe one final point over here. I actually don't know how things are going to evolve because it also feels There are these two competing forces. One is this open AI model, or maybe you can even say, Google's model is where it's the model center sitting in some server or some in some cloud somewhere.
And then if you look at Apple, on the other hand they're trying to put like stable diffusion kind of models on the phone. And so those seem to be two competing trends with respect to how these models are going to [00:39:00] evolve. LLMs are a different beast. I think stable effusion, I could, didn't expect that model could be compress and put on the phones so quickly, but that did happen.
But LLM I think would be a little bit more tricky to do that. And so it may also be like, which technology wins out, because if you can have like a personal component of element sitting on your phone, then that's really cool and that's another way of democratizing this technology and having access to more people. And that might end up happening ultimately.
John: Something that's interesting about Google's deployment strategy is, they've been very public actually about what they're doing. So they have these papers they have not released parameters, which is pretty understandable for most models. With a couple exceptions, like I'm kinda excited about flaunt five, for example.
It's cool that they released this parameters. They haven't really released like a product in the way that say OpenAI has. How is Google hoping to have a big impact in the world with the approach they have of not really releasing models either for inference or the parameters?
Vivek: I think it's hard to predict how things would [00:40:00] evolve. But if you look at it, open ai, I was also not released any of their model parameters. It's an api. I would say it's very hard to predict. I think what big tech companies in general have is distribution. Yeah. And so perhaps what they're all gonna be looking at is how do we integrate it into our existing sphere of products and, just make them more delightful and more magical for people to use. And that might mean a different strategy for meta or Microsoft or a Google, because they all own different kinds of surfaces, different kinds of products. Cloud is a different question over here.
I think people would be hoping that these models stay more centralized and you have a lot more cloud customers, and that's probably a very natural evolution of cloud. But I don't know if that will necessarily play out just looking at how stable, de efficient has evolved. I think what we need to watch out for is how quickly is there a Chat GPT eqp open source?
And if that comes out very soon before say A G P T four comes out, then I think the trends are kind of obvious. But that might also, what that might also trigger is maybe [00:41:00] open AI would want to talk even less about its research and be more secretive. And that's not great for all, and that might further slow down the open source application, but, open source is an amazing thing. I mean, this will be people working from, "hey, we're just coming together and creating." I think that it's, so, it's hard to predict how open source dynamics and things really well, but I think that's the one thing that I will watch out for, like, how quickly do we get a chat GPT equivalent. And if that comes out rather soon performing as good as say whatever we have right now, then I think that changes the calculus for everyone. I think. So people are just like, at this point of time, still not sure, and it's mostly a wait and watch game for everyone, not just Google, but for all.
Bryan: So you've been able to play with Google's internal tools and you've also obviously have played with ChatGPT. I just wanna know, just from your subjective opinion, which one's cooler, but also after you answer that question, I wanna know what brought you to SPC? How did you find yourself becoming a member of SPC?
Vivek: I think chat-GPT is awesome. For me it was one of the most magical experiences that I've had with ai. So I was working on [00:42:00] conversational AI five years back. And one of the projects that I was tasked with at that point of time was to build a system that can help you set an alarm. And what that entailed was me writing out thousands of rules and thousands of different ways in which someone can say, set an alarm.
And one night I just said, I don't wanna work in conversation AI anymore because it's not gonna scale. Yeah. And so if at that point of time you had told me that, in five years we're gonna have the kind system, I would've said, you're kidding me. And so for me, Just thinking where we were as a field.
And I l P was far behind computer vision at that point of time to where we are right now. I think this is one of the most incredible advances that I've ever seen. I can't really compare with Google Systems, but I can just say that is incredible and I hope to see more and more of these systems.
And with respect to SPC, no, it's just an incredible community. I felt. I've always wanted to be a part of s spc. I knew people who are at Open AI or Deep Mind. I got a lot of the people who were very early into deep planning and AI back in 20 16, 20 17. I know that they had, they have SPC sub connections. [00:43:00] So the community have always found it interesting and exciting. And so that was one of the motivations just to meet more interesting people and share knowledge, learn about what people are doing and also be exposed to opportunities.
I've had this like pretty incredible opportunity to work with a few non-profits. One of them that come to mind is Rocket Learning in India, which is trying to scale education, primary care education to school children. And through SPC I, I got connected to them and I've been advising them on some of their like using AI in their product stack.
And we've been using, trying to use that for grading assignments, but we want to do more personalized content generation and curriculum generation and so on and so forth. And again, just similar to medicine, I think AI is going to have a huge impact on education maybe even sooner. So those sort of opportunities where you have this domain expertise that you have built in, if you can share it more freely, like with people who are trying to do some incredible things in the world. I think that's one of the unique value props that s SPC has, where people are trying to do amazing things and you can, tag along with the journey. And it could [00:44:00] be directly as a co-founder or it could also be like, more indirectly where you're an advisor. And sometimes all you need is maybe just a few hours where you just say, oh, you know, take this model and do this thing. And I've had a few of those interactions as well where people have come back and said, oh, you saved days or months for me. And so those sort of things.
And it's the same reverse as well where we call SPC. This is if you're going minus one to zero, right? And you looking for new ideas. And for me, beyond medicine and ai, I am very interested in biotechnology. And I actually think that the next few decades that is the decade of bio and ai, and that a few people have said this before: the amount of biological data that we are generating and how for example, our sequencing technology has progressed, it's progressing faster. For example I think like all levels of the stack single cell data to, clinical data genomics data. So this incredibly rich amount of data that's being generated and biology is messy enough that you can't have like hard rules like math or physics.
The perfect description language for that is ai. [00:45:00] And for me, SPC felt like a very natural place to engage and learn more about this field. And I've been fortunate enough to meet like a few people who have product expertise in bio biotech. And so that's been amazing as well. We are putting together a tech bio forum now. It's coming up later this month where we are going to host like a series of talks from researchers, founders, venture capitalists in the biotech space. And the hope is like SPC becomes uh, also the go-to place for people who are interested about biotech as much as say about AI and crypto.
And if that happens, I would be really delighted. I think that would make my time at SPC really worthwhile. And hopefully I think there's connections and networks the day I go down the entrepreneurial path, I don't have to look too far to find a co-founder.
Bryan: We're glad you're here, Vivek. Thank you so much for being part of our show today, and thank you so much for staying and answering some great questions.
We'd like to finish things up with asking you for a recommendation on something you're either reading, watching, or listening to. What is one recommendation you'd give people that you are listening to [00:46:00] you now?
Vivek: Actually the book that I'm reading right now is a neuroscience textbook, so maybe I'll stay away from recommending that to I think our last one.
Bryan: Yeah, I think the last interview may have also recommended a textbook.
Vivek: Principles of Neuro Design. That's maybe it's blur, but Yeah. Yeah, I'm super interested in neuro neuroscience and trying to get inspiration. building more low power AI systems because while I work at a place which promotes large models, I like just looking at how the human body is engineered how low power it is how efficient it is.
I think we can do better. So just trying to get more inspiration. So yeah, but I don't know if that's for general audience though.
Bryan: That's fine. I think this is an audience of a lot of nerds, so it'll fall in familiar ears.
John: Yeah. Sounds cool to me. ,
Vivek: It's good to know. Maybe we should do a reading group session for this one.
I dunno.
John: Yeah, you should make a s SPC forum about neuroscience.
Vivek: That'd be amazing. This is why SPC is awesome. Yeah. Yeah.
Bryan: Well, thanks so much for being part of a part of Pioneer [00:47:00] Park. We're so happy to have spoke with you today.
Vivek: Thank you so much. This was great. And as I said the reason for being at SPC is being the opportunity to have these kind of interactions where we can go deep into certain topics or learn more about stuff.
And with the peer group and the peer network, I'm just glad that we have SPC and I hope like more smart people decide to come and join us over here. Thanks so much feedback. Thanks John. Thanks Bryan. Take care. See you around. Bye. Bye.
The frontiers of clinical AI with Vivek Natarajan