OpenAI DevDay 2024 | Community Spotlight | Grab

my name is gorov I'm a data scientist from grabs mapping team it's fantastic to see such a large group of developers machine learning Engineers uh here today I hope you're having as much as fun as me uh okay yeah so some of you may in this room may have seen grab you may have used uh grap today while arriving at this venue or may have taken a grab bike during your holiday to Bali uh to give you an idea 12 years back when we started uh our mission was to make uh taxi rid safer in Malaysia we were in one country one city today grab has become one of the leading super apps in Southeast Asia with one in 20 people using it for food rides and payments that's over 41 million monthly transacting users we are committed to driving southeast Asia forward not just through traditional services like ride hailing and food delivery but by literally putting our region on the map today I'm excited to share a snippet of how our very own grab Maps is truly disrupting the traditional mapmaking space together with open AI okay so for some context grab Maps began its journey in 2017 uh before covid uh and we increasingly found out that the third party apps were not localized enough for the region we had several issues in terms of uh not having granular enough view of the region or the data was uh quickly obsolete with significant lab with respect to how quickly the real world evolves today grab Maps intelligence services not just serves our internal requirements for eight countries we operate in but is also an Enterprise grade solution that supports businesses across house Asia and beyond our approach to grab Maps is rooted in community- based mapping centered around Precision we collect street level imagery using 360° cameras uh which are inhouse leveraging our vast driver Network and similar to the image that you see in the middle of the screen from these images we extract a lot of details like turn restrictions traffic signs speed limits places Road accessibility and so on helping us build the map uh Road topology this level of complexity is key for creating reliable and Hyper detailed maps now Switching gears a bit uh as you know uh GPT 40 text models fine tuning has been there uh from early this year two months back openi released this uh Vision fine-tuning capability to customize Vision models with strong image understanding capabilities we have been one of the early adopters for vision fine tuning API and in the next part I shall show uh an example of how we leverage this newly introduced feature on uh a data matching problem for grab maps to briefly introduce the task given Street imagery with traffic sign we intend to match it to the road the traffic sign is placed on given criticality of information like speed limits presents its own unique challenges for example you can have geometries that can be very intricate visual occlusions that can make automated match matching very difficult to address this we turned to GPT 4 fine tuning with our indain data allowing us to uh handle these complexities effectively at scale all right to give an overview of the experiments we started with a small fine-tuning data set combining street level imagery uh and map tiles much like you see on the screen to add some context we have two consecutive map views uh at the top of the screen and uh corresponding street level imagery for uh let's call them as frame one and frame two each map tile contains the position of the vehicle marked by the Red Dot that you see at the very intersection of the two roads and the position of a traffic sign that is marked by the letter small letter U that you can see on the left of that red dot there's a green bar