# Unlocking the Power of Spatial Data in Data Platforms with ArcGIS

## Метаданные

- **Канал:** Big Data LDN
- **YouTube:** https://www.youtube.com/watch?v=V9obL4KDVZU
- **Дата:** 11.11.2025
- **Длительность:** 29:41
- **Просмотры:** 195

## Описание

Analytics, Visualisation & Storytelling Theatre
Wednesday, 24th Sep
10:40 - 11:10
In this session, we will explore how organisations can leverage ArcGIS to analyse spatial data within their data platforms, such as Databricks and Microsoft Fabric. We will discuss the importance of spatial data and its impact on decision-making processes. The session will cover various aspects, including the ingestion of streaming data using ArcGIS Velocity, the processing and management of large volumes of spatial data with ArcGIS GeoAnalytics for Microsoft Fabric, and the use of ArcGIS for visualisation and advanced analytics with GeoAI. Join us to discover how these tools can provide actionable insights and enhance operational efficiency.


Dominic Stubbins
Chief Architect, Esri UK
Dominic Stubbins is the Chief Architect at Esri UK, where he has dedicated 20 years to advancing spatial data management and analysis. Throughout his career, Dominic has led teams in developing innovative solutions for various industries, including water utilities, national mapping agencies, environmental agencies, and defence. As an expert in GeoAI and spatial big data, he has extensive experience working with tools such as Databricks, Microsoft Fabric, and ArcGIS. Dominic's expertise and leadership have been instrumental in helping organisations harness the power of spatial data to drive informed decision-making.

## Содержание

### [0:00](https://www.youtube.com/watch?v=V9obL4KDVZU) <Untitled Chapter 1>

Okay, hopefully you can all hear me either with your headphones or without your headphones. Um, so welcome to this session, unlocking the power of spatial data in data platforms, uh, with ArcGIS. It's a bit of a wordy title, but I'm going to talk about most of those words as we go through the session today. And who am I? So, I'm Dom Stubbins. Um, and I'm a geographer. Sounds kind of weird when you say that out loud. bit like I've got an addiction and maybe I have but I'm not here to get better. I'm here uh to bring you along with that addiction to geography and spatial data uh as we go. Uh but the actual job that I get paid to do, I'm chief architect uh here at EZRI in the UK. And that means that I work with lots of organizations like the organizations you work for building uh data systems and platforms for working with spatial and location data. Um and so I'm going to talk about some of uh the things that our customers are doing uh with spatial data and how you can do that as well when working with big data platforms. [clears throat] I'm going to talk about spatial data, data platforms, and the way that we work with different types of data in those platforms, but really with a focus on spatial data. So, let's start at the beginning. Let's start with data. What do we even mean by data? I mean, you're all at a big data conference, so I kind of feel like you probably know something about data, otherwise you're here by mistake. Um but the dictionary definition um it's information facts numbers collected to be examined and used to help decision- making and how do we do that? So data on its own is not very useful. We need tools, we need techniques, we need approaches for working with data. Um so let me give you a kind of little example. Now I'm just going to apologize in advance. We're having some AV issues. So if this doesn't show up very well on the screen, you can come and see everything we're talking about at our stand later on. Um, so there are lots of tools that people use for working with data and we're at a conference where there's hundreds of stands of people trying to sell you different tools uh to work with your data. That's probably what I'm doing as well. But the tool that most people start off with when they look at data is probably a spreadsheet. We're all used to seeing tabular data in a spreadsheet. Now, you probably can't make any sense of that on that screen. Um, but it's a data set. And as a geographer, I can see something kind of interesting about that data. The first two columns, lat and long, are how we describe where things exist in the world. They're coordinates of where this data might be located somewhere in the world. Um, and because that's in the data, I can kind of think maybe I can do something more interesting with this data. I mean there's lots of things that I can do with a spreadsheet. I can look at the dump the raw numbers. Uh I can create graphs. I can look at the data. I can slice and dice it in different ways. Um but it's quite hard to discern a pattern in that set of numbers. So let's try and do something a bit more interesting with that data. Uh what I'm going to do is use one of my favorite bits of uh Excel pivot tables. I'm going to insert a new pivot table into my spreadsheet. Okay. And these are the fields that are in our data. Now, latitude, that's how we as geographers uh and I'm including you in that now because you're at my talk. Um that's how we as geographers describe how north or how far south uh data is in the in the world. So, higher latitudes, North Pole, lower latitudes, South Pole. And Excel has the same idea about how to uh look up how to manage data. It has rows. So if I take the latitude data, pop that into the rows of my pivot table. And I'm going to do the same with longitude and that describes how far round the world we are. So how far east or how far west Excel has the same idea. It has columns that kind of explain how the kind of leftiness or rightiness of that data. So I'm going to drop my longitude data into the columns. Uh, and I'm also going to and I want to kind of look at some of this data. I'm going to choose what am I going to choose? I'm going to choose this field here. Pop this into the values. Now, we can't really see any data on there at the moment, so I'm just going to uh rearrange this site slightly. Now, uh, Excel has the smallest numbers at the top with rows and the biggest numbers at the bottom. Uh, as geographers, I guess it's probably uh, scientific colonialism. We have the big numbers in the northern hemisphere and the small numbers in the south. So, uh, I'm just going to reorder the, uh, the rows. And some of our data is starting to appear in the spreadsheet. Now, I'm just going to make change the display a little bit. So, uh, first of all, I'm going to turn everything uh, black because everything including data looks much cooler when you show it in dark mode. Um, and then I'm going to apply some conditional formatting to that data. Now as a geographer I would call this ctography but really it's just kind of statistics and using Excel. Uh I am going to use a three color scale. Now the data that we have here is actually data about uh wildfires and natural fires across the that across the globe. So I'm going to choose a color scheme that kind of matches that data set. Uh we're going to start with a um coolish kind of blue for the low for the u kind of lowle data we're going to go kind of a fiery red in the middle and then for the highest data um we're going to go with like a bright hot yellow. Let's apply that to our data. I'm also just going to uh adjust the size of these cells a little. And if I zoom out from my spreadsheet, all of a sudden we can see some patterns in the data. It's not a spreadsheet of numbers anymore. It's a map of the world showing uh wildfires and how they're distributed around the globe. And we can see the intensity of those wildfires based on the color. Now, why am I here demonstrating how to make maps in Excel? That's not what I'm supposed to be doing here. Um, but the point is, if we look at our data through a different lens, we will see different patterns. things that are important in that data. And spatial is one of those lenses that we need to apply to our data. So, and there are many different reasons, many different um kind of business reasons and applications of that spatial data across lots of different industries. Now, I'm just going to kind of highlight one or two on this slide here around some things we can do with uh customer data. You know, can customers access our premises easily? Uh what does our supply chain look like on a map? Where are our competitors? uh how does our competitor's uh footfall match up with our footfall? Can we compare where our stores are with our competitor's stores? And that's just one example in the retail space where uh location data uh that's embedded in all the data that you have in your organization uh might be important and like Excel is a great tool for uh working with numbers and if you try really hard you can make it display a map like we did at the beginning of this talk. Um but there are lots of things that it would be really quite hard to do in Excel. What if I wanted to understand uh how those clusters of fires had changed over time? What if I wanted to understand how they were likely to impact uh my supply chain, for example, or assets that I have out in the world? Uh what if I wanted to compare that data with data about those assets? That's something that's much harder to do uh in Excel. And so we need dedicated tools uh to carry out this type of spatial analysis. Now I don't think you can read what's on here but these are some of the different types of spatial analysis that are commonly used uh with data. So the first thing is about understanding where things are and that was what we did simply in Excel at the beginning. Um but then we might want to think about well how big are those things? What's the distribution of those things? We need to measure those things. So that's uh kind of the second step of spatial analysis. And then how are these different things related? Is there a relationship between say uh fires and um logging activity for example? How do we compare those different data sets? How do we look for relationships between where our customers live and where our uh distribution centers are? Um and then we can move on from that to thinking about well where should we put our distribution? Where are our potential customers? Can we find new locations uh using spatial data? We can take that a bit further and start analyzing those patterns, looking for clusters, looking for areas that we're seeing increases uh in customer demand, areas where we're seeing decreases in customer demand. Uh and then finally, we can use those to make predictions about where we might want to locate new facilities, for example, predictions about what might happen in the future. How will the spatial distribution uh change in the future? How will that change over time? So, there are lots of different types of spatial analysis uh that you can apply to your data. [clears throat] And spatial data isn't just about tables uh in Excel. There's lots of different types of spatial data, whether that's uh observations from satellite, which is what we were actually looking at in Excel, whether it's uh LAR surveys from satellites, from airplanes, even from your mobile phone has a LAR detector on it. Now, um there's vector data, there's real time data, and we'll talk a little bit about that as we go through this presentation. So obviously I'm here from Ezri and I'm talking to you about the tools and techniques that we use for working with spatial data. And in Ezri we've been

### [10:40](https://www.youtube.com/watch?v=V9obL4KDVZU&t=640s) 11:10

building tools uh for working with spatial data for more than 50 years now. Um uh the tools that we build are ARGIS which is a comprehensive platform for managing analyzing and working with spatial data. And often when I'm talking at conferences or to audiences, I'm talking about the types of applications that our end users are using in order to work with spatial data. They might be mobile applications or web applications. We have desktop applications and analytical uh tools for working with that data. Um, but actually the reason that we're here today at Big Data London is not so much to talk about the tools that your end users, your customers of your data might be using with spatial data. It's really to focus on how organizations are managing spatial data. So increasingly the organizations that I work with are building out data platforms. So these are um well what is a data platform? So if you go to many of the stands around here, there will be a slide that looks something like this. There's uh I've put the one from Microsoft up. Um but if you went to data bricks or IBM or uh Amazon, they will have a very similar slide describing what is a data platform. Um and really a data platform is about taking a organizational approach to managing your data um across the whole uh set of workflows and business applications that you might have. And I typically kind of split it into three areas. So one is the ability to pull in uh different data sources to get data from your kind of other business systems into that data platform. The ability to process, store and manage that data in a central repository. Um clean that data, look after that data. Uh and typically those these platforms are built on Spark. They're built on some of the big data technologies. And then the third part is about delivering that data to your end users, serving that data up to those users. And in the same way that we need to have a spatial lens when we're looking at our data, thinking about our data platform. How do I make sure that spatial data that I want to bring into that platform uh retains that spatial intelligence? How do I work with spatial data storage? How do I do analytics on spatial data in my Spark system? and how do I serve that spatial data up to those end applications? So, I'm going to talk about each of those three sections in a little bit more detail and hopefully you'll get some ideas about how you can take spatial data into a data platform and start working with that. So, I'm going to start with the data sources. How do we ingest spatial data? So, uh one of the kind of most voluminous sets of spatial data that you get are real time observations. So real-time data. So that could be things that move like vehicles, um airplanes, uh people with mobile phones, um they move, you can track them, you can track the location, that's important data. Um the second set of real-time data is about things that don't move, but they have a location and they monitor something. So think about things like uh river gauges. How high is the river? Is there going to be a flood? Think about uh infra sensors on infrastructure. Is this uh device in the field overheating for example? So these have a location. They don't move but the data changes over time. And then finally there are events things that happen somewhere. So things like crimes or uh vehicle accidents uh lightning strikes these all happen somewhere. So this is spatial data that's real time and uh provides real-time feeds. Now one of the ways you can bring that into a data platform is using um a part of our platform called uh ARGIS velocity which is designed to work with real-time data streams. Now I'm just going to run a small demo. I'm going to do it as a video because the screen's not great. Um but if you come to our stand hopefully you should be able to see it uh in a bit more detail [clears throat] um once we get there. So, uh, this is the, uh, kind of the way into ARGIS velocity. We're able to create data feeds and data output, so we can move spatial data around. And I'm just going to create a feed uh, of real-time data to show you how we would do that. And there's lots of different types of data feeds. We can bring data from a GIS system. your cloud data platform, whether that's uh, something like Azure Event Hub or something like AWS uh, IoT. Um and then there are a whole set of data and messaging formats that we can use to bring data in. Uh whether that's HTT HTTP data things like CFKA using websockets using message cues. Now I'm going to bring the data in uh from an HTTP uh service and I'm going to pull that service for real time updates of the data. So I'm just going to go through the process of configuring this. Now in this case what I'm actually working with is some real-time traffic data from ways. I want to bring that into my data platform so that I can use it uh alongside information about vehicles that I have. Uh perhaps it's my delivery vehicles and I want to understand what's the impact uh on my customers. So I can connect to the feed. Um I can access that data. Um I need to configure how that data is going to be turned into something more useful. At the moment it's just a big blob of JSON. So I'm going to do a little bit of work uh to configure that. we can derive a data schema from that. I'm not sure if you can see on the screen, but there's a whole bunch of data fields there describing uh the ways data. So things like uh what's the nearest location? What's the confidence I have in that um observation? What is it? Is it a road closure? Is it an accident? Is it um planned road works, for example? Um is it some kind of event? But most importantly at the bottom we've got a location X and a location Y field which I can use uh to bring in the location of those events because that's what I'm really interested in that spatial aspect of the data. So I'm going to bring that data uh into my system. Now, I'm just going to jump ahead a little bit. So, we can bring that data in. Um, we can set some time and date uh parameters for that data because it's not just about where it is, it's also about when that data uh happened. Um, and I can publish this as a real-time data feed that I can then feed into um some advanced analytics or bring that into my data platform. So the first thing I can obviously do is uh display that data in a map. I can click on the blue dots which are the events appearing uh from that live feed in my map just to make sure I'm getting the data that I want. But more interestingly um I'm going to bring that data into what we call uh an analytic. And an analytic is where I can take these different data feeds and combine them. So, I'm taking in this case traffic data. I'm taking some information about uh buses and I'm joining those spatially. I'm looking to see which of my vehicles are being impacted right now by traffic. Uh and I'm building out this pipeline of spatial data and then at the end of the pipeline um I can deliver that data either to an application or into a data platform for uh further analytics. So this is the application that sits at the end of that data platform. It's actually a visualization of this data. Um so the red bits are the traffic uh incidents taken from the ways feed. Uh and then you can see the buses that are in the city moving around and down the bottom we get alerts when those buses are involved in uh traffic incidents. Uh so somebody sitting uh in a control center can look at this dashboard and see what the impact on all of their fleet is or they can zoom into individual vehicles and see whether those are being impacted by traffic. So that's an example of bringing in a real time data feed that has some spatial information. I'm ingesting that data into the platform. uh I'm doing some analysis to bring it together with other spatial data that I might have and then I'm delivering that data out to my end users uh via an application and there are lots of different ways to bring that data in. There are lots of I'm not expecting you to read these because on the screen there are lots of different types of analytics that you can do uh as you bring that data through the pipeline and there are lots of ways to deliver that data to your end platform and that could be feeding it back into your data lake so that data that's been enriched and enhanced you can use for further analytics which brings me on to the second section which is about processing and working with big data. Now, typically this part of uh a data platform is built on some kind of spark-based system where we're able to do massive processing of data. And again, if your data has a spatial aspect, your big data system needs to understand that data is more than just text and numbers. It represents a location. And there are a whole number of uh types of analysis you can do to with that data. Now one of the tools that we use for doing that is what we call geo analytics engine which uh allows you to embed uh spatial functionality directly into your spark cluster. So it adds uh nearly 200 different uh spatial analytical tools and capabilities uh to your spark platform and you can read and write your data from anything that you can access from that platform. Now last year when I spoke here we talked about um and in fact we demonstrated running that platform on uh the datab bricks platform uh and uh geo analytics engine is supported on all of the cloud platforms and it works with both uh datab bricks and the native spark uh environments in inside those platforms. Um but what I wanted to talk about uh this year is um the work that we've been doing with Microsoft. So we have a partnership with Microsoft and we have been building Geo Analytics directly into Microsoft Fabric. So you don't need to install it separately. It's a part of Microsoft Fabric. Um it's currently in public preview and it's going into general availability uh in the next month or so. So, we're really excited about this partnership because it allows you to work with spatial data directly inside uh fabric [clears throat] um particularly on the kind of data science and data engineering working with uh notebooks uh signapse notebooks uh but also bringing your data into PowerBI um and also some visualization through some of the applications that we provide at EZRI. So, what does this add to uh fabric? Well, the key thing is turning your data into location data. Either if you've got coordinates, X and Y data, whether you've got address well-known geographies like uh postcodes, uh like census areas, for example. It allows you to turn that data in and visualize and analyze that data spatially. [snorts] Uh so this is just a kind of quick demo of that and I'm sorry you not can't see the details. Um so this is uh signaps notebook and you can see at the top we're bringing in the geo analytics library directly into the notebook. We're able to uh then access any data that's available within fabric. So whether that's spatial data, whether that's tabular data, we can ingest that data uh directly into the signaps uh platform, the fabric platform. Um and then we can start to uh turn that data into spatial data. Now obviously we can visualize it as tables. Uh charts like you can uh with any other data. Um but because we've got some spatial information, we can also start to join it to some of our other spatial data. So we can use tools like uh data wrangler to bring to clean up that data. Uh we can join it to sensors data which is what we've done here. The data in this notebook is actually uh phone call data, emergency phone calls. So we can bring that in and join it with demographic data to find out where these uh emergencies are happening. [clears throat] Uh and once we've done that, we can either write it back into one lake um or we can do some further analysis inside of our uh notebook environments using the geo analytics analytical tools. Uh so we can do things like aggregating all those phone calls into uh bins. So small uh block small areas. So we can look for clusters. Uh we can look at comparing that data over time. So uh comparing 2023 with 10 years ago for example. Um we can uh as I said before we can join it to other data and we can serve that out if we want to something like PowerBI where people can start to slice and dice that spatial data um more deeply. So being able to use those geospatial tools inside Signapse notebooks in fabrics allows you to integrate them into all of your workflows that you may have in fabric. So for example, you can use uh data pipelines and data factory to process, clean uh and manage your spatial data across fabric. You can use the geo analytics tools uh to do that. Um and when you think about how people use uh data platforms, uh many organizations are building um data platforms around this idea of a medallion architecture where all of your raw data feeds into your bronze uh data sets. So that's the raw spatial data that you get from other systems or uh that's fed into your data platform. So at that point you can start to add uh that kind of spatial you can uh spatially enable those data sets. uh you can then start to transform, clean them, enrich them, aggregate them and bring them into the kind of the silver tier of data. So this is the data that's clean and fit for different business applications. And then the final part of that is uh converting that data, transforming that data to support your end users and the business applications that they use. So your so-called gold data sets and all these steps of the process you can use uh geo analytics inside of arch inside sorry inside of fabric or these other data platforms uh to to enrich those processes with your spatial data and that kind of just brings me on to the last bit about serving out that data. Um there are lots of different ways that you could deliver that data and something we're working on as part of the partnership with Microsoft uh is to embed mapping capability directly inside Microsoft Fabric. So that's something you'll see uh later this year or at the beginning of next year coming into uh public preview inside of Fabric. But there are also lots of other applications that you can access that data from. uh whether that's something like PowerBI or whether that's tools uh that we build such as ArcGIS Pro which is a desktop analytical tool um into your Jupyter notebooks or your data science environments for more advanced visualization of spatial data. There are lots of different enduser applications and this wouldn't be a talk at a tech conference if I didn't touch on AI a little bit at the end. Um so when we think about some of the systems that we feed that data in uh using it for training and feeding into AI inference systems is also a big part of data platforms. Um and the same way that we need a spatial lens for our data and our data platform, we also need to have that spatial lens when we think about uh using some of the AI tools for uh extracting content from the data, extracting information from that data. And when we think about the things that AI is really good at, being able to see what's in the data, being able to read and understand uh textual documents, being able to analyze that data, being able to learn from data, these are all things that have a spatial aspect to them as well. So, uh being able to detect objects in imagery, in video, for example. We have a number of uh what we call geoai models that are able to extract data from uh earth observation and imagery data that are able to extract spatial data from documents and um to analyze the tabular data that you have from a geographical perspective and so we need that spatial lens on geo on AI as well. Now at EZRI we build uh what we call geoai tools and these are distributed as part of our platform for all of you to use when you're using the platform uh to take that spatial approach to data. So I guess the takeaways from the talk today are you know there's lots of different types of spatial analysis that you can do and it's a very powerful tool but you need the right tools to be able to do that with your data work with that data and you need those tools to be able to build a data platform that understands spatial data that allows you to work with that spatial data. Uh there are tools there for bringing in real time data. doing uh large batch analytics across your Spark data platform. And then you need to think about the applications that you deliver to your customers and the ways that spatial data can feed in to the business applications that you have to the uh machine learning models to the uh workflows that you have that sit on top of that um data platform. So, I hope you found this kind of interesting and I hope you're a little bit more geographers than you were before you came to this session, even if all you've learned is how latitude and longitude works. But if you want to know more about spatial data, about how to use spatial data across a data platform, come and visit us. I'll be there. There's lots of my colleagues. We've got some of the demos you can see uh in more detail and work out how to uh work with this data if you come to our stand um after this talk. So, thank you.

---
*Источник: https://ekstraktznaniy.ru/video/45749*