While explaining Optuna to a client in the context of hyperparameter tuning, and performing more research on the topic, I came across AutoGluon to perform "AutoML for images, text, and tabular data". After a quick scan of the documentation, I decided to give it a try and see how it performs on a simple project. I always loved the Kaggle competition NYC Taxi Trip Duration, as the data has spatial, temporal, and other traditional attribute information and is a great dataset to test various models and feature engineering techniques. I used a local Apache Spark i ..read more

 Hello, everyone. It has been a while since my last post, and I wanted to explain my absence. I have been working on demanding client projects requiring confidentiality, so I couldn't share anything. But now I'm back and excited to dive into something new and exciting.  Generative AI (GenAI) has gained much attention lately, but I'm taking it to a different level by merging Large Language Models (LLMs) with insights from geospatial analysis. It's GenAI with a GeoSpatial twist. I want to introduce a simple project, "ReAct geospatial logic with Ollama," which uses resources such as the ..read more

Artificial Intelligence / Machine Learning implementations have been part of Esri software in ArcGIS for a very long time. Geographic Weighted Regression (GWR) and Hot Spot Analysis are Machine Learning algorithms. ArcGIS users have been utilizing supervised and unsupervised learning like clustering to solve a myriad of problems and gain geospatial insight from their data.  We just did not call these learning algorithms AI/ML back then.  Not until the recent popularity of DeepLearning that finally blossomed from the AI Winter and made AI a household name. And guess wha ..read more

Yet another post in the continuing saga of the usage of Apache Spark from a Jupyter notebook within ArcGIS Pro. In the previous posts, the execution was always within the ArcGIS Pro environment on a single machine, albeit taking advantage of all the cores of that machine.  Here, we take a different angle, the execution is performed on a remote cluster of machines in the cloud. So, we author the notebook locally, but we execute it remotely. In this notebook, we demonstrate the spatial binning of AIS broadcast points on a Databricks cluster on Azure. In addition, to colocate the data sto ..read more

Yet another continuation post regarding Pro, Notebook, and Spark :-). In this notebook, we will demonstrate a parallel, distributed, share-nothing spatial join between a relatively large dataset and a small dataset. In this case, virtual gates are defined at various locations in a port, and the outcome is an account of the number of crossings of these gates by ships using their AIS target positions. Note that the join is to a "small" spatial dataset that we can: Broadcast to all the spark workers. Brutly traverse it on each worker, as it is cheaper and faster to do so that spatia ..read more

This is a continuation of the last post regarding ArcGIS Pro, Jupyter Notebook, and Spark. And, this is a rehash of an older post in a more "modern" way. Micropathing is the construction of a target's path from a limited set of a consecutive sequence of target points. Typically, the sequence is time-based, and the collection is limited to 2 or 3 target points.  The following is an illustration of 2 micropaths derived from 3 target points: Micropathing is different than path reconstruction, in such that the latter produced one polyline for the path of a target. Path reconstruction los ..read more

Been a while since I posted something, and thank you, faithful reader, for coming back :-) I'm intending to writing a series of posts on how to use Apache Spark and Machine Learning within a jupyter notebook within ArcGIS Pro.  Yes, you can now start a jupyter notebook instance in ArcGIS Pro to create an amazing data science and data exploration experience. Check out this link to see how to get started with a Jupyter notebook in Pro.  But...my favorite hidden "GeoGem", is that Pro comes with built-in Apache Spark, and y'all know how much I love Spark. People think that Spark is inte ..read more