Mistral NeMo 12B SOTA. Standford TexGrad. Patch-Level Training. Stanford STORM. State of Open AI. State of Txt2SQL. 450 Real World ML Systems. Convolutional Kernel Networks. EV-5 Universal Embeddings ..read more

Generative AI + Time-Series Forecasting? Many world-class organisations are starting to invest in new GenAI+TS forecasting methods that involve for example: developing new specialised VAEs, using Vision-Language Models, pre-training the model with trillions of TS data points, or incorporating text embedding and tokenisation into the TS forecasting method. Checkout these 6 very recent, interesting papers that show the impressive, rapid evolution in this area. Re-programming LLMs for time-series modelling. This a great post about how researchers are trying to align the information gap between ti ..read more

Prompt Engineering 2.0. Prompt engineering is not going anywhere any time soon. The AI Goliaths have invested 10’s of billions in LLMs & Large Multimodal Models (LMMs), which today -for better or for worse- dominate “modern AI” totally. Due to the way these models were designed and developed, inevitably, to get solid output results from these models you need to instruct them with natural language prompts. Prompt engineering is like Marmite. Many of my hardcore s/w engineers friends absolutely hate prompt engineering. They dismiss it as “random, unreliable pseudo-programming using English l ..read more

AI and While You Were Out IRL. The speed and breadth of AI R&D these days is mind-boggling! This w/e I’ve been immersed IRL joys, including being trapped in airplanes, trains and automobiles. (Apologies for publishing this a day later than usual.) This issue is a bit like an AP News bulletin on what happened in AI when I was AWK. The latest version of DeepSeek-Coder is now the top open model for coding. DeepSeek-Coder-v2 is an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. Repo & paper: DeepSeek-Coder ..read more

On Compound AI Systems, Txt2SQL & Data Agents. A year ago or so, a client enthusiastically presented us with a long list of “AI LLM projects.” Among them, there was one project listed as: “use text-to-sql to automate all data analysis tasks” … We thought: “Umm… this is going to be an interesting project”… Months later the client abandoned the project. Pioneers in Text-to-SQL at enterprise scale. afaik, Pinterest was one of the first companies that deployed Tex2SQL at scale in enterprise production. Importantly, they were one of the first ones in sharing their experience. This is an excelle ..read more

State Space Models (SSMs): An Alt to Transformers? SSMs are not something new; they’ve been successfully applied in many fields like control systems, physics, economics... For now, transformers are the kings in sequence modelling. But recently, some researchers and startups are starting to use a specific type of SSM to solve some of the sequence modelling issues that Transformers suffer from. SSMs as an Alt to Transformers? Let’s see: First, let me share 3 nice intros to SSMs: A gentle introduction to SSMs. In this post, jorgecadete explains SSMs from a basic point of view. By the end of thi ..read more

New Trends in AI-RAG and Graphs. I’ve been doing a bit of research on how to improve RAG with graphs. I’m especially interested in augmenting agentic-RAG with the knowledge graph. A while back, Maya @Neo4j wrote a nice article on The Future of AI: Machine Learning and Knowledge Graphs. I think that makes sense to me. Let me share some new, interesting stuff on RAG and Graphs: Graph RAG. Unlike RAG approaches that focus solely on text-based entity retrieval, GRAG maintains an acute awareness of graph topology, which is crucial for generating contextually and factually coherent responses. The re ..read more

The Google AI Blast . This week OpenAI released a new closed model called GPT-4o (as in omni): Hello GPT-4o, a model that can reason across audio, vision, and text in real time. It seems the model performance in many benchmarks wasn’t as good as many AI pundits expected. And while many people in the AI community were befuddled and discussing the “flirtatiousness” aspects of GPT-4o, then Google came in and blasted a massive AI storm including SOTA models, new powerful open models, and pretty amazing tools. Here’s my summary on what Google released: Gemini 1.5 Pro model updates: Lots of improvem ..read more

Diffusion, FM & Pre-Trained AI models for Time-Series. DeepNN-based models are starting to match or even outperform statistical time-series analysis & forecasting methods in some scenarios. Yet, DeepNN-based models for time-series suffer from 4 key issues: 1) complex architecture 2) enormous amount of time required for training 3) high inference costs, and 4) poor context sensitivity. Latest innovative approaches. To address those issues, a new breed of foundation or pre-trained AI models for time-series is emerging. Some of these new AI models use hybrid approaches borrowing from NLP ..read more

Six Nerdy AI Activities for the Long W/E. I’ve just read that lots of AI engineers in the US are running the rate race, feeling burnout. Here in the European AI scene things are innately a bit more relaxed. Aah… A long bank holiday in London; so much stuff to do in this amazing city! But if you are feeling the AI FOMO kick and can’t survive a long weekend IRL, here are six AI activities for you: Generate comics with AI. I gave it a go, generated a few short comics, and having fun so far. The AI team at Bytedance just introduced an impressive diffusion-based, zero-shot, text-to-image and imag ..read more