Convert JSON to nd-json for bulk indexing into Elasticsearch. movies.json [ { "title_x": "Student of the Year 2", "imdb_id": "tt7255568", "poster_path": "https://upload.wikimedia.org/wikipedia/en/thumb/3/3c/Student_of_the_year_2_Poster.jpg/220px-Student_of_the_year_2_Poster.jpg", "wiki_link": "https://en.wikipedia.org/wiki/Student_of_the_Year_2", "title_y": "Student of the Year 2", "original_title": "Student of the Year 2", "is_adult": 0, "year_of_release": "2019", "runtime": "146", "genres": "Drama|Romance|Sport", "imdb_rating": 2.5, "imdb_v ..read more

Introduction In this gist we will quickly spin a sample Kubernetes cluster and deploying the nginx pod. Additionally, we will implement monitoring using Elastic. Setup K8s cluster Cluster architecture 3 Node cluster Machine - Centos7, 4GB RAM kube1.local - Control plane node kube2.local - worker node kube3.local - worker node Here I am setting hostname kube1.local, kube2.local, kube3.local. Login into all of the servers and perform below command on all three nodes. Swap off sudo swapoff -a Install docker You can refer docker documentation but here are the quick steps: Contanerd runtime ku ..read more

Introduction ES|QL is a new query language for Elasticsearch. It is the unified language for all kinds of use cases like simple queries, aggregations, performing correlations, finding logs, etc. It provides simple easy syntax to perform complex queries. If you come from SQL background, You going to find this very handy. It is a piped separated langugage with a combination of source commands and process commands. The Elasticsearch Query Language (ES|QL) makes use of “pipes” (|) to manipulate and transform data in a step-by-step fashion. This means output of the first step will go as an input fo ..read more

Introduction Search is not just traditional TF/IDF any more but the current trend of machine learning and models has opened another dimension for search. This talk gives an overview of: Classic search and its limitations. What is a model and how can you use it. How to use vector search or hybrid search in Elasticsearch. Where OpenAI’s ChatGPT or similar LLMs come into play to with Elastic. Check how to leverage Leverage ChatGPT with Elasticsearch . Talk Video ..read more

What this talk is all about ? Elastic Stack (Elasticsearch, Logstash, Kibana and Beats) is such a platform which is built for scalability, performance and “You know… for Search”. When you have a system which scales to the horizons of your data, helps you in your data quest, shows you insights - imagine what you can do with it. Talk Video Feel free to comment below, If you have any doubts or suggestion about this talk ..read more

Introduction Bring logs, metrics, and traces from your Kubernetes cluster and the workloads running on it into a single, unified solution. Elastic observability gives better visibility on your kubernetes ecosystem where you can monitor your pods, services, workload etc. Use a centrally managed Elastic Agent to gain visibility into your Kubernetes deployments on EKS, AKS, GKE or self-managed clusters. Talk Video ..read more

Objective In this hands-on workshop, We will learn how to connect ChatGPT to proprietary data stores using Elasticsearch and build question/answer capabilities for your data. In a demo, We will quickly convert your website, FAQ, or any documentation into prompt chat where your user can directly ask a question on your data. Flow Prerequisites You have used ChatGPT :) Good to have understanding around Elasticsearch (Not mandatory, Introduction will be cover) System + Internet connection Docker - Install it from https://docs.docker.com/engine/install/. Having basic python knowledge wil ..read more

In this blog, we will see how you can quickly setup ELK (Elasticsearch, Logstash, Kibana) stack to receive the HTTP webhook. Mostly ELK stack is known for logging purposes. But Elastic stacks are much more beyond the logging use case. Elastic provides Search, Observability & Security you can check more on this with official documentation. What is Webhook ? Webhook enables the two programs to communicate or transfer the data with the help of callback functions / hooks. Now in the modern tech world it is also known as Reverse API, Push API etc. Mostly it is used to send small amounts of data ..read more

1. unknown service runtime.v1alpha2.ImageService Error: pulling image: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.ImageService System configuration centos 9 / 2GB RAM / 2CPU Master Node Same issue on master node. Command [root@kube-master-1 ~]# kubeadm config images pull failed to pull image "registry.k8s.io/kube-apiserver:v1.26.0": output: E0107 14:52:09.997544 4134 remote_image.go:222] "PullImage from image service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.ImageService" image="registry.k8s.io/kube-apiserver:v1.2 ..read more

Sample Queries for Elasticsearch Workshop CRUD # Insert POST meetup/_doc/ { "name":"Ashish Tiwari" } # Insert with id POST meetup/_doc/1 { "name":"Ashish Tiwari" } # Search GET meetup/_search # Update POST meetup/_doc/1 { "name":"Ashish", "company":"elastic", "address":"Navi Mumbai kharghar", "skills":{ "language":["php","java","node"], "database":["mysql","mongodb"], "search":"elasticsearch" } } # search with query GET meetup/_search { "query": { "match": { "address": "navi" } } } # delete DELETE meetup BULK POST _bulk {"index":{"_inde ..read more