Dear readers of Data Double Confirm, I hope the content on this blog has been useful so far. I'm equally glad if you happened to chance upon it while doing your data science or data engineering project. Here I would like to take this chance to announce that Data Double Confirm has been invited to Educative's Affiliate Program. If you are looking for more resources relating to data science, software engineering or coding in general, do check out their courses. They have over 700+ courses spanning over different programming languages. You can create an account for free and it will show you some ..read more

Text classification has evolved significantly over the years, driven by advances in natural language processing (NLP) techniques and the availability of larger and more diverse datasets. From rule-based approaches to sophisticated deep learning models, leveraging transfer learning, attention mechanisms, and multimodal approaches, ongoing research continues to push the boundaries of accuracy, efficiency, and ethical considerations in text classification. Specifically, the recent increase in availability of Large Language Models (LLMs) have enabled users to perform text classification tasks more ..read more

Bold.org, the largest independent scholarship provider in the US with millions of students, has recently launched the $90K Barbara J. DeVaney Memorial Scholarship Fund and would be delighted to receive more applications across the country. Here are the basic details of the scholarship: Eligibility: Any female-identifying high school, undergraduate, graduate, or trade school student who’s a single mother, widow, immigrant, woman of color, first-generation college student, or survivor of abuse may apply for this scholarship. Students pursuing careers in the medical field, trades or STEM are str ..read more

June 7th, San Francisco, USA - AI Time Journal, a leading publication in the field of artificial intelligence, is thrilled to unveil its latest release, the highly anticipated eBook "AI Trends 2023". Created by incorporating valuable insights from 36 distinguished industry experts, this publication provides an extensive exploration of the latest advancements, challenges, and emerging trends shaping the AI landscape. The eBook covers a wide range of topics, including: ● The pervasive influence of ChatGPT across diverse industries and its implications for various professional roles ● Ethical Con ..read more

Data Double Confirm is proud to be a media partner for AI Trends 2023 Ebook Launch. AI Time Journal explores how AI and Exponential Technologies bring opportunities for people and organizations to increase their wealth and health. Their audience is anyone interested in improving in their career, their business, their investments, living a healthier and more fulfilling life, or simply understanding how Exponential Technologies are changing the world. Their upcoming ebook covers the latest AI trends and features thought leadership quotes from influencers with a combined following of over 300,000 ..read more

This post shows an example of how to query data using the MAS API. The results returned are in bytes format and has to be transformed to json first which we will then convert to a pandas dataframe. import pandas as pd import urllib.request user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'headers={'User-Agent':user_agent,} url = 'https://eservices.mas.gov.sg/api/action/datastore/search.json?resource_id=5f2b18a8-0883-4769-a635-879c63d3caac' request=urllib.request.Request(url,None,headers) #The assembled request response = urllib.request ..read more

Public can create an account to get data related to courses published on the SkillsFuture Singapore website via their API. This notebook shows how to use the SkillsFuture API to retrieve courses. The results of the API call is converted from json to pandas dataframe and then uploaded as csv to a Dataiku folder. I've also written an analysis on the course data here: https://blog.dataiku.com/using-ml-to-identify-factors-and-biases-that-influence-course-fees ..read more

I'm doing a presentation on 26 Nov 2022 at Google DevFest Singapore 2022 on "Creating a Great Technical Presentation". Join us for an exciting line-up of talks: https://sites.google.com/view/devfest-singapore-2022/home?authuser=0 ..read more

Data Double Confirm is proud to be a media partner for AI Asia Expo 2023. 1. AI Asia Thailand 2023 Date: 21st - 22nd February 2023 Location: Centara Grand at Central Plaza Ladprao Bangkok, Thailand About the event: https://aiasiaexpo.com/thailand/ In Thailand, early adopters are transforming their businesses using AI technologies, and are now successfully scaling its usage throughout their companies. AI Asia Expo, Thailand edition, brings together innovators and thought leaders from across the diverse AI ecosystem and delivers key knowledge for enterprises. This also allows enterprises to bri ..read more

This notebook presents two ways of scraping table data off website, one using pandas and the other using BeautifulSoup: https://github.com/hxchua/datadoubleconfirm/blob/master/notebooks/table.ipynb. This is the table that we want to scrap off the website. And this is the final output in the form of pandas dataframe ..read more