VQ-VAEs are used successfully to transform images into a representative latent space for diffusion models (LDM). For self supervised learning, however, I can’t find people using them much to create an embedding that can later be used as input to downstream models to predict eg image classes. Do you have an idea why that is? Intuitively, I would assume VQ-VAEs should also yield quite nice embeddings. submitted by /u/That_Phone6702 [visit reddit] [comments ..read more

KL divergence loss too high I’m trying to perform knowledge distillation . For my student loss I have used cross entropy loss and for my knowledge distillation loss I am trying to use KL divergence loss. Here is the code that I used for my KL divergence loss. The values that I am getting from this are extremely huge. I am simply adding my knowledge distillation loss and cross entropy loss from student model. Since my CE loss is very small this is all from the KLdiv loss. Could you tell me how to reduce the loss? Or if I am doing something wrong. submitted by /u/Dapper-Plantain-9120 [visi ..read more

Hey everyone, So I was reading a plethora of approaches, especially regarding research done in Multivariate Time Series and real-time Human Activity Recognition (HAR). Though, I recently stumbled upon Eamonn Keogh’s amazing and comprehensive work on Matrix Profiles and ended up in a rabbit hole. But out of curiosity, in general how does the Matrix Profile compare with Deep Learning methods (e.g. MLP, LSTM, etc..) in the context of Multivariate Time Series and real-time streams of data? I would love to hear other people’s perspectives! submitted by /u/peachjpg111 [visit reddit] [comments ..read more

What are some good resources to do reasearch in understanding in context learning. I am planning to do my masters thesis on this. Just thinking of which researchers I need to follow in this topic. submitted by /u/One_Definition_8975 [visit reddit] [comments ..read more

I submitted a paper. Gets accepted to conference. Got email from some random dude from _insert_university_. Sending to both the chair and conference head. Accuses me a plagarism and says 92% matching of publish papers... Check cross reference. Title, authors (me and the mentor), data, conclusion, and almost the entire paper is highlighted. Only source says Arkiv. I have my pre-print on there by chance. I followed their policies with pre-prints and put the notices. Now, this is very stupid. I done a lot of due diligence and if its matching the authors, it has to be referencing my pre-print. Wh ..read more

As we know, PINNs are done by modifying the loss function where we put the physics equations and their constants in the training model. I wonder if the model can be used to replace manual computation for different conditions, e.g. the constants involve other parameters. submitted by /u/angerbit [visit reddit] [comments ..read more

Video: Husky AI: An Ensemble Learning Architecture for Dynamic Context-Aware Retrieval and Generation (youtube.com) Pleases excuse my video, I will make a improved one. I would like to do a live event. Abstract: Husky AI represents a groundbreaking advancement in generative AI, leveraging the power of Advanced Information Lifecycle (AIL) management to achieve unparalleled adaptability, accuracy, and context-aware intelligence. This paper delves into the core components of Husky AI's architecture, showcasing how AIL enables intelligent data manipulation, dynamic knowledge evolution, and it ..read more

Hi r/MachineLearning, We've developed a tool called CaptureFlow: https://github.com/CaptureFlow/captureflow-py Open-source Python project that combines AI (yes, GPT-4) and execution tracing to generate tests based on how your application actually runs. This is particularly useful for older apps with poor test coverage. What It Does: CaptureFlow captures runtime information from your app, uses it to understand how the code behaves, and then automatically generates tests. It's a bit like reversing the usual test-driven development, starting with the code you have and working backwards to create ..read more

I'm wondering how people normally approach reinforcement learning when working/training with real-world systems. submitted by /u/MusicianMike [visit reddit] [comments ..read more

I've been for the past 5 months working on a from scratch PPO implementation. I am doing most of the work from scratch except numerical computation libraries such as numpy. It started with supervised learning networks to now this. And I just can't seem to get it. Every paper I read is A. Outdated/Incorrect B. Incomplete. No paper has a full description on what they do and what Hyper Params they use. I tried reading the SB3 code but it's too different from my implementation and I just don't understand whats happening as it's just so many files, I can't find the little nitts and gritts. So I'm ..read more