Schatt

This is the website of Subhajit Chatterjee. I have a passion for deep learning and machine learning. This is my effort to share my experiences and knowledge that I gain with the world.

• Shrinking Models, Expanding Possibilities: A Dive into Quantization

  My previous experiences of being GPU-broke along with irritating OOM errors led me to wonder if there was a way to compress the models for use on edge devices and devices with lesser compute. This led me to discover quantization.¹ 1. I remember someone telling me that Tim Dettmers wrote the CUDA kernel for quantization in one sitting. Absolutely legendary!

  I am writing this blog in an effort to share the learnings and also to test my own understanding in this topic.

• Some thumb rules for efficient inferencing and cost planning of transformer models

  Inferencing models can be costly. This blog explores cutting-edge techniques for optimizing model inference, enabling high-performance with reduced computational costs.² 2. This exploration was inspired by several blog posts including kipply's blog.

  It is interesting how optimizing model inference through techniques like reducing memory-bound operations and GPU kernel fusion can significantly boost performance and resource efficiency.

• LoRa & QLoRa: A Deep Dive into Efficient Fine-tuning Methods

  LoRA and QLoRA are like the secret sauce for model fine-tuning—less weight, more power. You get all the benefits with a fraction of the overhead, making you wonder why you didn’t try them sooner.³ 3. I heard the idea of LoRa in 2024. This post explores various techniques for optimizing memory access in parallel computing environments.