Preet Sojitra

Echo-ViLD: Zero-Shot Acoustic Object Localization

CS 6384 — Computer Vision, UTD

Extended ViLD (open-vocabulary object detection) by replacing CLIP with Meta's PEAV (Perception Audio-Visual-Text) foundation model, enabling a tri-modal detection framework where users can query with audio instead of text. A custom Segment-Guided Distillation pipeline uses SAM to produce background-free, mask-accurate embeddings as teacher targets, eliminating the noisy rectangular crop artifacts that degrade standard OVD distillation.

The result is true Zero-Shot Acoustic Object Localization: given a .wav audio clip, the model localizes the sounding object in an image with no category-specific training. Validated on a 5K COCO subset (resource-constrained), with results on par with baseline ViLD.

Adaptive Speculative Decoding for LLM Inference

CS 6320 — NLP, UTD

Built a speculative decoding engine from scratch in PyTorch targeting Python code generation. Standard speculative decoding uses a small draft model to speculatively generate tokens that a larger target model then verifies, fast when accepted, wasted compute when not.

Introduced two optimizations: Teacher-Student Alignment, aligning the draft model's distribution to the target via top-K KL-divergence loss to directly improve token acceptance rate (α) and Dynamic Lookahead Halting , a lightweight 2-layer MLP trained on draft entropy and token confidence that halts speculation early when uncertainty is high, cutting wasted FLOPs. Both hypotheses were experimentally confirmed.

SVMLite

CS 6375 — Machine Learning, UTD

A lightweight Python SVM library built from scratch for educational and experimental use. Implements both primal (hard/soft margin via SGD) and dual (QP-based via cvxopt) formulations, with full kernel support (linear, polynomial, RBF, sigmoid, custom) and the SMO optimization algorithm.

Supports multiclass classification via One-vs-One and One-vs-All strategies, with grid search and cross-validation for model selection. The goal was to build a thorough understanding of SVM internals rather than wrap scikit-learn.

Depression Detection through Speech Analysis

An end-to-end research project to detect depression from speech recordings using the DAIC-WOZ dataset. Systematically evaluated a range of models, from classic CNNs (VGGNet, ResNet) to Vision Transformers (ViT), on spectrograms and raw audio data.

Accomplishment: Gained experience in the full research pipeline, including feature engineering for audio, comparative model benchmarking, and handling class imbalance.

imgcv: A Image Processing Library from Scratch

Developed and published a Python package on PyPI that re-implements core image processing OpenCV functions from the ground up using only NumPy. The goal was to build a deep, fundamental understanding of how computer vision and image processing algorithms work under the hood.

Accomplishment: Solidified my understanding of algorithms for filtering, color space manipulation, morphological operations, and edge detection.

nanoGPT: Building a Transformer from Scratch

Implemented a decoder-only GPT-style language model from scratch in PyTorch, training it to generate text in the style of Shakespeare. This project was a deep dive into the fundamental mechanics of the Transformer architecture.

Accomplishment: Gained a code-level understanding of self-attention, which I am now extending to train a similar model on the more complex Sanskrit language.

Micrograd: An Autograd Engine in Python

A from-scratch implementation of a scalar autograd engine, demystifying the backpropagation process used in neural networks. Later extended to a PyTorch-like structure.

Accomplishment: Developed a deep understanding of how autograd works, including the mechanics of computing gradients and the chain rule.

Chorale Music Generation with LSTMs

Trained a Long Short-Term Memory (LSTM) network on a dataset of Bach chorales to generate novel, harmonically coherent musical sequences.

Accomplishment: Gained insights into sequence modeling and the unique challenges of generating structured outputs like music.

End-to-End News Classification with MLOps In Progress

Building a full-stack text classification system for the AG News dataset. This project moves beyond a simple notebook to create a deployable, end-to-end MLOps pipeline, featuring custom tokenizers and models built from scratch in PyTorch.

Goal: To gain hands-on experience in MLOps, CI/CD for machine learning, and to solidify core PyTorch concepts by building a production-ready application.

Fine-Grained Animal Classification In Progress

Developing a classification model for a challenging dataset of 90 animal classes with a limited number of samples per class. This project focuses on techniques to handle data scarcity and achieve high accuracy in a fine-grained classification scenario.

Goal: To explore and implement advanced techniques for training robust models on datasets with long-tail distributions and limited data.

Multimodal AI Story Generator

A Streamlit application that generates a unique story from a sequence of user-selected images. The pipeline scrapes images based on user queries, generates descriptive captions for each selected image using a Vision Transformer (ViT), and then feeds the captions to a Mistral LLM to write a coherent narrative.

Accomplishment: Successfully integrated multiple AI models (vision and language) into a seamless, interactive web application, demonstrating skills in building full-stack multimodal systems.