Abrit Pal Singh

ML Engineer with 3+ years of experience in edge AI deployment, LLM-based agentic systems, and speech processing. Currently focused on on-device LLM deployment and GUI automation for automotive head units on Qualcomm hardware.

GitHub · LinkedIn · X · abritpal@gmail.com

Experience

Machine Learning Engineer — Toyota Connected India

July 2022 – Present · Chennai, Tamil Nadu

• Optimized an on-device multimodal GUI agent (VLM + text LLM via llama.cpp) for an 8-core ARM64 automotive SoC with no usable GPU — recompiled with ARM DOTPROD int8 + KleidiAI kernels for a 6.4× gain in prompt-eval throughput (18→115 tok/s), targeting the compute-bound prefill phase.
• Designed a multi-layer caching architecture — perceptual screen-similarity cache, content-hashed embedding cache, and KV prefix-cache reuse — cutting end-to-end step latency from ~53s cold to ~7s warm (7–8×) on repeated/looped tasks, with ~95% prompt-token reuse on cache hits.
• Evaluated integrated GPU offload and rejected it after benchmarking (~2.5× slower than CPU due to shared-RAM SVM and per-kernel dispatch overhead for single-token decode) — kept inference CPU-only and fully local, with no cloud dependency.
• Packaged the agent as a single statically-linked ARM64 binary (PyInstaller, cross-compiled via Docker buildx/QEMU) with mixed 8/5/4-bit quantization across both models, enabling offline deployment with no Python runtime on-device.
• Designed and implemented a POC for LAM (Large Action Model) on Android devices for Automation Testing and Computer Use, using Sonnet 3.5 and GPT-4o. Finetuned LLAMA/OpenCUA-7B for Computer Use tasks.
• Designed a graph-based UI navigation system for LAM using structured JSON scene representations and BFS traversal for generating action trajectories on automotive head units.
• Built the retrieval and response pipeline for an internal AI-powered chat application (ChatGPT-like interface, data privacy within a VPC), and refactored a Gen-AI app to support multiple LLMs with token-cost estimation.
• Built a translation application that processes and maintains formatting across PowerPoint, Excel, PDF, and image-based documents.
• Developed VQ-VAE based models for Acoustic Unit Discovery in end-to-end speech systems, cutting training time 60% with One-Cycle/Warmup LR schedulers; refactored evaluation toolkits (ABX Discriminability, Phoneme Prediction/PER). Paper preprint with IIT Madras: Textless NLP – Zero Resource Challenge with Low Resource Compute.
• Own CI/CD and IaC pipelines for production environments on AWS.

Software Developer — Blod.in

July 2020 – July 2021 · Chennai

Built the front-end of a health-tech web application using React.js and Firebase. Worked closely with the back-end team on API integration and contributed to UI/UX decisions.

Frontend Developer — Freelance

Feb 2020 – May 2020

Designed and developed the front-end of a student assignment-help platform with React, building screens for tutor and student workflows.

Research & Open Source

Cohere Aya Expedition — Multilingual LLM Safety Research

2025

Interpretability research on Tiny Aya (3.35B): replicated and extended Zhao et al. (2025) across six languages — 85% → 11% refusal collapse on template removal, cross-lingual SAE refusal features (AUC 0.93); built the layer-wise type-alignment analysis module.

Publications

1. Textless NLP – Zero Resource Challenge with Low Resource Compute. arXiv. With IIT Madras.
2. Agent Enhancement using Deep RL Algorithms for Multiplayer Game (Slither.io). Journal.
3. Humor and Offense Detection and Classification using ColBERT Embeddings. ACL Anthology.
4. Humor Detection and Funniness Score Prediction using Deep Learning Techniques. Academia.edu.

Talks

October 2025, talk about “Agentic RAG in Edge Applications” at Techceleration #23: Let’s Talk Tech, Toyota Connected India. Watch on YouTube.

Skills

Education