Back

All Projects

Quantized KV Cache Preallocation for Long-Context LLM Serving (mlx-lm)
PR #1920 open

2026 · Open source contribution

Diagnosed macOS kernel panics (completeMemory() prepare count underflow in IOGPUMemory.cpp) when serving a 27B model at 64k context on 24 GB unified memory. Traced the root cause to step=256 KV cache reallocation churn — each mx.concatenate call during prefill is a new wired-memory allocation from Metal. Patched mlx-lm's KVCache and QuantizedKVCache to preallocate the full N-token buffer in a single mx.zeros + mx.eval call at creation time, then write in-place. Zero reallocations. Stable 65,536-token serving across a 10-turn stress test with zero crashes.

PythonMLXApple SiliconKV CacheMetalLLM Inference
ml-explore/mlx-lm PR #1920 →
HF Dataset Survey & Multimodal Validation Pipeline

2026

4-stage automated pipeline for systematic CV dataset discovery on Hugging Face Hub. Stage 1: metadata harvesting via HF Hub + Dataset Viewer APIs. Stage 2: extension bucketing and fuzzy deduplication (difflib.SequenceMatcher, similarity ≥0.90). Stage 3: VLM-based visual classification (Qwen2.5-VL/Qwen3-VL) with majority voting across N random samples per dataset, plus heuristic annotation detection. Stage 4: zero-download Parquet schema inspection and disk-safe controlled download. Screened 500k+ items in under 5 hours. Fully resumable with incremental CSV appending.

PythonHugging Face HubQwen-VLVLMffmpegData Curation
github.com/Py528/hf-dataset-survey →
exportrace
🚧 active

2026

CLI tool that benchmarks a YOLO model across every export format available on your hardware — PyTorch/CUDA, ONNX, CoreML, TensorRT — and produces real FPS, latency, and accuracy-delta numbers. Motivated by daily INT8/TensorRT profiling work: every engineer answers "should I use ONNX or TensorRT?" by hand, inconsistently, and throws the results away. exportrace aims to be the Cinebench of CV model deployment — one command, reproducible benchmark report you can drop into a README. Waitlist open.

PythonONNXTensorRTCoreMLYOLOBenchmarking
github.com/Py528/exportrace →
Skill Learn — Sandboxed Code Execution Platform

Sep 2024 – Dec 2024

Web platform that pools compute/RAM across multiple low-end machines into one usable resource for 3D rendering, game dev, and AI inferencing. Engineered secure sandboxed code execution using Docker with CPU/memory limits on AWS ECS Fargate. Implemented PostgreSQL connection pooling (PgBouncer) reducing connection overhead by 70%. Built scalable backend supporting 1000+ concurrent users with strict security isolation.

Next.jsTypeScriptAWS S3DockerSupabasePostgreSQL
Athlete Connect — AI Sports Talent Assessment

Aug 2024 – Oct 2024 · Top 10 National Finalist, Smart India Hackathon 2024 (50,000+ teams)

On-device pose estimation (Google ML Kit) on Android for low-latency, privacy-preserving fitness evaluation. Firebase-backed pipeline for nationwide athlete profiling and leaderboards. Built during Smart India Hackathon 2024, reached national top 10 out of 50,000+ competing teams.

FirebaseKotlinML KitReal-time DB