Varesh Patel / Portfolio 2026Machine learning & computer vision

ML & CV ENGINEER / RESEARCHER / BUILDER

Varesh Patel — Machine Learning and Computer Vision Engineer, researcher, and builder. Making data
make sense.

I’m Varesh — I build computer vision and predictive systems, and the interfaces that make them useful.

Research-minded, product-obsessed. I care about evidence, evaluation, and the gap between a model that works and a tool people trust.

16 CASE STUDIES / 3 FLAGSHIP SYSTEMS

Varesh PatelIDENTITY / 001
V / PENGINEERING
WITH INTENT
FOCUS / 01

Vision
systems

Classification, detection, and visual understanding.

FOCUS / 02

Predictive
models

Learning from time-series and real-world signals.

SCROLL TO EXPLORE ↓
VARESH PATELML / CV ENGINEER

BACKGROUND / THE APPROACH

Background & philosophy.

Building reliable systems at the intersection of machine learning and the real world.

My work moves from unstructured data to models people can actually use. I have trained language and vision models from scratch, studied how representation edits change model behavior, and shipped interfaces around the result. I care about the complete system: the data, the evaluation, and the interface.

I’m an M.S. computer science candidate at Rutgers (expected May 2027, GPA 3.98/4.00), working across computer vision, diffusion language modeling, representation editing, and applied AI systems.

Make the result measurable. Make the system understandable.
Read my background ↗
01 / DATA

Work from evidence

Understand the dataset before making a claim about a model.

02 / EVALUATION

Measure what matters

Compare against baselines and make limitations visible.

03 / DELIVERY

Build for people

Turn research into workflows others can inspect and use.

ENGINEERING / SELECTED WORK

Flagship projects.

Three systems chosen for depth: a model trained from scratch, a computer-vision safety prototype, and an evaluated representation-editing framework.

● Deep Learning / Generative Models / ResearchPROJECT / 01

DiffuLLM — Training a Diffusion Language Model From Scratch

A from-scratch discrete diffusion language model, trained and evaluated as an alternative to left-to-right text generation.

500k+TRAINING ITERATIONS
D3PMABSORBING-MASK DIFFUSION
DDPSLURM CLUSTER TRAINING
MY CONTRIBUTION

Designed and built the DiffuLLM system: a bidirectional Transformer, D3PM-style absorbing-mask corruption, masked reconstruction, iterative unmasking, DDP/SLURM training, checkpointing, and qualitative evaluation. The comparative study also includes two independently built SEDD systems by collaborators.

TECHNICAL CAPABILITY

Model training from scratch, distributed GPU infrastructure, and failure-aware evaluation.

ENGINEERING BREAKDOWNPROBLEM · METHOD · EVIDENCE · LIMITS
01 / PROBLEM

Autoregressive generation is the default for text. Can a bidirectional model learn language by denoising masked tokens in parallel?

02 / METHOD

Absorbing [MASK] corruption, logit-normal timestep sampling, masked-only cross-entropy, and confidence-based unmasking from a fully masked sequence.

03 / EVIDENCE

The selected checkpoint (best_step503000.pt) was evaluated on structured recipe completion. Short and medium generations stay coherent; longer outputs show repetition, malformed control tokens, and drift. The README's comparison table also references a 1.2M-step run, so the page treats 503k as the evaluated checkpoint until that discrepancy is reconciled.

04 / LIMITATIONS

A recipe-domain research study with qualitative evaluation, not a broad benchmark or a production model. The 1.2M-step figure is not used as a headline result.

● Computer Vision / Applied AI / HackathonPROJECT / 02

CareWatch — A Computer Vision Safety Monitoring Prototype

A human-in-the-loop safety prototype that turns pose and scene signals into short evidence clips for review.

YOLOv8POSE ESTIMATION
30 SECROLLING EVIDENCE BUFFER
BITCAMPBEST SOCIAL GOOD HACK
MY CONTRIBUTION

Hackathon team build. I worked across the pose-heuristic pipeline, evidence capture, and review workflow, while teammates contributed to the surrounding app and iOS client. The repository is the source of truth for the combined implementation.

TECHNICAL CAPABILITY

Real-time computer vision, multimodal verification, and human-in-the-loop product design.

ENGINEERING BREAKDOWNPROBLEM · METHOD · EVIDENCE · LIMITS
01 / PROBLEM

Caregivers cannot monitor hours of footage. The prototype explores how computer vision could surface potentially concerning interactions without claiming to detect abuse automatically.

02 / METHOD

YOLOv8 pose and person tracking, proximity and fast-motion heuristics, hosted or local scene verification, a rolling 30-second frame buffer, SQLite event logging, push alerts, a React dashboard, and an iOS guardian client.

03 / EVIDENCE

Externally recognized with Bitcamp 2026's Best Social Good Hack on Devpost. The result is a staged prototype, not a validated detector: there are no published precision, recall, sensitivity, or false-positive figures.

04 / LIMITATIONS

A scan or alert is evidence for human review, not proof of abuse. Camera conditions, staged demos, and false positives limit what the prototype establishes.

● LLM Research / Representation Learning / EvaluationPROJECT / 03

ICONOCLAST — Evaluating Targeted LLM Representation Edits

A representation-editing framework that measures the utility cost of changing model behavior, rather than treating refusal removal as a single number.

10MATCHED MODEL COMPARISONS
6RELEASED EDIT ARTIFACTS
OPTUNAPARAMETER SEARCH
MY CONTRIBUTION

Built the Python framework: contrastive residual activation analysis, per-layer direction estimation, benign PCA subspace projection, LoRA edits, Optuna parameter search, a holdout evaluator, a CLI, tests, and released model artifacts. The original research workspace lives in NLP_Project.

TECHNICAL CAPABILITY

Controlled representation experiments, optimization, and utility-aware model evaluation.

ENGINEERING BREAKDOWNPROBLEM · METHOD · EVIDENCE · LIMITS
01 / PROBLEM

Directional abliteration can remove refusals while damaging benign behavior. Can a benign-subspace-preserving edit reduce that collateral damage?

02 / METHOD

Residual activations from harmful and benign prompts, mean/median/variance/hybrid direction estimators, benign-subspace projection, rank-one or low-rank LoRA edits, and holdout evaluation over 20 harmful and 64 benign prompts with first-token KL.

03 / EVIDENCE

Ten matched open-weight comparisons. The repository's selection rule — fewer harmful refusals, then fewer overrefusals, then lower KL — favors ICONOCLAST in all ten, with lower KL in eight. The tradeoffs remain visible: Falcon3 is a behavior win but a KL outlier (6.1448 vs 0.1648), and Yi-1.5 has lower KL under HERETIC. Treat this as optimized holdout benchmarking, not broad statistical confirmation.

04 / LIMITATIONS

No large-N evaluator outputs were available, and the framework changes refusal behavior rather than providing a safety guarantee. It is not a universal intelligence-preservation claim.

● Recent Applied CV / HackRU 2026RECENT / 01

MedCheck — Vision-Assisted Medication Label Verification

An OCR-first medication-label verifier designed around uncertainty, provenance, and human confirmation.

YOLO11nBOTTLE DETECTION
RapidOCRMULTI-FRAME OCR
HUMANCONFIRMATION REQUIRED

CONTRIBUTIONOriginated and directed the project, set the OCR-first architecture and safety invariants, and coordinated integration across the React dashboard, Flask/TimescaleDB backend, and inference services. Teammates and agent-assisted implementation contributed parts of the codebase; the repository's provenance docs record that split.

METHODYOLO11n bottle detection, RapidOCR multi-frame extraction, structured fields with confidence and source metadata, conditional Qwen3-VL escalation, explicit caregiver confirmation, and backend-owned dose and inventory rules.

EVIDENCE & LIMITSThe release manifest reports 523 inference and 81 backend tests at commit 57eb80b; those numbers were not re-executed for this portfolio build. A UI-only mock mode is documented separately. There is no clinical validation. A label scan is evidence about a bottle, not proof of ingestion. The repository documents a working prototype; it is not a clinical decision system or a hosted medication service.

FURTHER EVIDENCE / RECENT BUILDS

More engineering.

Evaluation harnesses, agent tooling, and applied healthcare systems.

AI Systems01

Crucible

A sandboxed adversarial evaluation harness for AI agent security.

Offline deterministic demo
PythonFastAPIReact
EXPLORE PROJECT ↗

RECOGNITION / EXTERNAL VALIDATION

Awards.

Two hackathon builds with externally verified awards. Both are prototypes; neither is a production safety or payment guarantee.

OPEN SOURCE / RESEARCH

Released work.

Software that exists beyond a portfolio card: an installable package, embedded inference, an inference benchmark, and a computer-vision research pipeline.

PYPI / ML TOOLING

lootqdm

A published training-loop toolkit that maps progress, metrics, goals, and run profiles into a terminal interface, with PyTorch and Keras integrations and tests.

EMBEDDED ML

Zypher EdgeAI

A Zephyr / TensorFlow Lite Micro gesture-recognition pipeline with INT8 quantization, CMSIS-NN, RTOS thread diagnostics, and a documented stack-overflow debugging story. The public demo runs in QEMU with mock sensor data.

INFERENCE SYSTEMS

WebGPU Edge AI

A collaborative 16-prompt benchmark comparing browser-local quantized LLM inference with cloud-hosted inference across latency, throughput, memory, and output quality.

COMPUTER VISION RESEARCH

Event-Based Arm Pose

A pipeline for arm-joint regression from event-camera point clouds, using MediaPipe pseudo-labels and a PointNet-based model over DHP19 and MoCA workflows.

EXPERIENCE / IN PRACTICE

Where the work happens.

Research, engineering, and community work across teams and disciplines.

  1. AI & Software Development Intern

    Impruvon Health

    • Built a Python and FastMCP integration exposing nine typed tools for operational and transformed HL7 queries.
    • Implemented multi-location analytics across orders, dispenses, patient records, and clinical data.
    • Automated pull-request demos with Playwright recordings and validated narrated walkthroughs.
  2. Campus Ambassador

    Adobe

    • Created content around Adobe Express and ran workshops for the Rutgers student community.
  3. AI Intern

    Foundation for Ecological Security

    • Built AI bird classification system for IBIS 2.0; handled ~710GB image dataset, ~1,150 species.
    • Developed a two-stage deep learning pipeline achieving ~79% validation accuracy.
    • Delivered GUI/CLI via Gradio for researchers.
  4. AI Intern

    Arcascope

    • Built a fatigue prediction pipeline using heart-rate and step data from 135 participants; evaluated ~20 model iterations.
    • Improved MAE by ~78% (1.17 → 0.25) using PyTorch model with CORAL layer + ordinal losses.
    • Engineered features and used Runge–Kutta integration.
  5. Secretary

    IEEE GCET Student Branch

    • Organized more than 39 technical and networking events and coordinated branch operations and communications.

Ordered by start date, newest first. Overlapping ranges reflect concurrent roles.

CAPABILITIES / TOOLKIT

The technical stack.

Tools and methods selected for the problem, from training to deployment.

Languages

Python · C++ · Java · Swift · TypeScript · JavaScript · SQL

Machine learning

PyTorch · TensorFlow · Scikit-learn · DDP · SLURM

Computer vision

OpenCV · YOLO · CNNs · Image classification

Data & tools

Git · Docker · Linux · Qdrant · Hugging Face

Applications

Discrete diffusion · FastMCP · FastAPI · React · Gradio

RESEARCH / WRITING

Research & publications.

Selected work in pose estimation, recognition, and emotion-aware systems.

SPRINGER / ICDAM 2025 · SHAH, PATEL, JOSHI

OpenPose, PoseNet and MoveNet: the evolution of deep learning methods in yoga pose classification

READ PAPER ↗
2024 / IJIRMF · VARESH PATEL

Machine learning-based hand gesture recognition for immersive gaming

READ PAPER ↗
2024 / IEEE FMLDS · VARESH PATEL, KHYATI MEHTA

Emotion-aware music recommendations: custom CNN vs VGG16 vs InceptionV3

READ PAPER ↗

BEYOND THE CODE / COMPETITIVE SWIMMING

Two Decades in the Water.

Before I was building machine learning systems, I was spending hours in the pool. I’ve been swimming since I was three and competing since 2012. Over the years, I repeatedly won district-level events and advanced to state-level competition through Gujarat’s Khel Mahakumbh, later representing my university at an inter-university aquatics tournament.

  1. 01 / DISTRICTRepeated event winsMultiple competitive seasons
  2. 02 / STATE200m IM medalGujarat Khel Mahakumbh · 2015
  3. 03 / UNIVERSITYCVM UniversitySouth West Zone · 2023–24
2015 / STATE LEVEL

200m Individual Medley medal

Gujarat Khel Mahakumbh · State-level competition

Medal color, race time, and placement are not recorded here.
2023–24 / UNIVERSITY REPRESENTATION

Represented CVM University through GCET at the South West Zone Inter-University Aquatics Tournament, SRM University, Chennai.

OFFICIAL UNIVERSITY RECORD ↗
  • ~20YEARS IN THE WATER
  • 2012COMPETITIVE SWIMMING BEGAN
  • 2015STATE-LEVEL 200m IM MEDAL
  • 2023–24UNIVERSITY AQUATICS

LET’S WORK TOGETHER

Let’s build something
worth understanding.

Have a challenging problem in machine learning or computer vision? I’d love to hear about it.

Start a conversation ↗View résumé ↗
GITHUB ↗LINKEDIN ↗
Ask AIABOUT MY WORK