RTL Optical Flow Accelerator (Lucas–Kanade Structure Tensor) — Details

GitHub repo Demo video (Nextcloud) ⚡ Launch Live Demo

Date: May 2026

Categories: FPGA, Verilog, RTL Design, Computer Vision, Computer Architecture, Hardware Acceleration

Tech stack: Verilog, Vivado 2025.1, Verilator, Yosys, Artix-7 FPGA (XC7A35T), Basys 3

Overview

The structure tensor (second-moment matrix) is a foundational primitive in computer vision algorithms, including Lucas–Kanade optical flow, corner detection, and feature tracking. For a pixel with spatial gradients I_x, I_y and temporal gradient I_t, the engine computes five tensor products summed over a 5×5 spatial neighborhood: S_xx = Σ I_x², S_xy = Σ I_xI_y, S_yy = Σ I_y², S_xt = Σ I_xI_t, and S_yt = Σ I_yI_t.

Computing these sums in software is memory-bound and throughput-limited. This project implements a fully streaming RTL computation pipeline in Verilog that sustains one pixel per clock cycle throughput after pipeline fill, yielding ~122 MP/s at 100–122 MHz.

Pipelined Architecture

The accelerator operates as a 4-stage streaming pipeline with five identical sub-pipelines executing concurrently for each tensor component:

• Stage 1 — Multiplier: Ingests 16-bit signed spatial and temporal gradients (I_x, I_y, I_t) at 1 sample/cycle and computes 5 × 32-bit products (1 cycle latency).
• Stage 2 — Horizontal 5-Tap Box Filter (box_filter_h5): Evaluates a causal sliding window sum across each scanline using a running sum (sum_new = sum_old + new - oldest). Valid after 4-cycle window fill; auto-resets per row.
• Stage 3 — Vertical Line Buffer (line_buffer_5): Circular buffer maintaining 4 prior scanlines in BRAM and presenting a 5-row window to Stage 4 simultaneously using column-major addressing.
• Stage 4 — Vertical 5-Tap Accumulation (box_filter_v5): Combinatorial direct sum of the 5 row values across all components (1 cycle registered latency).

Resource Utilization & ASIC Area Analysis

Synthesized and implemented in Vivado 2025.1 targeting the Xilinx XC7A35T (Artix-7) on Digilent Basys 3:

• LUTs: ~515 / 20,800 (1.48%)
• Flip-Flops: ~459 / 41,600 (1.13%)
• Block RAM: 1 / 50 (2.00%)
• DSP48E1: 1 / 90 (1.11%)
• SRL16E / CARRY4: 32 SRL16E (0.33%), 47 CARRY4 (0.58%)

ASIC gate count mapping with Yosys against the Nangate45 standard cell library revealed that line_buffer_5 accounts for ~25,374 gates out of ~142,841 gates total (vs. ~675 for box_filter_h5 and ~722 for box_filter_v5). This demonstrates quantitatively that data movement and buffering—not arithmetic—is the dominant silicon cost in streaming vision accelerators.

Timing & Performance vs. CPU

• Clock Timing: 100 MHz target constraint (10.0 ns period) achieved with Worst Negative Slack (WNS) of +1.840 ns (critical path ~8.16 ns), yielding an estimated Fmax of ~122 MHz.
• Throughput: Sustained 1 pixel/clock cycle after initial pipeline fill (~122 MP/s at 122 MHz).
• Software Baseline Comparison: Benchmarked against a Verilator-compiled cycle-accurate C++ reference model on CPU:
  • 512×512: 17.48 ms software (15.0 MP/s) → ~7.6× speedup on FPGA
  • 1024×1024: 62.44 ms software (16.8 MP/s) → ~7.3× speedup on FPGA

Power Analysis (SAIF-Driven)

Characterized in Vivado using Switching Activity Interchange Format (SAIF) files captured from dedicated simulation testbenches:

• Idle Mode (Reset Held): 72 mW total (3 mW dynamic, 68 mW static leakage).
• Active Mode (Streaming Gradients): 114 mW total (42 mW dynamic, 72 mW static leakage).
• Dynamic Power Breakdown: I/O pin switching was the largest contributor (~24 mW), followed by line buffer BRAM/signal switching (~9 mW) and register logic (~7 mW). Static power was dominated by baseline device leakage (~68 mW).

FPGA Demo & Verification

• Basys 3 Demo Wrapper (top_fpga.v): Integrated synthetic gradient pattern generator (cycling 8 patterns every 2²⁶ cycles), 16-cycle power-on reset, and real-time hardware status display.
• Readback: Lower 8 bits of S_xx displayed on LED[7:0] (displaying 0x19 / 25 decimal for constant I_x=1), valid_out on LED[8], and full output streamed over UART at 115200 baud 8N1.
• Verification: Multi-phase testbench (accel_tb.v) validating corner-case resolutions, pipeline signal synchronization, and unit testbenches for isolated horizontal/vertical filter blocks.

Key Takeaways

• Streaming Efficiency: Sustaining 1 px/cycle amortizes the vertical fill latency over frame pixels, maximizing throughput without requiring power-hungry high clock frequencies.
• Memory Hierarchy Dominance: Area characterization proves line buffer storage dominates logic footprint; optimizing memory structures (e.g. ping-pong SRAM or line buffer tiling) provides the highest architectural leverage.
• I/O Power Bottleneck: Because dynamic power was concentrated in I/O switching, system-level designs benefit significantly from output bus gating or downstream accelerator integration on-chip.

Files & Reports

Open README Open Project Findings Open Final Report (DOCX) Open Power Report (Active) Open Power Report (Idle)

Ubuntu Server — Details

Overview

Repurposed a custom-built PC to run an Ubuntu Server that hosts ROS2 simulations, Nextcloud, and personal web services with secure remote access via WireGuard.

Stack & Components

• Ubuntu Server LTS
• Docker / docker-compose for service isolation
• Nextcloud (files, calendar, contacts)
• ROS2 containers for simulation workloads
• WireGuard VPN for secure remote access

Operations & Highlights

• Automated container deployments and health checks via systemd timers and simple CI scripts.
• Managed user accounts, scheduled backups (rsync + incremental snapshots), and monitoring with basic alerting.
• Locked down public access with firewall rules and WireGuard-only admin access; used Let's Encrypt for HTTPS on web apps.

Open video demos (Nextcloud)

Sanitized server excerpts

Below are short, public-safe excerpts from the server artifacts (full excerpts available via the links).

server {\n    listen 443 ssl http2;\n    server_name example.com;\n    client_max_body_size 1G;\n    location / { proxy_pass http://127.0.0.1:8000/; }

[Service]\nUser=hibbitr1\nWorkingDirectory=/var/www/portfolio_web\nExecStart=/var/www/portfolio_web/venv/bin/gunicorn --workers 3 app:app -b :8000

PubkeyAuthentication yes\nAuthorizedKeysFile .ssh/authorized_keys\nUsePAM yes

naughty_gates collabora/code  Up 12 days\n\nUFW: default deny incoming; allow 22,80,443,51820

View README & full artifacts

Smart Bench PSU — Technical Breakdown

1. Power Input & Transformation Zone

• USB-C Laptop Charger (Type-C): Provides the initial DC (example: 20V) via USB-C PD.
• USB-C PD Trigger Board: Negotiates Power Delivery with the charger to request the higher voltage (configurable via dials/jumpers), and provides the negotiated DC on dedicated terminals.
• Input Fuse Holder & Fuse: Fast-acting fuse placed immediately after the PD trigger to protect the upstream charger and wiring from downstream faults.

2. Power Regulation & Output Zone (The "Muscle")

• HiLetgo XL4015 DC-DC Step-Down Module: Main adjustable regulator. Steps the PD voltage (e.g., 20V) down to the target output (1.25V–~20V) efficiently.
• Manual / Digital Control: The XL4015's potentiometer is used for manual tuning. The ESP32 later adjusts output through a control circuit to make the supply programmable.
• INA219 Current/Voltage Sensor: Placed at the supply output to measure voltage and current (I2C interface to the ESP32). It measures a small voltage drop to compute accurate current and power values.
• Output Fuse Holder & Slow-Blow Fuse: Protects the supply and load from sustained overcurrent; slow-blow type avoids nuisance trips from inrush currents.
• Banana Jack Binding Posts: Positive/negative output terminals mounted on the enclosure, wired after INA219 and output fuse.
• NAOEVO 16 AWG Silicone Wire & Heat Shrink Tubing: High-current wiring with heat-shrink-insulated joints for safety and reliability.

3. Brains & Smart Interface Zone

• Seeed Studio XIAO ESP32C3: Reads INA219 data via I2C, broadcasts telemetry and accepts commands over BLE to adjust setpoint and limits.
• Android App (Custom): Connects over BLE to display live voltage/current and send commands (sliders/buttons) for voltage/current setpoints.
• Calibration & Test Tools: KAIWEETS HT118E DMM used during assembly to verify voltages, calibrate INA219 offsets, measure continuity, and validate the final system.

How it flows

1) Plug in your laptop charger → PD trigger negotiates 20V → 2) 20V passes through input fuse to XL4015 → 3) XL4015 steps down to desired output (e.g., 5V) → 4) Output flows through INA219 (for sensing) and output fuse → 5) Banana jacks deliver power to the load.

Safety & build notes

• Use properly-rated fuses (input: fast acting near charger max current; output: slow-blow to tolerate inrush).
• Use thick gauge (16 AWG) for high-current runs; shorten high-current traces/wires to reduce voltage drop.
• Insulate all solder joints with eventronic heat shrink and provide strain relief at enclosure pass-throughs.
• Log INA219 and calibration values in firmware; provide software overcurrent cutoffs in ESP32 to protect against faults.

Components referenced: USB-C PD trigger board, HiLetgo XL4015 DC-DC module, INA219 sensor, Seeed Studio XIAO ESP32C3, 16 AWG silicone wire, banana jack binding posts, appropriate fuse holders.

Download BOM (Markdown) Open block diagram

ESP32 (XIAO) — INA219 + BLE example (pseudo-code)

            #include <Wire.h>
            #include <Adafruit_INA219.h>
            #include <BLEDevice.h>

            Adafruit_INA219 ina219;

            void setup() {
                Serial.begin(115200);
                Wire.begin();
                ina219.begin(); // initialize INA219
                // start BLE peripheral, advertise service + characteristic for telemetry/control
            }

            void loop() {
                float v = ina219.getBusVoltage_V();
                float i = ina219.getCurrent_mA() / 1000.0; // amps
                // publish v and i over BLE characteristic
                // check incoming BLE commands for new setpoints, apply to control circuit
                delay(250);
            }
                            

FPGA Systolic Array Accelerator — Details

GitHub repo Open final report (PDF) LED demo video (Nextcloud)

Date: Jan 2026 - May 2026

Categories: FPGA, Verilog, RTL Design, Computer Architecture, Hardware Acceleration

Tech stack: Verilog, Vivado, Verilator, Yosys, FPGA, Fixed-Point Arithmetic

Overview

Real-time graphics pipelines rely on repeated matrix-vector transforms. This project accelerates those transforms using a pipelined systolic array that keeps data local and sustains high throughput. Systolic arrays are well-suited to regular MAC workloads because each processing element performs a small, fixed operation while data streams through in a predictable rhythm. Results are compared against a sequential C++ baseline and a sequential RTL reference on a Basys 3 (Artix-7).

Architecture

Hierarchy from MAC to PE to the full 4x4 array, plus pipeline placement in the graphics flow.

Each PE hosts a MAC pipeline stage; PEs are tiled into a 4x4 array for matrix-vector transforms. The array uses 16 DSP48E1 blocks (one per PE).

Implementation

• Fixed-point Q16.16 arithmetic for stable performance across FPGA resources.
• Verilog RTL with parameterized widths and array dimensions.
• Pipelined dataflow with deterministic 9-cycle latency.

Results

• Throughput: 100M ops/sec at 100 MHz.
• Latency: 9 pipeline cycles (~90 ns).
• Resource utilization: 16 DSP48E1, 556 LUTs, 1339 registers.
• Power: ~9 mW dynamic compute power, 0.09 nJ/op energy efficiency.
• Dynamic power computed from SAIF-based active vs idle runs in Vivado.
• Simulation waveforms and testbench results are included in the final report.

Demo Behavior

• SW0 selects idle or active mode without changing the clock tree.
• Active mode lights LD3 solid and flickers LD0 with data-dependent activity.

Key Takeaways

• Throughput gains come from aggressive pipelining, with a known latency cost.
• Localized communication in systolic arrays reduces routing pressure and improves timing closure.
• DSP-heavy designs can deliver strong energy efficiency for structured math workloads.
• Hardware design requires explicit tradeoffs that software abstractions often hide.

Macro Keyboard — Details

Overview

Custom 12-key macro pad built for productivity: hand-wired matrix, per-key RGB, and flexible multi-layer firmware for OS-specific profiles.

Hardware

• Raspberry Pi Pico (RP2040) running CircuitPython / KMK
• 12 mechanical switches, diode matrix, WS2812B LEDs
• 3D-printed enclosure and custom keycaps

Firmware & Features

• KMK for multi-layer maps, macros, NKRO, and HID reports
• Layer-tap combos, sticky modifiers, and custom macros for repetitive workflows
• RGB profiles with presets and low-power behavior when host sleeps

More Pictures

Debian ARM Install — Details

Overview

Installed Debian ARM on a Lenovo Duet 5 by building a UART-over-USB SuzyQable to talk with the Cr50 and resolving bootloader and device-tree issues.

Key tasks

• Built SuzyQable (UART-over-USB) to access Cr50 debug console and bootloader.
• Flashed Debian image, patched device-tree entries for touchscreen and peripherals, and configured bootloader.
• Maintained a recovery image and detailed rollback steps.

Notes

• Documented kernel patches and DT overlays required for hardware support.
• Used serial logs and JTAG where available to diagnose boot failures.

Arduino Rover — Details

Overview

Developed navigation and obstacle-avoidance algorithms for an Arduino autonomous rover used in coursework. The project includes a control-algorithm block diagram and the navigation codebase (now included in the repository).

Download navigation code (INO) Open control algorithm diagram

Hardware

• Arduino Uno
• Wheel encoders, motor driver, and DC gearmotors
• Screw-wheel omnidirectional locomotive system: an innovative screw-wheel design that enables omnidirectional movement without a conventional differential/holonomic drive. This system provides superior traction in loose terrain and allows lateral, diagonal, and rotational motion via coordinated wheel actuation.

Software

The rover firmware handles position-aware navigation, sensor-based data collection, and low-level motor/actuator control using the Enes100 location API. It is organized into a small set of responsibilities:

• Initialization: calls Enes100.begin(), configures PWM/digital pins for motors and an arm, starts Serial, and sets tuning parameters (e.g., error, EPSILON).
• Sensors: reads a magnet sensor on A4 and measures a pulse input (duty cycle) on PulseIN; uses Enes100.updateLocation() for x/y/theta.
• Actuators: an arm driven via PWM (pin 5 and control pin 13) and four wheel motors controlled by per-wheel PWM + direction pins. Helper functions armUp/armDown/armStop and motion primitives (strafeLeft, strafeRight, forward, backward, turnLeft, turnRight, stopMotor) encapsulate hardware commands.
• Navigation: higher-level helpers implement waypoint steering: setTargetLocation(x,y), calcTargetTheta(), orientToTarget() (rotate until angular error within threshold) and goToTarget() (iteratively orient then drive forward until within position tolerance).
• Task flow: the main loop sequences an approach to the pylon, runs getHeading() to position and raise the arm, then executes getData() (magnet check + duty-cycle averaging). After data collection the code drives to a top location, proceeds to an end location and performs an end-strafe maneuver.
• Data collection & decision logic: getData() samples the magnet sensor for a fixed window and accumulates counts to decide magnetic vs non-magnetic. Duty cycle measurements are averaged over several pulse readings. A boolean pylonUp tracks which side the pylon was found on and changes strafing accordingly.
• Safety & tuning: the code uses simple time-based waits and explicit stopMotor() calls; motion thresholds (error, positional tolerances) and parameters like displacement and PWM values are exposed as constants to tune behavior.

Notes

• Optimized for microcontroller constraints and robust to sensor dropouts. The navigation code is organized into modular components: sensor/, control/, navigation/, and utils/ (see the INO file for inline comments).
• The control block diagram shows how sensor inputs flow into state estimation, the waypoint manager, and the motion controller which outputs per-wheel velocity setpoints for the screw-wheel actuators.
• Because the screw-wheel locomotive system accepts vector velocity commands, the motion controller translates desired chassis velocities (vx, vy, omega) into individual wheel setpoints using the kinematic mapping defined in the block diagram.

MPECWatch — Details

Overview

MPECWatch is a web application that aggregates and visualizes minor-planet observation records for astronomers and station operators.

Project website Project journal GitHub repo

Tech & Contributions

• Python scraping and ETL pipelines to ingest MPEC and station data
• SQLite database optimizations for a dataset with 1M+ records (indexed queries, batched inserts)
• Visualization tools using Matplotlib and client-side plotting for observatory browsing

Notes

• Improved runtime by ~40% on key scraping jobs and added robust deduplication logic.
• Implemented station filters and user-friendly pages for per-observatory analysis.

MCP Personal Shopping Assistant — Details

GitHub repo Demo video (coming soon)

Focus: LLM-ready tooling for customer segmentation and campaign drafting

Tech stack: TypeScript, Node.js, MCP, Ollama/OpenAI integrations

Overview

Backend service that exposes MCP tools to retrieve customer insights (LTV, recency, categories) and generate email/SMS copy. Ships with mock data for demos and an optional live Klaviyo API client.

Key capabilities

• Segment analytics with LTV and recency summaries.
• Tool-driven campaign drafting for email and SMS.
• Dual-mode client for mock data or real Klaviyo API calls.
• CLI demos for fast, repeatable evaluation.

Files

Open README View entrypoint (index.ts)

EMIT Image Labeller — Details

Overview

Image labeling tool built to accelerate annotation of methane super-emitter hotspots using map-based workflows and batch labeling by geoproximity.

Features

• Google Maps integration to visualize image coordinates and nearby infrastructure
• Batching by geoproximity to label clusters efficiently
• CSV export with metadata for ML training pipelines

Tech stack & notes

• Python back-end for image processing and CSV generation
• Front-end map UI leveraging Google Maps API
• Designed for scale and reproducible labeling workflows; integration with training pipelines yielded ~83% CNN accuracy in early experiments

Examples

Download EMIT_categories.csv View batch_images.py

Real-time diagnostics from this self-hosted Ubuntu machine

Intro