LARGflow - Netflow visualizer and DDOS detection

The Ghost in the Arbor: Engineering a Custom DDoS Solution

For years, a ghost haunted our network. Its name was Arbor, a DDoS mitigation tool that had long since reached end-of-life. While it once stood as a sentry, it had become a source of noise, regularly triggering false positives like a common large member’s Wednesday Microsoft Teams meetings that spiked bandwidth just enough to set off alarms.

For an ISP, DDoS attacks are a paradox: they are "rare" threats, yet the majority of modern botnets are capable of overwhelming our entire BGP peering, potentially taking down the entire infrastructure. Balancing this existential risk against the massive capital expenditure of a new enterprise appliance is a constant struggle.

The Risk of Legacy

The old Arbor appliance was kept primarily as a NetFlow visualization tool because the network team was accustomed to its interface. However, it was becoming more of a liability than an asset. We surveyed the market for NetFlow and DDoS tools, but nothing fit our specific needs or budget.

That’s when I decided to reach for the stars. Armed with years of Python experience and a recent "skill upgrade" in AI-augmented coding, I set out to build a replacement from scratch. I understood how Arbor worked. I knew the architecture and design of the project.

A Coder’s Quest: AI as a Force Multiplier

Building an enterprise-grade replacement as a solo developer sounds like a multi-year odyssey. However, by leveraging AI for rapid code generation, debugging, architectural planning, and security reviews, I accelerated the development process dramatically. What would have normally taken months became a focused, manageable project. I built an MVP in only 2 weeks.

The goal wasn't just to replicate the old tool, but to exceed it.

Phase 1: Unprecedented Visibility

The result is a custom-built engine that visualizes NetFlows in real-time. Unlike the rigid filters of the past, this tool allows the team to drill down using any CIDR address from /16 to /32, and everything in between).

The initial insights were eye-opening:ASN Mapping: We discovered Harvard’s ASN (1742) was our #1 Source 95% percentile at 9 Gbps and our #2 Destination.

Protocol Breakdown: Our traffic is 72% HTTPS, while legacy HTTP has dwindled to just 1%.

Phase 2: Intelligence-Led Security

A network tool without a security focus is just a fancy graph. To give this "Ghost hunter" teeth, I integrated my existing threat feed project directly into the core logic. This visibility is in front of member’s firewalls so we saw millions of attacks per day, but it’s unknown what % were getting blocked.

IOC Detection: The tool constantly cross-references flows against known malicious connections.

Smarter DDoS Logic: I addressed my biggest gripe with Arbor—false positives triggered by single sources (like Google or Microsoft IPs). By strictly defining the "Distributed" in DDoS, we eliminated those "one-source" alarms.This logic was put to the test recently during a massive, legitimate attack: 101+ million sessions from 85+ million unique IPs, peaking at over 10 Gbps for three hours. The system caught it instantly.

The Road Ahead: Proactive Defense

The system is already detecting simulated attacks in under 60 seconds. But detection is only half the battle.The next phase of development includes:

MISP/IOC Integration: Automated reporting and deeper threat intelligence.

BGP Flowspec: Integrating ExaBGP to allow the tool to dynamically inject flowspec routes. This will move us from passive alerting to active mitigation, dropping malicious traffic at the edge before it hits our core.The "Ghost" is finally being laid to rest, replaced by a solution that is faster, cheaper, and significantly more intelligent.