Skip to content

IDS / IPS

The firewall and access control article ended on a specific gap: a firewall only checks whether traffic matches an allowed rule, and an allowed connection can still carry an attack — a SQL injection string inside an HTTP request to port 443, a brute-force login attempt over a permitted SSH connection, a scan crafted specifically to blend in with normal traffic. The nmap article already noted that a SYN scan's half-open attempts often leave no trace "beyond a firewall or IDS specifically watching for that pattern" — this article is about what that watching actually is.

An intrusion detection system (IDS) is, in NIST's own phrasing, software for "monitoring the events occurring in a computer system or network and analyzing them for signs of possible incidents." An intrusion prevention system (IPS) does the same analysis but sits inline and can block what it finds, rather than only reporting it. The distinction sounds small — detect versus detect-and-block — but it changes where the device sits on the network and what happens when it's wrong.

Where an IDS looks, and how it decides

Two placements matter, because they see different traffic entirely.

A network-based IDS (NIDS) watches traffic on the wire — usually fed by a switch's mirrored port or a network tap — and inspects packets as they pass, the same vantage point Wireshark uses for packet capture, just automated and running continuously instead of triggered by an engineer. A host-based IDS (HIDS) runs on an individual server and watches what happens locally: file integrity changes, unexpected process launches, modifications to files like /etc/passwd that shouldn't change on their own. A NIDS catches an attack in transit across the network; a HIDS catches what already landed on a specific machine — the two answer different questions, and a mature security setup typically runs both rather than one instead of the other.

However it's placed, an IDS decides "is this an incident" one of two ways:

Signature-based detection matches traffic against a database of known attack patterns — a specific byte sequence in a known exploit, a request path associated with a particular vulnerability scanner, a DNS query pattern typical of a specific malware family's command-and-control traffic. This is fast and produces very few false alarms for attacks it already knows about, and it is completely blind to anything new. A signature-based IDS cannot flag an attack that doesn't match any signature in its database yet, which is exactly the gap a zero-day exploit — an attack targeting a vulnerability nobody has written a signature for — walks straight through.

Anomaly-based detection takes the opposite approach: build a baseline of what normal traffic or system behavior looks like, then flag deviations from it — a server that normally handles a few hundred connections a minute suddenly handling tens of thousands, a user account that has never once logged in outside business hours doing so at 3 a.m. This can catch genuinely new attacks that no signature yet describes, at the cost of a real, ongoing problem: legitimate but unusual behavior — a marketing campaign driving a traffic spike, a developer running a rare but authorized batch job — triggers the same alert a real attack would, and a system tuned loosely enough to avoid those false alarms usually ends up missing some real ones too.

A signature-based IDS tells you when something matches an attack it already knows; an anomaly-based one tells you when something looks wrong, whether or not it's actually an attack. Most production deployments — Snort and Suricata being the two most widely used open-source engines — run primarily on signatures, with anomaly detection layered on top for the categories signatures can't cover.

What actually gets inspected

A signature match can happen at more than one layer, and knowing which layer explains what an IDS can and can't see.

  • A signature written against IP or TCP header fields — an unusual flag combination, a spoofed-looking source address — works on any traffic, encrypted or not, because headers are never encrypted even when the payload is.
  • A signature written against the payload itself — a specific string in an HTTP request body, a known exploit's byte pattern in a file transfer — needs the payload in plaintext. Traffic protected by TLS hides exactly that content from a NIDS sitting passively on the wire, which is the single biggest limitation of network-based detection as HTTPS became the default rather than the exception. Some enterprise deployments address this by terminating TLS at a proxy the IDS also has visibility into, inspecting traffic in plaintext at that point — which is a deliberate, consequential trust decision, not a free upgrade, since it means something in the middle now holds the private keys or a trusted intercepting certificate for traffic that was supposed to be end-to-end encrypted.

IPS: the same analysis, with the power to act

An IPS runs the identical detection logic — signatures, anomaly baselines, or both — but sits inline in the traffic's actual path rather than watching a mirrored copy, which means it can drop a malicious packet, reset a connection, or block a source address before the traffic ever reaches its target.

That inline placement is also the IPS's biggest operational risk. An IDS that misclassifies traffic produces a false alarm — an analyst investigates, finds nothing, and no legitimate traffic was ever affected. An IPS that misclassifies traffic the same way blocks it — a false positive on an IPS actively drops real, legitimate connections, and depending on placement, an IPS failing or being misconfigured can take an entire link down rather than just failing to warn about something. This is why IDS deployments are common in places an IPS would be too risky to run in blocking mode: a new signature set is frequently rolled out to a NIDS in pure alerting mode first, specifically to observe its false-positive rate against real production traffic before anyone trusts it to block anything automatically.

Never enable IPS blocking mode against production traffic without first observing it in detection-only mode

A newly deployed or newly updated signature set can misfire against traffic that's unusual but entirely legitimate. Run it as an IDS — alerting only, nothing blocked — against real traffic for long enough to see its false-positive rate, and only then switch the same device or signature set to inline blocking. Skipping this step risks an outage caused by the very tool meant to prevent one.

A signature firing, read end to end

A Suricata sensor watching a mirrored switch port logs the following alert, in its standard fast.log format:

07/27/2026-14:22:18.334567  [**] [1:2010935:2] ET SCAN Potential SSH Scan [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 198.51.100.44:51223 -> 203.0.113.10:22

Reading it left to right: a timestamp, then [1:2010935:2] — the signature ID, revision, and generator ID that identify exactly which rule matched — followed by the human-readable rule name (ET SCAN Potential SSH Scan, from the Emerging Threats open ruleset), a classification and numeric priority, the protocol, and finally source and destination addresses with ports: 198.51.100.44 connecting repeatedly to port 22 on 203.0.113.10.

None of that alone proves an attack happened — a single connection attempt to port 22 is completely ordinary. The rule fired because this particular signature isn't looking at one packet; it's counting connection attempts from the same source against the same destination port within a time window, and 198.51.100.44 crossed that threshold. Confirming it as a real scan, rather than a monitoring tool or a legitimate client retrying a flaky connection, means checking the SSH server's own auth log for a matching burst of attempts:

sudo grep '198.51.100.44' /var/log/auth.log | tail -5
Jul 27 14:22:15 web01 sshd[19204]: Failed password for invalid user admin from 198.51.100.44 port 51201 ssh2
Jul 27 14:22:16 web01 sshd[19205]: Failed password for invalid user test from 198.51.100.44 port 51207 ssh2
Jul 27 14:22:17 web01 sshd[19206]: Failed password for invalid user oracle from 198.51.100.44 port 51215 ssh2
Jul 27 14:22:18 web01 sshd[19207]: Failed password for invalid user admin from 198.51.100.44 port 51223 ssh2
Jul 27 14:22:18 web01 sshd[19208]: Failed password for invalid user postgres from 198.51.100.44 port 51223 ssh2

That log corroborates the alert independently: a rapid sequence of failed logins against different usernames, from the same source, in the same second the IDS flagged — a credential-stuffing or username-enumeration attempt, not a fluke. This is the actual workflow an IDS alert triggers in practice: the alert says look here, and a second, independent source — an application log, a different sensor, a host-based check — is what turns a signature match into a confirmed incident worth acting on.

Practice exercises

  1. A newly deployed IPS blocks a burst of legitimate traffic from a company's own load-testing tool during a scheduled performance test, mistaking it for a DoS attempt. Using the detection-only-first principle above, explain what step in the deployment process was skipped.
  2. Explain why a NIDS positioned to inspect a company's internal HTTP API traffic loses most of its payload-level detection ability the day that internal traffic is switched from plain HTTP to HTTPS, and describe the one architectural change that would restore that visibility.
  3. Given the Suricata alert log line shown above, identify which three fields you would need from a second data source to independently confirm the alert wasn't a false positive, without re-reading the rest of the article.

An IDS or IPS still only sees what's actually reachable — a scan, a login attempt, an exploit payload, all of it arriving as ordinary-looking traffic that simply carries bad intent. Some attacks don't try to sneak anything past detection at all; they just send an overwhelming volume of otherwise unremarkable requests until the target can't keep up. That's a different failure mode entirely, and it's next.

Sources