Network usage log by IP address

[KryptoNyte]

Is anyone aware of a router, or other hardware that can monitor the Internet only usage of a particular IP address on a peer to peer network of about 25 computers? I would prefer it were something like a router or managed switch with exactly this type of logging feature rather than software that would need to be installed on each Windows machine.

Windows 10 upgrade has wreaked havoc in our small office with unnecessary [default] Internet bandwidth usage, and even after disabling all the bullcrap peer to peer update sharing with the rest of the world, it's been a headache and we need to track down what machine is causing the problem at any given moment.

[Fuzzy]

pretty sure this type of thing can be done by pfSense. I know I can at least monitor bandwidth usage on my home network by local IP address. A really cheap old Atom PC can make a very powerful pfSense router (mines been running quite well without a hiccup for almost a decade now I think).

[KryptoNyte]

We're checking out pfSense, thanks, Fuzzy.

If anyone knows of an "out-of-the-box" router or switch that is capable of this, please let me know. I'm amazed this isn't a more common issue, with so many ISP's capping data these days.

[KeithAbbott]

I bought one of these about six months ago:

http://store.netgate.com/ADI/RCC-VE-2440.aspx

and loaded pfSense on it. Took about 1/2 hour to get it up and running. If that's still too much work, the same company sells an identical unit for $150 more that is preloaded with pfSense and includes some technical support.

I bought it primarily because my old router had slower throughput than what I was purchasing from my internet provider. My new router has gigabit throughput, at least the way I have it configured (minimal configuration, no add-on packages installed). I can't take full advantage of this throughput, but I also shouldn't need to buy another router for quite some time.

This unit sips electricity thru a very small straw, 7 watts when I connected it up to a Kill A Watt.

[KryptoNyte]

I did find some pfSense forum conversations where they claim that if the user wants reports/logs, that they must be written to the system, and that it can't be the flash memory. Not sure why that would be, but I assume an SSD could be added [by the user] to units like the one Keith had linked.

I see what you mean about it being exactly the same unit you'd get from pfsense, but saving the 150 bucks by user installed OS. Is pfSense basically an operating system of sorts on its own?

[KeithAbbott]

Originally Posted by KryptoNyte

Is pfSense basically an operating system of sorts on its own?

From Wikipedia: pfSense is an open source firewall/router computer software distribution based on FreeBSD.

[KryptoNyte]

Do you folks have any concerns with an open source software acting as a entity's primary defense against attacks? Maybe it's better than the corporate baked software on any common router today ... I'm thinking more in terms of critical business data here.

We've always shied away from running our own email server, primarily for the uptime issues relative to the reliability of crappy DSL, but also with regard to the knowledge required to do it right, and securely.

[KarylFStein]

Originally Posted by KryptoNyte

Do you folks have any concerns with an open source software acting as a entity's primary defense against attacks? Maybe it's better than the corporate baked software on any common router today ... I'm thinking more in terms of critical business data here.

We've always shied away from running our own email server, primarily for the uptime issues relative to the reliability of crappy DSL, but also with regard to the knowledge required to do it right, and securely.

The argument for open source security tools is that while the bad guys have access to the source which makes it easier to search for flaws, so do the good guys. With something as popular as pfSense, you're probably gaining on the security side having it open source and running on FreeBSD. Critical business data you might want in a different security zone too, (e.g. behind a second firewall).

As far as logs I imagine that you can also set up a syslog server if you don't already have one and direct the logs there.

[Tiki]

Originally Posted by KarylFStein

The argument for open source security tools is that while the bad guys have access to the source which makes it easier to search for flaws, so do the good guys. With something as popular as pfSense, you're probably gaining on the security side having it open source and running on FreeBSD.

That's true, but only if you make sure to keep it patched and up to date.

[Fuzzy]

Originally Posted by Tiki

That's true, but only if you make sure to keep it patched and up to date.

Which pfsense makes it very easy to do. Regarding the logs, I'm pretty sure you can direct them to a network share somewhere as well.

[wayner]

And isn't a lot of the common software used for web servers, etc. Open Source. Like Apache or Nginx for instance?

[reggie14]

I'm another pfsense user.

My main warning is that pfsense has pretty crappy traffic monitoring tools. You can get total incoming/outgoing traffic pretty easily, but to see what individual devices are doing you either need to use the darkstat package or ntopng.

Darkstat isn't going to tell you much except for traffic totals from individual hosts broken down by incoming and outgoing ports. I imagine you'd find that useful but perhaps not detailed enough to really pinpoint the problem.

ntopng should give you enough information to track things down, but it doesn't archive statistics about each host for long. I really don't understand how long it keeps that data. Looking at my own data, it looks like it's most the traffic totals come from the last 3-5 days.

[hoep]

Have you ever looked on the Ubiquity Unify (Unify Security Gateway) or Edgerouter equipment ? Does everything you want including DPI and isin the case of the unify line manageable with an raspberry or already existing computer. My total home is on Unify equipment - fairly easy to manage.
br
hoep

[panteragstk]

I use Sophos UTM. Short version is that it is an alternative to pfsense. This is not open source and is an enterprise UTM that has a free version limited to 50 IP addresses with some (that most won't use) features disabled.

Check it out. Rock solid on my system.

[KryptoNyte]

Originally Posted by reggie14

My main warning is that pfsense has pretty crappy traffic monitoring tools. You can get total incoming/outgoing traffic pretty easily, but to see what individual devices are doing you either need to use the darkstat package or ntopng.

That part is a little off-putting. The only reason we need a new router is to closely monitor traffic for any specific IP address on the network. There are a lot of devices in-house now that communicate automatically over the Internet, and it's annoying that we can't determine exactly how much, per IP, per day, and simple chart.

[KarylFStein]

Originally Posted by KryptoNyte

That part is a little off-putting. The only reason we need a new router is to closely monitor traffic for any specific IP address on the network. There are a lot of devices in-house now that communicate automatically over the Internet, and it's annoying that we can't determine exactly how much, per IP, per day, and simple chart.

As hoep mentioned Ubiquity does have a nice dashboard. I use it at home although just for WiFi. The wireless APs don't have DPI (deep packet inspection) so I can only tell things like how much data a certain client has up/downloaded in the past period of time, how long they've been connected, etc. But one of their security gateways or (I believe) edge routers with DPI can give you information about destinations, etc. I've been pretty impressed with the WiFi management and when it comes time to get a new gateway or switch or camera I'm going to seriously consider them.

[reggie14]

Originally Posted by KryptoNyte

That part is a little off-putting. The only reason we need a new router is to closely monitor traffic for any specific IP address on the network. There are a lot of devices in-house now that communicate automatically over the Internet, and it's annoying that we can't determine exactly how much, per IP, per day, and simple chart.

I wrote my last post late at night after being frustrated that ntopng isn't working as it should and there aren't any devs with pfsense that particularly cares to keep it up-to-date. It probably came across a little more negative than I intended.

Yes, it's disappointing that traffic monitoring is at-best an afterthought in pfsense, but using pfsense with the ntopng package would probably get you plenty of information to diagnose the problem as long as it occurs regularly. ntopng will keep some basic history information for a while, but it works better to see what's going on current/recently.

I wish the pfsense guys would put a little more effort into traffic monitoring, but part of the problem is that the open source tools aren't great. ntopng is probably the best, but it doesn't give good historical stats.

[hoep]

Originally Posted by KarylFStein

As hoep mentioned Ubiquity does have a nice dashboard. I use it at home although just for WiFi. The wireless APs don't have DPI (deep packet inspection) so I can only tell things like how much data a certain client has up/downloaded in the past period of time, how long they've been connected, etc. But one of their security gateways or (I believe) edge routers with DPI can give you information about destinations, etc. I've been pretty impressed with the WiFi management and when it comes time to get a new gateway or switch or camera I'm going to seriously consider them.

Indeed for DPI you need a (quite cheap) security gateway, which also works as my main router connected to a UTM firewall.
regards
hoep

[KryptoNyte]

Originally Posted by hoep

Have you ever looked on the Ubiquity Unify (Unify Security Gateway) or Edgerouter equipment ? Does everything you want including DPI and isin the case of the unify line manageable with an raspberry or already existing computer. My total home is on Unify equipment - fairly easy to manage.
br
hoep

I just watched a couple Youtube videos on the Edgerouter Lite - it looked really good for a bit, but it seems that the Tx and Rx traffic monitor doesn't discriminate between local traffic and Internet traffic. Because the computers on the LAN communicate with gigabytes of data locally, it seems that the traffic monitoring wouldn't give us any indication of a specific computer using all the Internet bandwidth. Is there a feature that could do this?

[KarylFStein]

Originally Posted by KryptoNyte

I just watched a couple Youtube videos on the Edgerouter Lite - it looked really good for a bit, but it seems that the Tx and Rx traffic monitor doesn't discriminate between local traffic and Internet traffic. Because the computers on the LAN communicate with gigabytes of data locally, it seems that the traffic monitoring wouldn't give us any indication of a specific computer using all the Internet bandwidth. Is there a feature that could do this?

If you plug devices into a switch then plug the switch into the router, the router should only see traffic to/from the WAN, (as well as possibly multicast/broadcast traffic).