Mandiant’s APT1 Domain/MD5 Intel and Security Onion with ELSA

Posted on February 20, 2013 by Brad

Last night’s post on using domain and md5 lists from Mandiant’s Intel Report APT1: Exposing One of China’s Cyber Espionage Units focused on Security Onion for Splunk. If you’re using ELSA with your Security Onion deployment you can use the bulk_query.pl tool to run ELSA queries against a text file.

Download the Digital Appendix and Indicators zip file (md5 hashes are provided by Mandiant) and extract the files. Copy “Appendix D (Digital) – FQDNs.txt” and “Appendix E (Digital) – MD5s.txt” to your Security Onion Server (ELSA web node) and rename them to apt1_fqdn.txt and apt1_md5.txt.

sudo perl /opt/elsa/contrib/bulk_query.pl -f /path/to/apt1_fqdn.txt -t > fqdn_hits.log

sudo perl /opt/elsa/contrib/bulk_query.pl -f /path/to/apt1_md5.txt -t > md5_hits.log

The commands above will generate a log file containing each query term searched and any matching results that were returned.

Mandiant’s APT1 Domain/MD5 Intel and Security Onion for Splunk

Posted on February 19, 2013 by Brad

Mandiant’s Intel Report APT1: Exposing One of China’s Cyber Espionage Units is loaded with indicators that organizations can use to identify whether they are or have been a victim of APT1. If you’re running the newest version of Security Onion, then you’ll definitely have data available to comb through for network, domain and md5 hash indicators. ELSA can help you with this process as can Security Onion for Splunk.

If you want a quick way to leverage the domain and md5 data in Security Onion for Splunk download the Digital Appendix and Indicators zip file (md5 hashes are provided by Mandiant) and extract the files. Make a copy of the files “Appendix D (Digital) – FQDNs.txt” and “Appendix E (Digital) – MD5s.txt” and rename them to apt1_fqdn.csv and apt1_md5.csv. Then you’ll need to edit the files so the first row contains the field header. For apt1_fqdn.csv add “domain” to the first row; for apt1_md5.csv add “md5″ so they should look like this:

Next, we need to upload the .csv files we edited so head to Security Onion for Splunk and click Manager > Lookups > Lookup Table files > New. The Destination app will be securityonion, then choose the files and specify the Destination filename to be the same as what we named the files (apt1_fqdn.csv and apt1_md5.csv).

When you’ve uploaded both files you’ll need to change the permissions if you want other users to be able to run the queries by setting the permissions to Read for “This app only (securityonion).” Now head back to the Security Onion app, click the Search menu and run the following searches over all the Bro historical data you’ve got (dns.log and http.log specifically).

sourcetype=bro_dns [|inputlookup apt1_fqdn.csv | fields + domain] | fields dest_ip src_ip domain

sourcetype=bro_http [|inputlookup apt1_md5.csv | fields + md5] | fields dest_ip src_ip md5

If you get no matching events back, breathe a sigh of relief. If you do get results, start digging deeper!

A Dress for ELSA – Web Activity Dashboard

Posted on January 27, 2013 by Brad

The most impressive new addition to Security Onion 12.04 is Enterprise Log Search & Archive (ELSA). ELSA’s creator, Martin Holste (Twitter @mcholste), liked Splunk but had concerns about speed, scalability and cost, so he set out to develop his own log collection, indexing and searching platform and succeeded. Thanks to the efforts of Scott Runnels (Twitter @srunnels) and Doug Burks (Twitter @dougburks), ELSA can be enabled with the click of a button when deploying Security Onion.

ELSA makes it pretty easy to build and share dashboards using Google Visualizations. For details on building dashboards in ELSA see Martin’s post at his Open-Source Security Tools blog. If you want one to play with, I put together an overview of HTTP activity that demonstrates some of the chart types available.

If you want to check it out in your Security Onion ELSA, click the ELSA menu then Dashboards and the “Create/import new dashboard.” Give it a title, an alias (“web_monitor” for example), specify who has access then paste the following in the “Paste here for import” box:

{
   "charts" : [
      {
         "y" : "1",
         "options" : {
            "width" : 500,
            "displayMode" : "markers",
            "hAxis" : {
               "viewWindowMode" : "pretty",
               "minValue" : null,
               "viewWindow" : {
                  "min" : null,
                  "max" : null
               },
               "maxValue" : null,
               "useFormatFromData" : true
            },
            "vAxes" : [
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "logScale" : false,
                  "useFormatFromData" : true
               },
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "logScale" : false,
                  "useFormatFromData" : true
               }
            ],
            "backgroundColor" : "#ffffff",
            "booleanRole" : "certainty",
            "colors" : [
               "#DC3912",
               "#EFE6DC",
               "#109618"
            ]
         },
         "queries" : [
            {
               "query" : "get post put head groupby:BRO_HTTP.dstip | geoip",
               "label" : "GeoIP Map"
            }
         ],
         "x" : "0",
         "type" : "GeoChart"
      },
      {
         "y" : "2",
         "options" : {
            "width" : "333.333333333333",
            "is3D" : true,
            "legend" : "right",
            "hAxis" : {
               "viewWindowMode" : "pretty",
               "minValue" : null,
               "viewWindow" : {
                  "min" : null,
                  "max" : null
               },
               "maxValue" : null,
               "useFormatFromData" : true
            },
            "vAxes" : [
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "useFormatFromData" : true
               },
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "useFormatFromData" : true
               }
            ],
            "booleanRole" : "certainty",
            "colors" : [
               "#3366CC",
               "#DC3912",
               "#FF9900",
               "#109618",
               "#990099",
               "#0099C6",
               "#DD4477",
               "#66AA00",
               "#B82E2E",
               "#316395",
               "#994499",
               "#22AA99",
               "#AAAA11",
               "#6633CC",
               "#E67300",
               "#8B0707",
               "#651067",
               "#329262",
               "#5574A6",
               "#3B3EAC",
               "#B77322",
               "#16D620",
               "#B91383",
               "#F4359E",
               "#9C5935",
               "#A9C413",
               "#2A778D",
               "#668D1C",
               "#BEA413",
               "#0C5922",
               "#743411"
            ],
            "pieHole" : 0,
            "title" : "Source IPs"
         },
         "queries" : [
            {
               "query" : "get post put head groupby:srcip",
               "label" : "Sources"
            }
         ],
         "x" : "0",
         "type" : "PieChart"
      },
      {
         "y" : "3",
         "options" : {
            "title" : null
         },
         "queries" : [
            {
               "query" : "get post put head groupby:minute",
               "label" : "get post put head"
            }
         ],
         "x" : "0",
         "type" : "ColumnChart"
      },
      {
         "y" : "4",
         "options" : {
            "width" : 500,
            "sortColumn" : null,
            "page" : "enable",
            "legend" : "right",
            "hAxis" : {
               "minValue" : null,
               "viewWindowMode" : "pretty",
               "maxValue" : null,
               "viewWindow" : {
                  "min" : null,
                  "max" : null
               },
               "useFormatFromData" : true
            },
            "vAxes" : [
               {
                  "minValue" : null,
                  "viewWindowMode" : "pretty",
                  "maxValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "title" : null,
                  "useFormatFromData" : true
               },
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "useFormatFromData" : true
               }
            ],
            "pageSize" : 20,
            "booleanRole" : "certainty",
            "showRowNumber" : false,
            "alternatingRowStyle" : true
         },
         "queries" : [
            {
               "query" : "get post put head groupby:BRO_HTTP.site",
               "label" : "Top Sites"
            }
         ],
         "x" : "0",
         "type" : "Table"
      },
      {
         "y" : "1",
         "options" : {
            "width" : 500,
            "legend" : "right",
            "hAxis" : {
               "viewWindowMode" : null,
               "minValue" : null,
               "viewWindow" : null,
               "maxValue" : null,
               "useFormatFromData" : true,
               "title" : "Destination Ports"
            },
            "vAxes" : [
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "useFormatFromData" : true
               },
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "useFormatFromData" : true
               }
            ],
            "booleanRole" : "certainty",
            "isStacked" : false,
            "title" : "Activity by Destination Port",
            "backgroundColor" : {
               "fill" : "#ffffff"
            },
            "animation" : {
               "duration" : 500
            }
         },
         "queries" : [
            {
               "query" : "get post put head groupby:dstport\n",
               "label" : "class=BRO_HTTP"
            }
         ],
         "x" : "1",
         "type" : "ColumnChart"
      },
      {
         "y" : "2",
         "options" : {
            "width" : "333.333333333333",
            "is3D" : true,
            "legend" : "right",
            "hAxis" : {
               "minValue" : null,
               "viewWindowMode" : "pretty",
               "maxValue" : null,
               "viewWindow" : {
                  "min" : null,
                  "max" : null
               },
               "useFormatFromData" : true
            },
            "vAxes" : [
               {
                  "minValue" : null,
                  "viewWindowMode" : "pretty",
                  "maxValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "title" : null,
                  "useFormatFromData" : true
               },
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "useFormatFromData" : true
               }
            ],
            "booleanRole" : "certainty",
            "colors" : [
               "#3366CC",
               "#DC3912",
               "#FF9900",
               "#109618",
               "#990099",
               "#0099C6",
               "#DD4477",
               "#66AA00",
               "#B82E2E",
               "#316395",
               "#994499",
               "#22AA99",
               "#AAAA11",
               "#6633CC",
               "#E67300",
               "#8B0707",
               "#651067",
               "#329262",
               "#5574A6",
               "#3B3EAC",
               "#B77322",
               "#16D620",
               "#B91383",
               "#F4359E",
               "#9C5935",
               "#A9C413",
               "#2A778D",
               "#668D1C",
               "#BEA413",
               "#0C5922",
               "#743411"
            ],
            "pieHole" : 0,
            "title" : "Destination IPs"
         },
         "queries" : [
            {
               "query" : "get post put head groupby:BRO_HTTP.dstip",
               "label" : "Destinations"
            }
         ],
         "x" : "1",
         "type" : "PieChart"
      },
      {
         "y" : "2",
         "options" : {
            "width" : "333.333333333333",
            "is3D" : false,
            "legend" : "right",
            "hAxis" : {
               "viewWindowMode" : "pretty",
               "minValue" : null,
               "viewWindow" : {
                  "min" : null,
                  "max" : null
               },
               "maxValue" : null,
               "useFormatFromData" : true
            },
            "vAxes" : [
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "useFormatFromData" : true
               },
               {
                  "viewWindowMode" : "pretty",
                  "minValue" : null,
                  "viewWindow" : {
                     "min" : null,
                     "max" : null
                  },
                  "maxValue" : null,
                  "useFormatFromData" : true
               }
            ],
            "booleanRole" : "certainty",
            "colors" : [
               "#3366CC",
               "#DC3912",
               "#FF9900",
               "#109618",
               "#990099",
               "#0099C6",
               "#DD4477",
               "#66AA00",
               "#B82E2E",
               "#316395",
               "#994499",
               "#22AA99",
               "#AAAA11",
               "#6633CC",
               "#E67300",
               "#8B0707",
               "#651067",
               "#329262",
               "#5574A6",
               "#3B3EAC",
               "#B77322",
               "#16D620",
               "#B91383",
               "#F4359E",
               "#9C5935",
               "#A9C413",
               "#2A778D",
               "#668D1C",
               "#BEA413",
               "#0C5922",
               "#743411"
            ],
            "pieHole" : "0.5",
            "title" : "Method"
         },
         "queries" : [
            {
               "query" : "get post put head groupby:method",
               "label" : "class=BRO_HTTP"
            }
         ],
         "x" : "2",
         "type" : "PieChart"
      }
   ],
   "auth_required" : "1",
   "title" : "Web Monitor",
   "alias" : "webmonitor"
}

Security Onion for Splunk 2.0 Released

Posted on January 3, 2013 by Brad

On New Year’s Day I released Security Onion for Splunk 2.0 and Security Onion Server/Sensor Add On 0.7 to support the new release of Security Onion 12.04. The new release includes updated Overview, IR Search and SOstat dashboards, and introduces a new dashboard for Bro IDS logs I’ve dubbed Bro(wser).

The new release requires Sideview Utils (available freely from Splunkbase). I also recommend performing a clean install if you are upgrading Security Onion for Splunk from 1.1.7 to 2.0. There were some dashboard files configured in the /local path that should’ve been in /default and might not be overwritten properly when you upgrade. To uninstall the app run:

sudo /opt/splunk/bin/splunk remove app securityonion

then install the app from Splunkbase.

So what’s new? Besides all the awesomesauce that is Security Onion 12.04 itself, I hope you find the upgrades in the Splunk app suitably useful and worthy.

Overview – The pie charts at the top provide summaries of Sguil, Bro Notice and OSSEC events. You can drill down on them, but they’re mainly there to tell you everything is working.
The tabs beneath the pie charts are where things get interesting and you’ll find a lot of data at your finger tips. The Sguil tab lets you view and drill down on Sguil events by Name:

or Classification:

Sorry, no capME integration yet but it’s on the list!

The Connection Byte Counts tab let’s you see Bro connection bytes by service, IP, port, country, protocol or connection description and you can toggle by originator, responder or both. The idea with a lot of these tabs is to give you a little visibility into trending normal on your network.

HTTP Files provides a summary of all the filenames detected in Bro’s http logs by extension and let’s you drill down to view the filenames and further details.

SSL:

The Software tab provides some more great visibility for trending your network and looking for the unexpected.

Top Level Domains:

Anomalous Domains tab borrows from Johannes Ullrich’s “A Poor Man’s DNS Anomaly Detection Script.” Every night at 1 a.m. a search will run exporting the top 9,999 domains Bro saw queried the previous day to a .csv file. The Anomalous Domains tab looks domains up against that .csv file and if there’s a match the domain is ignored. The only domains that should show up are domains that hadn’t been visited the previous day. For fun, you can also sort Anomalous Domain hits by originator/source IP if you want to keep an eye on more erratic and unpredictable hosts. When you first install the app you’ll get a .csv not found error when you view this tab until the first csv export runs.

The CIF tab performs external .csv lookups against CIF results similar to previous releases.

Bro(wser) is the newest addition and it is all Bro. With tabs for Notices, Connections, DNS, HTTP, SSL, SMTP, FTP, IRC, SSH and Software, you have comprehensive Bro visibility in a digestible structure from one dashboard.

Drill down on a selection and you’ll get 4 pie charts summarizing event data and a list of IPs involved.

Drilling down on the IP list will query all Bro log sources for uid’s matching the selected event(s) and you can drill again to see the raw events:

In the example above, you can see how the selected uid was a connection (bro_conn sourcetype) via ssh (bro_ssh) that also triggered a bro_notice alert (or two) followed by the raw Bro events.

The new IR Search you’ll find looks very similar to the Overview tabs and I’ve built a mini Bro(wser) summarizing all Bro activity for the searched IP..

Drilling on the Bro dest_ip list gets you the grouped uid results similar to Bro(wser):

From there you have access to the workflow menu. Following the Bro data, you get a breakdown of all activity detected for the specified IP during the time range designated, with the events grouped in buckets allowing you to adjust the the bucket size.

SOstat got a makeover as well, but I think I’ve passed my screenshot quota.

I mentioned the workflow menu briefly as there are several additions here. I’ve added VirusTotal MD5 and DShield.org lookups, in addition to the IR Search workflow.

I’m sure there’s more I’ve left off but that’s the bulk of the changes for 2.0. It’s a work in progress and I’ll be continuing to tune and tweak as it gets more production use and as always, feedback is welcome!

Special thanks to all of the Security Onion team for their efforts!

Enjoy the release and have a secure and happy new year!

Helping the Seekers – how to place “security onion”

Posted on August 20, 2012 by Brad

Monitoring activity on this site gives me a glimpse into what search terms are drawing people here and I’ve always found the results interesting. Search terms are hardly a cry for help, but they usually are a whisper in the dark and can highlight issues people are dealing with and particularly where they might be struggling to find answers.

So I bring you “Helping the Seekers.” As search terms bubble up the list in frequency, I’ll periodically spotlight a few and try to have something to help out the next person who comes along. Since the release of the Security Onion Splunk app, “how to place ‘security onion’” has bubbled towards the top. The fun part is going to be trying to cover all the possible intentions behind that search term.

The answer to the “how to place ‘security onion’” question, regardless of the size of your network, is first at your entry/exit points to the Internet (aka gateway or egress points) just inside your firewall or Internet router. Security Onion can then monitor all traffic coming into or out of your network. If you can get eyes on that traffic, you’ll quickly be able to assess the state of your network and be in position to respond if an incident did/does occur. Fronting critical servers is also highly recommended, such as authentication servers, DNS, SQL, Microsoft SCCM/SMS and anywhere else critical data rests or traverses.

Why wouldn’t you want to put it outside of your firewall? Well, you could, but you lose visibility of your internal hosts as the sensor will not see any of your private IP addresses, only the public IPs from the gateway router/firewall. The Internet is a very noisy place and by placing the sensor just inside the firewall, the sensor will only monitor traffic that is specific to and allowed by your network and firewall.

(This is where I get on my hobby horse about running Security Onion. Whether you know what you’re doing or not, it’s pretty easy and affordable to setup and maintain. Just pet Sguil and Snorby every now and again and most smaller deployments will hum once tuned a bit. If you EVER need to call in professional assistance responding to an incident, you will thank me for the bill because it will save you a fortune in money and time.)

That answers the “how” question in terms of location (or maybe that would’ve been for a “where” question?). Regardless, now you know where to put it, what do you do? For this example we’ll use a home or small office environment, but the difference between small and large is typically capacity and the ability to handle the load. The same concepts of getting traffic to the Security Onion host apply.

A basic home or small office setup will look something like this:

Your Internet connection enters through a cable or DSL type modem and typically a gateway router device, like a Netgear or Cisco/Linksys, which we’ll call the perimeter. The modem and router can potentially be the same device and if there’s a network firewall at play it will be here too. In the setup above, the gateway router is typically the DHCP server. We need to see those private addresses so we can identify which hosts are generating events. So we need a way to get between the endpoints and that router.

One easy and affordable way to accomplish this is with a Mikrotik Routerboard 250GS. They can be had for around $40 and will give you a fully manageable 5 port gigabit switch (example 1).

In this case, we’d want to drop the Mikrotik between the gateway router that is issuing IP addresses to the internal devices. The only connections to our gateway router will be the WAN connection going to your modem and a single LAN connection to the Mikrotik. The gateway router will continue to do what it’s been doing with the only difference being all network activity to or from it will pass through the Mikrotik first. If you want to add wireless to the above setup, you’d want to use a separate wireless access point device plugged into the Mikrotik.

Your Security Onion host will ideally have two network interfaces if you plan on managing/monitoring it from another host on the network. One interface is your management interface which gets an IP address and can be connected to remotely. The other interface is your sensor interface where you want to see mirrored traffic; this is the data that Security Onion will monitor and analyze.

We used to have hubs, which were dumb but great at the same time. They basically would pass all traffic across all ports connected to the hub. Just plug in your Security Onion sensor and it would automagically see the data. Hubs have given way to switched networking now, which is why we need one we can manage (like the Mikrotik). Most retail store type switches don’t allow for things like configuring port forwarding.

So this is what your Mikrotik switch setup might look like:

Plug your gateway router LAN port into Port 1 and using RouterOS configure it to span all traffic to Port 2, where you’ll plug your Security Onion sensor interface. Ports 3-5 are free for endpoints. You can connect another switch or a WiFi AP to extend access to additional devices if you need more port capacity. The key is that all traffic into and out of your network at this point will be going through Port 1 on the Mikrotik which will be mirrored to Port 2 where Security Onion is listening.

I’ll save the Mikrotik step by step guide to configuring it’s RouterOS for another day.

Mad props to @Diagramly (http://www.diagram.ly/) for their awesome diagramming tool!

IDS Rule Reference for Splunk 1.0

Posted on August 13, 2012 by Brad

I created a standalone version of IDS Rule Reference for Splunk for Snort/PulledPork users who are not running Security Onion. I’ve added a few dashboard views to provide a little more flexibility for searching or researching rule documents.

The initial IDS Rules view is what is included in the Security Onion for Splunk app. It can be used for researching rules and activity by filtering on enabled rules or by category, classtype or source file.

The first of the new additions is a more flexible search dashboard allowing for wildcard searches by rule name or sid. Simply enter keywords or some (or all) of a sid and you’re off and running.

Lastly, the IDS Rule Tome allows you to browse only the rules that have a matching rule reference document. Undocumented rules will not appear in these results.

Known issues: I’ve not sorted out how best to handle the small subset of rule documents that share a sid, so rule documents named genid-sid.txt will provide inconsistent results.

If you’re using IDS Rule Reference in Security Onion for Splunk and want the additional views, simply download this app and you’ll be good to go.

Setup is pretty simple if you’ve got Splunk rolling already.

Install:
Download the Snort Rule Documentation (opensource.gz) from http://www.snort.org/snort-rules then extract opensource.gz to the monitored path:

tar zxvf opensource.gz -C /opt/splunk/etc/apps/ids_ref/local/rules

Copy your Snort PulledPork *.rules files to the monitored path:

cp *.rules /opt/splunk/etc/apps/ids_ref/local/rules/

Restart Splunk.

Event Workflows:
You can modify the Event Workflows from Splunk Manager > Fields > Workflow actions. Edit the IDS Rule Reference “Apply only to the following fields” to apply the workflow link to your Snort sig_id field in Splunk). You’ll also want to edit the Search String variable field name ($sig_id$ is the default).

As always feedback and suggestions are welcome for improvements!

Securing Splunk Free Version When Installed On Security Onion Server (or anywhere else)

Posted on July 10, 2012 by Brad

This stroke of genius comes directly from the man behind Security Onion, @dougburks, and solves two problems, one serious the other functional. Splunk’s free version allows you to index up to 500 mb/day, but does limit some (even basic) capabilities, most important of which is disabling authentication. If you’re running Splunk free version on your Security Onion server and access the server remotely (from another workstation), I highly suggest you make this your standard access process. The instructions below work on Ubuntu distributions and if you followed Doug’s advice about using a Security Onion VM as your client, this should work perfectly as long as you haven’t configured the VM as a server.

The method can be used on a Windows or Linux client. The instructions below focus on Linux, but googling “windows ssh tunnel how to” should get you a good start. In the example below port 81 is the Splunk port. If you installed Splunk on a different port just replace 81 with it.

The approach uses an SSH tunnel and is really easy to setup. On your Security Onion/Splunk server you’ll want to make sure SSH is enabled in Uncomplicated Firewall (ufw).

sudo ufw status

You should see 22/tcp ALLOW in the results. If it says DENY, then enable it:

sudo ufw allow 22/tcp

Next configure ufw to block (yes I said block) Snorby, Squert and Splunk ports:

sudo ufw deny 81/tcp
sudo ufw deny 443/tcp
sudo ufw deny 3000/tcp

From a remote Linux host with openssh-client installed:

sudo ssh username@securityonion.example.com -L 81:localhost:81 -L
443:localhost:443 -L 3000:localhost:3000

Replace username with the Security Onion/Splunk server user and securityonion.example.com with the hostname or IP address of your Security Onion/Splunk server. This command essentially tells your client to pass anything destined to localhost ports 81, 443 or 3000 to your SO server on it’s localhost port 81, 443 or 3000 via the SSH tunnel. The command requires sudo due to accessing privileged ports, so you’ll be prompted for your local password then again for the remote SO server user’s password. After authentication, you’ll have an active SSH terminal session to the server.

Launch a web browser and browse to any of the following:

http://localhost:81 – Splunk
https://localhost:443 – SO Home/Squert
https://localhost:3000 – Snorby

It’s that simple.

If you recall I mentioned a “functional” advantage of using this approach. In the Security Onion for Splunk app, I provide links to Snorby and Squert, but unfortunately, the user must configure the urls to fit their environment if they access the tools remotely. The default config uses “localhost” as the server, so if you’re following, if you use the above method to access Splunk securely, the Snorby and Squert links work out of the box. =)

Thanks and hat tip to Doug for this little gem! I had to bite my lip whenever I recommended someone install the free version of Splunk due to the authentication limit, but now I don’t have to.

Announcing: Security Onion for Splunk Server/Sensor Add-on

Posted on July 8, 2012 by Brad

I wanted to do a blog post on deploying the Security Onion for Splunk app in a distributed environment, where Splunk, Security Onion server and Security Onion sensor were all on separate hosts. I found it was easier to just build an add-on and let the README do the blogging. The add-on shouldn’t change or require updating nearly as much as the full app, only requiring updates when new logging is added at the server or sensor, as Bro is want to do at times (thank you very much). All the field extractions and transformations happen on the Splunk server. You can download the add-on here (once approved).

README

Overview:
Security Onion Sensor Add On eases the configuration of a multiple Security Onion sensor deployment. Install the Splunk Universal forwarder and untar this app to /opt/splunkforwarder/etc/apps. Edit /opt/splunkforwarder/etc/apps/securityonion_addon/local/inputs.conf to disable specific logs depending on whether you’re indexing from a
server or sensor that is remote to the Splunk indexer.

Installation:

Install Splunk Universalforwarder:

sudo dpkg -i <filename>

Start Splunk and accept the license

splunk start
splunk enable boot-start

Configure the universal forwarder to forward to a specific indexer:

splunk add forward-server <host>:<port> -auth <username>:<password>

Default receiver port is 9997. Username is a Splunk administrative user. Optionally, to enable secure Splunk communications the following command can be used to specify a certificate, root CA and password.

splunk add forward-server <host>:<port> -ssl-cert-path /path/ssl.crt -ssl-root-ca-path/path/ca.crt -ssl-password <password>

Download the Security Onion for Splunk Addon and extract the file to the Splunk forwarder apps folder:

sudo tar -C /opt/splunkforwarder/etc/apps -xvf securityonion_addon.tar.gz

Edit the local inputs.conf file depending on your deployment scenariot:

cd /opt/splunkforwarder/etc/apps/securityonion_addon/local/

Default config files for the following deployments are included:

inputs_server.conf
inputs_sensor.conf
inputs_server_and_sensor.conf

Just copy the appropriate file to replace the default inputs.conf (default deployment is server/sensor). For example, if you are installing on a sensor:

sudo cp inputs_sensor.conf inputs.conf

When you’re done, restart the Splunk forwarder:

sudo /opt/splunkforwarder/bin/splunk restart

As long as your indexer is receiving events you should be good to go.

Querying CIF Data From Splunk

Posted on April 26, 2012 by Brad

Collective Intelligence Framework (CIF) is a feed parser that brings a vast wealth of collective threat intel to your fingertips. (Update: Kyle Maxwell has posted a great introduction to CIF at http://overhack.wordpress.com/2012/05/07/introduction-to-the-collective-intelligence-framework/.) There’s a client app (perl or python, which is being rewritten presently) and browser plug-ins for Firefox and Chrome to make running queries simple. If you have access to a CIF server, it’s very easy to incorporate CIF queries into Splunk’s event and field menus. For Security Onion app users, this feature is coming, but you’ll need to edit the workflow to configure your CIF server and api key. Once configured you’ll see the item added to the event menu:

and/or the field menu:

The search results will open in a new window, and if there were any matches you’ll see something like this:

To add a workflow menu item in Splunk, go to Manager > Fields > Workflow actions and reference the screenshot below. If you’re a Security Onion user, you’ll need to enter the IP or hostname of your CIF server and a valid API key.

In the Security Onion app for Splunk the dest_ip and domain fields can be queried via the fields menu. If you prefer to edit/create via terminal:

CIF provides an enormous amount of intelligence for very little time, money or effort on the end users part and it’s future is looking very bright due to it’s flexibility in parsing content and the ease of interacting with data from other applications like Splunk. Once the python client is released I’ll look at building more correlation with CIF directly into the Security Onion dashboards.

Clap…Be Amazed…Now Go Defend

Posted on March 7, 2012 by Brad

What does it tell you when SC Magazine’s Best Security Company of the Year bases it’s business on helping organizations recover from all the failures of the layers of prevention and mitigation on which we focus so much of our time and money? The typical IT security environment today is focused on two things: prevention and vulnerability management. Problem is you can’t prevent what you don’t know is coming and with most preventative solutions in place today your best hope is that you’re not the first (or earliest) to see an attack and that whoever is can and will share that intel. Risk worth assuming? Are we better served with the Sisyphean task of continuous patch management than we would be were we to focus most of those resources (people and money) on early detection and response?

Whether you agree with these types of awards or not, Mandiant deserves the recognition. They’re a great company and group of people and are model givers to the Info Sec community which helps people like me do my job better. They’ve assembled some of the best talent in the business as well. All deserving reasons.

I’ll give you one more.

For all the money we spend in Info Sec trying to prevent attacks, and a lot is spent, what happens when an attack gets through? We know prevention is going to fail. Yes. Prevention WILL fail and regularly does. If you don’t have any level of network monitoring in place then you’re doing it wrong. And there’s no excuse for it. In fact, there’s no excuse for not leveraging the latest and greatest technologies available to mitigate attacks when the financial cost is moot and your security posture could be strengthened immensely. For larger organizations who already have commercial solutions in place, are you able to afford the extent of coverage required for maximum visibility? What if you could get that coverage without the lofty hardware and support costs?

How do you save money on network security monitoring? You open your eyes to the low cost potential of open source initiatives and brilliant minds eager to share, like a few of those that Mandiant has shepherded through the Info Sec world. Doug Burks (@DougBurks), of Mandiant, has provided one such solution with Security Onion, a network monitoring Linux distribution that is only limited by the hardware you have at your disposal on which to run it. And what’s even better? He’s made it so easy to setup and configure that even complete strangers to Linux can figure it out. If you know how to boot off of a DVD you’re in business. Need a deployed infrastructure? Setup one Security Onion host as a server and deploy others as sensors. Standalone works just as well. Heck, I run it in a virtual machine on my laptop and it’s the best host-based IPS money can buy…but doesn’t need to. I’m not kidding about how easy this is to deploy and if you take 20-30 minutes and watch one of Doug’s presentations (links available on the Security Onion site) demonstrating just how easy it is, you’ll thank me. Then you’ll show him thanks by downloading it.

When you look at the arsenal of monitoring tools that Security Onion brings to the table you’ll be even more amazed as you peel back the layers and get a handle on just how powerful the solution is.

You get Snort, the great open source intrusion detection/prevention system from Sourcefire (another incredible company; see my Razorback post then go download the new version). Maintaining Snort with Security Onion via PulledPork for signature management is a breeze and easily supports Snort (get thee an Oinkcode! ) and Emerging Threats signatures.

You get full packet capturing with Sourcefire’s Open Source Daemonlogger.

You get OSSEC, a host-based intrusion detection system, which helps you monitor your network security deployment out of the box and the capability to extend that detection to Windows, Linux, and MacOS via the OSSEC agents.

You get Bro IDS, an alternative approach to IDS that I’m just now learning myself and have been continually blown away by the visibility it provides. I hope to cover it more specifically in a future post, as educational resources for using Bro are a little scarce. It’s described as a network analysis framework and it comes preloaded with a few obvious examples that will give you an idea of what the framework can do. It collects basic connection data for every connection, http, ftp, syslog and SMTP data, SSL certificates (both known and the suspicious unknowns), SQL injection attacks, and more. It will continue to amaze you when when the framework reveals its capabilities with events like HTTP::Malware_Hash_Registry_Match, indicating a file has been downloaded, the file hashed and the hash matches a known malicious software hash in the Team Cymru Malware Hash Registry.

All that and more, packaged up nicely saving you all of the headaches of deploying a comprehensive network monitoring suite of Open Source solutions from scratch.

Once you start collecting data you’ll have the powers of Squil, Squert, and Snorby at your disposal for monitoring and analysis. If you prefer, you can also grab a copy of Splunk.which installs fairly painlessly on Security Onion. (Only issue I had was the default Splunk port was in use; I typically use port 81 without problems.)

Take a day. Install a Security Onion VM. Run some sample pcaps through it. See how easy it is to deploy and detect the activity. Now think how much better you might be able to defend with this kind of visibility at next to no cost (how many old servers are being retired and replaced with virtual machines? Reuse them!). Then with all the money you’ll be saving go hire some gifted local talent with dedication and passion to learning and an accepted understanding that they’ll always be a day behind. Do this and you’ve improved your overall security posture immensely and put yourself in a much better position to detect and respond to an incident. Even if you aren’t doing the responding! If the FBI shows up at your door, you’re going to need help. The data you would be collecting would be invaluable to resolving an incident efficiently and effectively.

Now do you start to see why I think Mandiant deserves Best Security Company recognition? And this is just one example. Have you looked at the free tools they offer? Have you ever heard of TaoSecurity Blog? Dustin Webber, the author of Snorby (although he’s with Tenable now I believe)? Their efforts with OpenIOC.org? Jamie Butler, who literally co-authered the book on rootkits? The latest is Michael Sikorski’s contributions with his new book Practical Malware Analysis accompanied by the release of FakeNet, a malware network analysis tool. The list goes on. They’re a company of talented people, providing tools and sharing knowledge to build stronger and more capable communities to build a safer Internet.

Any company like that deserves the title Best Security Company, especially when compared to companies who profit from you for solutions that you buy with the expectation that they will fail you at some point. Mandiant profits when those companies fail you. And then they (Mandiant’s family) turn around and give, yes give as in free, you a way to do a lot if not more than what you’re paying good money for, or aren’t paying for at all because you can’t afford it. They’re doing something about enabling small/medium businesses and personal networks to adopt affordable security approaches. They’re providing security practitioners with tools and technologies to perform better, defend more wisely and “find evil” more efficiently with less technical skill than was required 2 years ago. They’re doing it in their 9-5 jobs. They’re doing it as hobbyists. They’re doing it as caring volunteer citizens.

They should be recognized and thanked…

And you should go download Security Onion and start protecting your personal and professional assets…like now.