-
created by Markus Selinger23rd April 2015
Endurance Test: Does antivirus software slowdown PCs?Critics
maintain that protection software for Windows really puts the
brakes on PCs. In a 14-month, extremelycomprehensive performance
endurance test, AV-TEST examined the trade-off of performance
versus protection, andcame up with some conclusive answers.
For the reader of an endurance test, the provided rating for
speed or performance of a product is only expressed in one
singleaggregate number. For the tester in the laboratory, it's a
long journey to arrive at this number. To get there, he had
toaccompany a solution through 7 test rounds over 14 months. Of the
19,000 individual ratings per product, 35 test area findingswere
aggregated. It requires this time and effort to arrive at the
rating for a single product. In the current endurance test
fromJanuary 2014 to the end of February 2015, a total of 23
products were tested for speed in the labs at AV-TEST. In this, the
testsalternated in the use of the operating systems Windows XP, 7
and 8.1.
23 products in the Performance TestIncluded in the test were
products from Ahnlab, Avast, AVG (freeware and purchase product),
Avira, Bitdefender, BullGuard,Comodo, ESET, F-Secure, G Data,
Kaspersky, McAfee, Microworld, Norman, Norton, Panda, Qihoo 360,
Quick Heal, Tencent,Threat Track and Trend Micro. In addition, a
Windows system with the freeware Microsoft Security Essentials or
Defender (forWindows 8/8.1) was put through all the tests as an
additional comparative figure. The reference system without any
protectionsoftware was used for later comparison of the measured
performance ratings. A more in-depth explanation on the topic
ofreference systems can be found later in the "Test Configuration"
section.
For all products in the tests, it was measured how long they
required for various test sets, in order to
- download files from the Internet;- launch websites;- install
applications;- open applications, including a file;- copy
files.
For all subsections, load points of 1 to 5 were assigned after
each test, whereby a score of 1 represents good and a score of
5poor. If an antivirus application slowed down a system between 0
and 20 percent, it received 1 load point. For 21 to 40 percentthere
were 2 points, for 41 to 60 percent drop in performance, 3 points,
and so on up to 5 points.
Thus for all 5 test sections, this resulted in a perfect score
of 5 and a worst score of 25 load points. All partial results of
thetests were aggregated and at the end divided by the number of 7
test rounds.
The Test ResultThe lowest influence and thus the best
speed-performance rating was achieved by the product from Kaspersky
with only 5.1 loadpoints. As previously mentioned, the maximum
perfect score was 5.0. Following close behind are the security
packages fromBitdefender and Qihoo 360 with 5.3 and 5.7 points
respectively.
The additional 6 products from McAfee, Bullguard, Trend Micro,
Norton, Avira and F-Secure also achieve good rankings at 6.1 to7.9
load points in their endurance test results. All other 14 products
are above that threshold. If the system is protected by theWindows
built-in Security Essentials or Microsoft Defender, the load is 8.7
points.
The solutions from Norman, Quick Heal and Threat Track already
slow down a PC significantly with 12, 12.4 and 13.9 loadpoints. No
product attained the worst score of 25 slow-down points.
-
In order to more clearly understand the figures, here is an
example with two individual scores: Every user quickly
becomesimpatient, especially when copying files under Windows. In
the test, the reference system without any protection requires
justover 141 seconds to copy the test set from AV-TEST containing
3.3 GB. A Windows system with an installed Kasperskysecurity
application examines the files and requires 165 seconds for
copying. The application finishing last, Threat Track,requires over
300 seconds on average for the same procedure.
The individual test sectionsAV-TEST refers to the 5 test
sections as the "Real-World Test", as all the tested operations are
typically performed daily multipletimes by users.
1. Downloading files from the Internet
In order to ensure that all the products have exactly the same
line for downloading, a server is used as a download server in
aseparate test network. All the programs offered on the server are
available unmodified in the same version during the entire
testperiod. This guarantees the identical download scenario during
the endurance test. Most security packages scan the data or thedata
stream during the download and may thus slow down the procedure.
Nearly all the products handle this task so well,however, that the
scores fall below the 20 percent mark, thus 21 out of 23 packages
consistently receive only an average of oneload point. Only the
products from Avira and G Data are slightly above this level on
average and for that reason are penalizedwith somewhat more load
points.
2. Launching websites
For this test, a few dozen premium websites are launched, such
as Amazon, Yahoo, Apple or Google. The Internet Explorer ofthe test
operating system Windows XP, Windows 7 or Windows 8.1 is used for
this. The selected websites are always highlyavailable in the web
and therefore perfect for the comparison.
The test sounds simple, but it is tremendously revealing.
Because in final analysis, only three products achieve a low load
levelunder 20 percent and therefore only sustain 1 point:
Bitdefender, Kaspersky and McAfee. An additional 20 products break
throughthe barrier of 20 percent more load and require up to 40
percent. For this reason, those products also receive 2 load
points. Theonly anomaly here is the product Threat Track, which
slows down the website launch tremendously and therefore scores
4.6load points on average.
3. Installing applications
In the test, applications are installed per command line
(without clicks), and the time is clocked for this operation.
Included in thistest section are popular programs such as the Flash
Player or the Adobe Reader. The applications and their versions
arenaturally identical throughout the entire endurance test.
This test is also part of a user's typical daily procedure. The
findings show that many security programs delay the
installationprocedure to a certain extent. Bitdefender, Bullguard,
G Data and Trend Micro remain within normal bounds, thus only
scoring 1load point. An additional 9 solutions are between 1 and 2
points on average, 9 products score between 2 and 3 points,
andMicrosoft Security Essentials or Microsoft Defender even above 3
load points.
4. Opening applications, including a file
In this test, a DOC file, a pdf file and a presentation file all
having a large size are opened repeatedly and directly
withLibreOffice. The security packages monitor the access and the
opening of files. Some solutions thus delay the procedure. In
thisarea of the endurance test, the products from Bitdefender, ESET
and Kaspersky were hardly noticeable and thus received only 1load
point. An additional 16 products were above 1 point on average, 4
solutions scored even 2 or more load points.
5. Copying files
Especially when copying data under Windows, most security
solutions cause frustration among users. That is why the lab
teamexamined how heavily the products delayed the copying of files.
The test featured a 3.3 GB set with a wide variety of file
typessuch as films, images, graphics, documents, pdfs and programs.
Up to this test phase, most of the products were still
virtuallyneck and neck. Due to the findings in this test category,
however, many security packages fell behind the top finishers.
The test demonstrates that Qihoo 360 and Kaspersky delay the
copying procedure only ever so slightly. 8 products
sustainedbetween 1 and 2 points for their load. 5 security packages
were between 2 and 3 points on average, 3 products between 3 and
4points, and 5 products even attained an average above 4
points.
-
The most conspicuous products in the endurance test are from
Ahnlab, ESET, Norman, Quickheal and Threat Track. Theyconsiderably
slow down the copying procedure under routine conditions. This fact
is also reflected in the overall findings.
Test CongurationFor the test, several PCs with absolute
identical hardware were used in the laboratory: Intel Xeon X3360 @
2.83GHz, 4 GB RAMand a 500 GB hard drive. Because these computers
might perform differently, however, they were selected from a pool
of 60test PCs. All the PCs were tested multiple times with
unprotected windows XP, windows 7 and Windows 8.1 systems.
Thecomputers of identical speed were then defined as test systems.
Their speed ratings serve as a reference for the test. Beforeand
after each product test, it was evaluated whether all the computers
still delivered the same reference ratings.
Each product goes through a cycle of all the tests multiple
times. Afterwards, a status quo ante of the PC is re-established
withthe help of a disk image, including the installed application.
Then the test is performed again, the system is rebooted, and
thebattery of tests to be conducted is repeated again. The PC is
rebooted and tested several times, then the system is restoredwith
the help of the disk image, and the procedure starts all over
again. At the end, there are a minimum of 20 and a maximumof 40
individual scores for each individual small test step. For some 90
test steps per product, from 1,800 up to 3,600 individualscores are
recorded per product. In an endurance test lasting 7 test rounds,
an actual 12,600 up to 25,200 scores are analyzed;on average around
19,000 per product. In the test with 23 products, some 430,000 test
scores were recorded.
430,000 individual scores all add upThe analysis of 20 to 40
individual scores from each test step is performed according to a
special methodology. The individualscores are calculated in groups
according to the mean value and standard deviation. The group with
the lowest standarddeviation therefore has the lowest possible
error ratio. This is then used for additional calculation. The
procedure of analysis usedis standard statistics and eliminates the
skewing of results due to outliers.
The aggregate calculation is subsequently compared to the
performance rating of the unprotected reference system.
Does antivirus software slow down PCs?The result indicates that
good security software does not heavily slow down a Windows PC. A
few systems resources arerequired, which slows down the PC, just
like with any other software application launched. The products
from Kaspersky,Bitdefender and Qihoo 360 exhibited the lowest load
on the systems, although they provide good protection, as proven by
theprevious security function tests ( "The best antivirus software
for Windows Home User").
The "subjective" slowing down of a PC frequently reported by
users is actually a proven objective fact. This is
clearlydemonstrated by the products from Norman, Quickheal and
Threat Track. These security packages cause the copying routine
torun 2.5 to 3 times more slowly than with Kaspersky software, for
instance.
The AV-TEST team occasionally receives inquiries from users
arriving at totally different results in their own tests. This
mayindeed occur for special configurations and particular hardware
and software combinations. Those who are uncertain whether
thefastest software is right for them ought to try the following
test: First, create a system backup, then load the test software
andtry everything out yourself. If something doesn't quite perform
the way it should, then you simply revert to the backup.
Test Methodology and Critics
-
Maik Morgenstern, CTO AV-TEST GmbH
Typical objections by critics against the way results were
obtained are theinfluences by the hardware used, by the software
such as Windows or byaccess to the Internet.
The point of criticism concerning hardware is rendered moot
based on the above-described procedure of the test PC selection
under "Test Configuration". Thereference scores are re-validated
after each test on each PC.
With respect to Windows, background tasks could influence system
behavior.That is why the testers deactivated all the tasks that can
be shut down. In theevent that an outlier does occur in a test
step, this is statistically balanced outbased on 20- to 40-fold
repetition of the step.
The third point of criticism pertains to the Internet access in
the test when visitingwebsites. On the one hand, the speed may
depend on the requested server andthe routing, and on the other, ad
banners or images may modify the websites.The test is completed
with the help of a 500 megabit line, and it runs over severaldays
and at different times. This minimizes any possible negative
effects. Thefindings also demonstrate that the slowing-down effect
caused by securitysoftware does not occur sporadically, but is
rather constant for all websitestested.
Antivirus products in the "Performance" endurance test.
-
Performance endurance test: Some protection solutions cause
quite a heavy burden on a Windows PC for everyday routines.
-
Category "Copying under Windows": While the Kaspersky product
hardly slows down the system, Threat Track and Quick Heal are a
different story entirely.
Performance endurance test: The products from Kaspersky,
Bitdefender and Qihoo 360 cause the lowest load on a Windows
PC.
-
5 categories in the performance test: Most of the products in
the endurance test slow down the PC in the category of "copying
data".
Copyright 2015 by AV-TEST GmbH, Klewitzstr. 7, 39112 Magdeburg,
GermanyPhone +49 (0) 391 60754-60, Fax +49 (0) 391 60754-69,
www.av-test.org