Antares & Laravel Benchmark

Antares Benchmark - Laravel

The sole fact of having an application that is perfectly efficient stimulates the discovery of new technological horizons, supports the growth of business and instills confidence. However, a large number of solutions that foster the development of modular applications lack the documentation about the operational efficiency. It usually stems either from the lack of time or resources, or from a deliberate omission of the subject due to it being perceived as not important enough or needed. The material presented in this article is an attempt to fill in the gap in relevant information by analyzing the productivity results of Antares in various load configurations and different input conditions.

The key objective of this text is to present Antares as a platform operating in the modular and scalable architecture. The use of this very architecture enables the addition of ever new modules – extensions that will broaden the software functionality without a substantial drop in its efficiency. The article also makes a comparison of productivity and complexity of the Antares source code and the code of Laravel 5.4.

The benchmark was made on the machine with the following parameters:

OS:

Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

CPU:

4 x Intel Core Processor (Haswell, no TSX), 3092.786MHz

RAM:

8GB DIMM RAM

Disk:

Size  Used Avail Use%

97G   11G   87G  11%

NET:

Download: 1922.01 Mbit/s

Upload: 1314.92 Mbit/s

 

Intro

Every programmer should not forget about the quality and performance of their code. It is often difficult to determine whether the code we create is efficient.

Performance tests may have impact on applications that are not optimized – for example, in terms of database queries. However, it may also happen that one instance of the application runs on several domains – then the load is significantly higher. Add to that the lack of optimization on the code side, and it turns out there is no way to test the performance without the right tool.

While searching for the right solution for testing, we decided to use Siege – http/https regression testing and benchmarking utility.

It has been designed to let web developers measure the performance of their code under duress, to see how it will stand up to the load on the Internet.

 

Let’s start by conducting a performance test for the login page. Using the Siege tool, the command:

siege localhost/login -d10 -r10 -c100

will perform 1000 login page calls, 10 reps using 100 concurrent users with 10 ms delay.

 

Login page with 100 concurrent users:

Transactions 1000 hits
Availability 100.00 %
Elapsed time 78.90 secs
Data transferred 2.71 MB
Response time 0.48 secs
Transaction rate 12.67 trans/sec
Throughput 0.03 MB/sec
Concurrency 6.13
Successful transactions 1000
Failed transactions 0
Longest transaction 2.19
Shortest transaction 0.17

As we can see, the whole test lasted 79 seconds. The longest answer is 2.19s, the shortest 0.17s. Please note that the measurements may be subject to a server configuration error. The average response time is 5 seconds.

 

The next test we did consisted in sending 10 repetitions of 100 concurrent users to the admin panel home page (dashboard).

For the purpose of the test, a middleware was created that enables the automatic login and creates a sample administrative user session.

Let’s compare the login response time with a fresh Laravel installation:

 

Laravel:

Date & Time Trans Elap Time Data Trans Resp Time Trans Rate Throughput Concurrent OKAY Failed
2017-07-20 14:30:26 1 2.02 0 0.01 0.50 0.00 0.00 1 0
2017-07-20 14:30:37 10 9.02 0 0.02 1.11 0.00 0.02 10 0
2017-07-20 14:30:49 20 10.01 0 0.01 2.00 0.00 0.02 20 0
2017-07-20 14:31:09 30 10.02 0 0.01 2.99 0.00 0.03 30 0
2017-07-20 14:31:23 40 10.01 0 0.02 4.00 0.00 0.07 40 0
2017-07-20 14:31:37 50 10.02 0 0.01 4.99 0.00 0.06 50 0
2017-07-20 14:31:52 60 10.01 0 0.02 5.99 0.00 0.10 60 0
2017-07-20 14:32:05 70 10.03 0 0.01 6.98 0.00 0.08 70 0
2017-07-20 14:32:19 80 10.03 0 0.01 7.98 0.00 0.09 80 0
2017-07-20 14:32:32 90 10.04 0 0.02 8.96 0.00 0.19 90 0
2017-07-20 14:32:45 100 10.03 0 0.02 9.97 0.00 0.23 100 0

Antares welcome page:

Date & Time Trans Elap Time Data Trans Resp Time Trans Rate Throughput Concurrent OKAY Failed
2017-07-20 14:36:00 1 5.26 0 0.26 0.19 0.00 0.05 1 0
2017-07-20 14:47:14 10 8.22 0 0.22 1.22 0.00 0.27 10 0
2017-07-20 14:47:33 20 9.26 0 0.25 2.16 0.00 0.55 20 0
2017-07-20 14:47:47 30 10.23 0 0.26 2.93 0.00 0.77 30 0
2017-07-20 14:48:07 40 10.26 0 0.26 3.90 0.00 1.01 40 0
2017-07-20 14:48:24 50 10.30 0 0.27 4.85 0.00 1.29 50 0
2017-07-20 14:48:36 60 10.25 0 0.32 5.85 0.00 1.87 60 0
2017-07-20 14:48:49 70 10.29 0 0.33 6.80 0.00 2.25 70 0
2017-07-20 14:49:03 80 10.29 0 0.33 7.77 0.00 2.57 80 0
2017-07-20 14:49:17 90 10.24 0 0.37 8.79 0.00 3.27 90 0
2017-07-20 14:50:01 100 10.26 0 0.40 9.75 0.00 3.88 100 0

Let’s see these data on the chart:

Antares vs Laravel - Response Time Comparison

The difference between the charging time of Laravel and Antares is approximately 0.25s. The main disparity between Laravel and Antares is the number of loaded service providers. This aspect is presented on the following comparative table:

Antares: Laravel:
Illuminate\Cookie\CookieServiceProvider

Illuminate\Database\DatabaseServiceProvider

Illuminate\Encryption\EncryptionServiceProvider

Illuminate\Events\EventServiceProvider

Illuminate\Filesystem\FilesystemServiceProvider

Illuminate\Foundation\Providers\FormRequestServiceProvider

Illuminate\Foundation\Providers\FoundationServiceProvider

Illuminate\Log\LogServiceProvider

Illuminate\Pagination\PaginationServiceProvider

Illuminate\Session\SessionServiceProvider

Illuminate\Hashing\HashServiceProvider

Illuminate\Cache\CacheServiceProvider

 

App\Providers\AppServiceProvider

App\Providers\EventServiceProvider

App\Providers\RouteServiceProvider

Arcanedev\LogViewer\LogViewerServiceProvider

Arcanedev\LogViewer\Providers\RouteServiceProvider

Arcanedev\LogViewer\Providers\UtilitiesServiceProvider

Barryvdh\Cors\ServiceProvider

Baum\Providers\BaumServiceProvider

Dingo\Api\Provider\HttpServiceProvider

Dingo\Api\Provider\LaravelServiceProvider

Dingo\Api\Provider\RoutingServiceProvider

Iber\Generator\ModelGeneratorProvider

Intervention\Image\ImageServiceProvider

Laravolt\Avatar\ServiceProvider

Thomaswelton\LaravelGravatar\LaravelGravatarServiceProvider

TwigBridge\ServiceProvider

Tymon\JWTAuth\Providers\LaravelServiceProvider

Antares\Acl\AclServiceProvider

Antares\Area\AreaServiceProvider

Antares\Asset\AssetServiceProvider

Antares\Auth\AuthServiceProvider

Antares\Authorization\AuthorizationServiceProvider

Antares\Automation\AutomationServiceProvider

Antares\Brands\BrandsServiceProvider

Antares\Breadcrumb\ServiceProvider

Antares\Customfields\CustomFieldsServiceProvider

Antares\Datatables\DatatablesServiceProvider

Antares\Extension\ExtensionServiceProvider

Antares\Foundation\Providers\DependableActionsServiceProvider

Antares\Foundation\Providers\FoundationServiceProvider

Antares\Foundation\Providers\RouteServiceProvider

Antares\GeoIP\GeoIPServiceProvider

Antares\Html\HtmlServiceProvider

Antares\Installation\InstallerServiceProvider

Antares\Logger\LoggerServiceProvider

Antares\Logger\Providers\RouteServiceProvider

Antares\Logger\RequestLoggerServiceProvider

Antares\Memory\MemoryServiceProvider

Antares\Messages\MessagesServiceProvider

Antares\Notifications\NotificationsServiceProvider

Antares\Notifier\NotifierServiceProvider

Antares\Routing\RoutingServiceProvider

Antares\Tester\TesterServiceProvider

Antares\Translation\TranslationServiceProvider

Antares\Translations\TranslationServiceProvider

Antares\UI\UIComponents\UiComponentsServiceProvider

Antares\UI\UIServiceProvider

Antares\Users\UsersServiceProvider

Antares\View\ViewServiceProvider

Illuminate\Cookie\CookieServiceProvider

Illuminate\Database\DatabaseServiceProvider

Illuminate\Encryption\EncryptionServiceProvider

Illuminate\Events\EventServiceProvider

Illuminate\Filesystem\FilesystemServiceProvider

Illuminate\Foundation\Providers\FormRequestServiceProvider

Illuminate\Foundation\Providers\FoundationServiceProvider

Illuminate\Log\LogServiceProvider

Illuminate\Pagination\PaginationServiceProvider

Illuminate\Session\SessionServiceProvider

Illuminate\Notifications\NotificationServiceProvider

Illuminate\View\ViewServiceProvider

Illuminate\Auth\AuthServiceProvider

Illuminate\Routing\RoutingServiceProvider

App\Providers\AppServiceProvider

App\Providers\AuthServiceProvider

App\Providers\EventServiceProvider

App\Providers\RouteServiceProvide

The difference of the number of loaded service providers is illustrated on the following chart:

Antares vs Laravel - Service Providers Comparison

 

Testing dashboard

Antares - Dashboard Test

 

Dashboard page without cache and with widgets:

Transactions 1000 hits
Availability 100.00 %
Elapsed time 138.89 secs
Data transferred 5.70 MB
Response time 0.75 secs
Transaction rate 7.20 trans/sec
Throughput 0.04 MB/sec
Concurrency 5.37
Successful transactions 1000
Failed transactions 0
Longest transaction 3.99
Shortest transaction 0.28

1000 page dashboard calls take 138 seconds, the longest response time is 4 seconds, the shortest 0.28, 7 calls per second and the average response time is 0.75 seconds. Sounds good 🙂

Let’s try to test the dashboard with cache enabled:

 

Dashboard page with cache and widgets:

Transactions 1000 hits
Availability 100.00 %
Elapsed time 146.13 secs
Data transferred 5.69 MB
Response time 0.58 secs
Transaction rate 6.84 trans/sec
Throughput 0.04 MB/sec
Concurrency 3.95
Successful transactions 1000
Failed transactions 0
Longest transaction 3.14
Shortest transaction 0.26

As we can see, the shortest transaction takes 0.26s, the longest 3s, and the average response is 0.6s.

 

Comparison for the dashboard page – no widgets and no cache, concurrent users between 10-100:

Date & Time Trans Elap Time Data Trans Resp Time Trans Rate Throughput Concurrent OKAY Failed
2017-07-20 13:23:02 1 5.36 0 0.35 0.19 0.00 0.07 1 0
2017-07-20 13:23:25 10 20.39 0 0.45 0.49 0.00 0.22 10 0
2017-07-20 13:23:50 20 20.30 0 0.37 0.99 0.00 0.37 20 0
2017-07-20 13:24:15 30 19.37 0 0.41 1.55 0.00 0.64 30 0
2017-07-20 13:24:38 40 20.31 0 0.47 1.97 0.00 0.93 40 0
2017-07-20 13:25:01 50 20.43 0 0.44 2.45 0.00 1.06 50 0
2017-07-20 13:25:24 60 19.35 0 0.52 3.10 0.00 1.62 60 0
2017-07-20 13:25:53 70 20.55 0 0.45 3.41 0.00 1.55 70 0
2017-07-20 13:26:22 80 20.39 0 0.52 3.92 0.00 2.05 80 0
2017-07-20 13:26:47 90 20.70 0 0.84 4.35 0.00 3.66 90 0
2017-07-20 13:27:10 100 20.39 0 0.58 4.90 0.00 2.86 100 0

Antares - Concurrent Users And Response Time

 

Users

Antares - Users

 

Users page without cache (1000 users registered):

Transactions 100 hits
Availability 100.00 %
Elapsed time 20.51 secs
Data transferred 0.81 MB
Response time 0.58 secs
Transaction rate 4.88 trans/sec
Throughput 0.04 MB/sec
Concurrency 2.82
Successful transactions 100
Failed transactions 0
Longest transaction 1.14
Shortest transaction 0.34

100 users page calls take 20 seconds, the longest response time is 1 second, the shortest 0.34, 5 calls per second and the average response time is 0.58 second.

 

Users page with cache (1000 users registered):

Transactions 100 hits
Availability 100.00 %
Elapsed time 20.56 secs
Data transferred 0.81 MB
Response time 0.47 secs
Transaction rate 4.86 trans/sec
Throughput 0.04 MB/sec
Concurrency 2.31
Successful transactions 100
Failed transactions 0
Longest transaction 0.74
Shortest transaction 0.31

100 users page calls take 20 seconds, the longest response time is 0.7 second, the shortest 0.31, 5 calls per second and the average response time is 0.47 second. As we can see, the caching of datatable response of users results in 0.11s of boost, as pictured below:

Antares vs Laravel - Response Time Comparison (1000 Users)

 

Summary

The above texts are intended to present the response times in different sections of the application. It is worth noting that cache is of great importance as it allows us to obtain valuable milliseconds. Moreover, much depends on the server and network parameters. For instance, the system’s response time is exactly the same for 10 and 70 users simultaneously browsing a website.

 

Code complexity

Code complexity is a metric used to indicate the complexity of a source code. This is a quantitative measure of the number of linearly independent paths using the source code of the program. To find out how complex our code is, we need to use quite a cool utility called phpmetrics. PhpMetrics is a static analysis tool for PHP.

 

Antares
Antares - Demographical Repartitions Of Logical Lines Of Code By Class
Laravel
Laravel - Demographical Repartitions Of Logical Lines Of Code By Class

 

Antares Laravel
Antares - Code Complexity Laravel - Code Complexity

Each file is symbolized by a circle. Size of the circle represents the Cyclomatic complexity. Color of the circle represents the Maintainability Index. Large red circles will be probably hard to maintain.

Antares

LOC
Lines of code 83862
Logical lines of code 47767
Comment lines of code 36103
Average volume 307.32
Average comment weight 31.82
Average intelligent content 31.82
Logical lines of code by class 42
Logical lines of code by method 9
Object oriented programming
Classes 1127
Interface 219
Methods 5277
Methods by class 4.68
Lack of cohesion of methods 1.39
Average afferent coupling 1.87
Average efferent coupling 3.87
Average instability 0.66
Complexity
Average Cyclomatic complexity by class 2.93
Average Relative system complexity 117.28
Average Difficulty 3.72
Bugs
Average bugs by class 0.1
Average defects by class (Kan) 0.36
Violations
Critical 0
Error 68
Warning 104
Information 42

Laravel

LOC
Lines of code 68108
Logical lines of code 31577
Comment lines of code 36543
Average volume 354.57
Average comment weight 30.94
Average intelligent content 30.94
Logical lines of code by class 61
Logical lines of code by method 7
Object oriented programming
Classes 514
Interface 89
Methods 4618
Methods by class 8.98
Lack of cohesion of methods 1.62
Average afferent coupling 2.77
Average efferent coupling 4.15
Average instability 0.51
Complexity
Average Cyclomatic complexity by class 3.22
Average Relative system complexity 165.86
Average Difficulty 5.35
Bugs
Average bugs by class 0.12
Average defects by class (Kan) 0.37
Violations
Critical 0
Error 40
Warning 56
Information 60

More details about the code complexity can be found here.

 

Summary

  • First, the comparison of the number of service providers between Antares and Laravel was taken into account. The ratio is 17/60, meaning that Antares uses over 3 times more service providers than a fresh installation of Laravel. This is reflected in the loading time of a website of the “welcome page” type which is approximately 0.02s for Laravel and 0.25s for Antares. Provided that one provider is usually assigned to a single module, the loading type of a clean website, when 30 modules are installed, should not exceed 0.35s. This is a very good result proving the scalable nature of software.

 

  • The next test showed the loading time of a website of the “dashboard” type accessible after the logging in to the admin panel. It covered the comparison of the loading time between a website with and without the enabled caching. The avarage loading time is 0.75s if the cache is disabled, while otherwise it is 0.58s.

 

  • Another aspect that was subject to testing is the loading type of the dashboard for 100 users in 10 incremental iterations. It should be noted that the website’s loading time is the same both for 10 and 70 users and it equals 0.45s. It is a very positive outcome as it proves that increasing the number of users does not overload the system.

 

  • The last test intended to present the operation of the application while displaying a website with the list of 1000 users and 100 simultaneous requests. The average loading time of the view with the disabled caching is 0.58s, and 0.47s when cache is enabled. The difference is 0.11s – it would probably increase in case of a bigger number of users.

 

Final Conclusion

Most of PHP developers are familiar with leading PHP frameworks. Framework is a foundation for applications and as such it should be consistent with the current experience you have in developing Internet applications. Particular frameworks usually serve as a basis for bigger platforms and systems that provide specific services or functionalities. Such a platform should have its own philosophy whereby the entire code is written and the relations between its subsystems and modules are set. Efficiency remains one of the most essential features of any framework and platforms built on it, especially if tens or tousands of users work on the application at once.

Antares uses Laravel as a foundation on which the entire structure is based. As a modular platform, it is comprised of multiple modules that form a whole, when combined. Each module has a different service or functionality to deliver. One of the essential aspects of every module is the code implementation, with emphasis being placed on retaining high efficiency and defining relations between modules without disturbances and duplications. Thanks to this, Antares outperforms Laravel with three times more loaded service providers, while the entire loading process is merely 0.2s longer. Such a slight difference in efficiency is practically invisible for end users, yet it translates into plenty more functionalities.

What are your thoughts on this project? Have your say in the comments section below!

One thought on “Antares & Laravel Benchmark

Leave a Reply

Your email address will not be published. Required fields are marked *

* Spambot verification