- For PC
- For MAC
- For Linux
- OS: Windows 7 SP1/8/10 (64 bit)
- Processor: Dual-Core 2.2 GHz
- Memory: 4GB
- Video Card: DirectX 10.1 level video card: AMD Radeon 77XX / NVIDIA GeForce GTX 660. The minimum supported resolution for the game is 720p.
- Network: Broadband Internet connection
- Hard Drive: 17 GB
- OS: Windows 10/11 (64 bit)
- Processor: Intel Core i5 or Ryzen 5 3600 and better
- Memory: 16 GB and more
- Video Card: DirectX 11 level video card or higher and drivers: Nvidia GeForce 1060 and higher, Radeon RX 570 and higher
- Network: Broadband Internet connection
- Hard Drive: 95 GB
- OS: Mac OS Big Sur 11.0 or newer
- Processor: Core i5, minimum 2.2GHz (Intel Xeon is not supported)
- Memory: 6 GB
- Video Card: Intel Iris Pro 5200 (Mac), or analog from AMD/Nvidia for Mac. Minimum supported resolution for the game is 720p with Metal support.
- Network: Broadband Internet connection
- Hard Drive: 17 GB
- OS: Mac OS Big Sur 11.0 or newer
- Processor: Core i7 (Intel Xeon is not supported)
- Memory: 8 GB
- Video Card: Radeon Vega II or higher with Metal support.
- Network: Broadband Internet connection
- Hard Drive: 95 GB
- OS: Most modern 64bit Linux distributions
- Processor: Dual-Core 2.4 GHz
- Memory: 4 GB
- Video Card: NVIDIA 660 with latest proprietary drivers (not older than 6 months) / similar AMD with latest proprietary drivers (not older than 6 months; the minimum supported resolution for the game is 720p) with Vulkan support.
- Network: Broadband Internet connection
- Hard Drive: 17 GB
- OS: Ubuntu 20.04 64bit
- Processor: Intel Core i7
- Memory: 16 GB
- Video Card: NVIDIA 1060 with latest proprietary drivers (not older than 6 months) / similar AMD (Radeon RX 570) with latest proprietary drivers (not older than 6 months) with Vulkan support.
- Network: Broadband Internet connection
- Hard Drive: 95 GB
Like other client/server products, our game has a very complex and distributed infrastructure, consisting of a variety of different servers. These are authorization servers, servers to store user profiles, battle servers, a squad server and a voice communication server. All of these are not on a single device, but dozens! Plus, a matching server, consisting of many physical devices of entrance gateways, that are proxies to eliminate points of failure, and a server that actually creates battles from players in the queue.
Now, a little more detail on what happened this past weekend. One of the gateway services was operating for 375 days, and during a scheduled reboot, a misconfigured version was loaded that used only a single core (we discovered it only on Sunday) and incorrectly proxying IP-addresses. Initially, it seemed to us that the server was overloaded due to the newly introduced vehicles and a gaming nation, as well as with the attention of the community for the major update. We decided to increase capacity as soon as possible by moving to the most powerful and expensive devices in Amazon. However, this cannot be done instantly, and moving itself took time. But even when the move was completed, the issue with incorrect configuration and single-threading remained. The server understood that all the players entering via this service had the same IP, it could not find them quickly and began lagging. After approximately 9:00 PM GMT, the issue of incorrect proxying was localized and corrected soon afterwards and the battles began to match again. During the night, however, the number of users decreased, and even the fact that the proxies were working with a single core did not interfere with the operation.
On Sunday, with the growth of users and battles, the load increased again, and we finally located a second configuration error and during the afternoon we fixed the single-threaded proxying by smoothly removing the incorrectly configured machines and introducing the new ones so as to avoid service denial for users already playing. This process also took some time. It should be noted, there was plenty capacity power available, even before the introduction of the more powerful servers - the power of a single core wasn’t enough, but there were many of them After the upgrade we utilized only about 6% of the overall capacity (i.e. 20 times reserve in power from the peak).
Conclusions
Based on the results, we made conclusions and planned the improvements of both operation and matching code. First, we plan to check the fault tolerance of the service by restarting all servers with over lengthy uptime, using the experience of large streaming services, such as Netflix, that uses special bots that check the operating time
Also, we have already introduced improvements with the matching code, that will allow us to keep an acceptable level of game operation even under heavy loads - queueing may take longer, but the service won't stop responding to players - staying operational.
In addition, an increase in the number of gaming nations and possible game configurations in battles has reached large values and algorithmic optimizations are required. Discovering all the possible combinations of all players and all gaming nations is the quadratic complexity of the algorithm, so it is necessary to create optimizations to find out perhaps not an absolutely perfect possible match, but at least good enough.
More good news. Login to the game from November 4th (11:00 GMT) until November 8th (11:00 GMT) to receive a Premium booster, +30% RP for 5 battles!
Comments (96)
Wow thanks for letting us now what happened.
And the hamsters? What happened to the hamsters? Did they get any food?
They get special enriched granules for hamsters!
At last we have a proper information about the servers issue as customers. Good work and please continue with this approach. This is the best way to start the cultivation of the community and establish a good working relation with a player/customer base.
transparency is good, keep it up!
Thanks for the transparency, anything you guys can do to help those who lost the daily tasks?
Something is still wrong tho, i just got put in a battle after leaving queue, being put back in, leaving again and switching to a different nation o.O
Thanks for the explanation but keep looking because the issues of this weekend are still here. As you said with high users server having difficulty with less users difficulty but not as bad. Well as of this morning that is still happening. Users are over 80 k and keep getting server is busy, have not been able to go into a game to re-experience sub 10 fps but trying to get into one it is at 20.
great, i had been wondering wtf was wrong with my connection the past few days
I am still getting constant "server busy try again later" messages - it also takes a lot of time with timeouts to get the box which shows the number of players waiting for battel
after each update - comes the problems such as bug... . Do only 1 MEGA update a year. Will be much time saved. And a lot new things after 1 year. :)
Submit a complaint