Similar to the sibling thread about Devau/Gatow I'll be using this one to keep you up-to-date about the patches I will apply to those two game worlds over the coming days. Please refrain from posting comments that are not related to the performance of the game worlds and/or problems you experience while I'm doing the upgrade and which might be related to them.
I'll be starting with Stapleton. After the patch, statistics will be missing except for the current week. The system will then start to re-sync the statistics one week at a time. This will most likely cause a slowdown both for Stapleton and for Aspern (if there can be such a thing, since both are slow as hell anyhow).
I just stopped the background job of Stapleton because the statistics transfer made hardly any progress. This will hopefully take some load off the database an accelerate the process.
I haven't had any lag on Aspern today which was a pleasant bonus considering yesterday everything was timing out. I was able to put in an order for some new planes today even. I'm gonna stay offline for a bit and take even a little stress off the server :P Every little bit counts!
Hang in there Martin, I know this has got to be pretty stressful!
Stapleton has managed to transfer a whopping 2 (two) week's data. I'm completely riddled by this and have absolutely no explanation why it is so slow. Either way, I can't keep the background job turned off much longer and I'll be away over the week-end, so later this morning I'll probably stop the transfer re-activate the background job so the game world can run "as normal minus statistics".
I observed a strange (for me) behavior on Aspern: the bank account statement is right now at 04:15 UTC (real time is 10:05) and it has this type of backlog since yesterday. In the mean while, it seams that all the expenses are deducted from the account as the financial schedule it looks to be on time (all weekly leasing payments are done). I do not know if it is related to the performance problem and all the updates but I hope it will help you. The general backlog for Aspern is now at 31 minutes.
Stupid AS!! Why you charge us money instead providing such lousy servers & services?? Can't bid for aircrafts, schedule flights & backlogs..please do take action!!
Stupid AS!! Why you charge us money instead providing such lousy servers & services?? Can't bid for aircrafts, schedule flights & backlogs..please do take action!!
If you mean our Aspern, Stapleton, Devau or Gatow server? Well, you don't have to pay at the moment if I'm not wrong ...
One more think that I have observed today: "No transactions found within the last 24 hours." and a missing day (estimated) in the last week statistics. I can not say if the financial transactions have been made but it looks like all the leasing fees have been payed to date. Server: Aspern.
One more think that I have observed today: "No transactions found within the last 24 hours." and a missing day (estimated) in the last week statistics. I can not say if the financial transactions have been made but it looks like all the leasing fees have been payed to date. Server: Aspern.
Yes, statistics (and the bookings in bank account statement are just that) are written in a deferred manner to not slow down actually important tasks. For some reason Aspern seems to be behind, which is probably a consequence of the general performance issues. Generally, all these statistics will be written eventually.
Here's what I intend to try today: I will stop the background job for both Aspern and Stapleton, than apply the patch to Aspern to see how fast its statistics are transferred. This will cause quite some backlog, but I hope they will quickly pick that up after the trial. Should it turn our to take forever to transfer the stats like in the case of Stapleton, I will probably have to think of some other solution.
I have transferred statistics data for both game worlds back to calendar week 26. This way, "Facts and Figures" statistics for enterprises and alliances display a full set of data. Timeline statistics like your load graphs in route management will only show data for 5 weeks in total, though. For my reasoning behind this decision and possible future actions concerning the statistics transfer, please see the respective thread I will open on this board in a minute.
Both game worlds are now without backlog, but at least judging from my measurements, the performance is still unsatisfactory. Given that performance on Devau/Gatow is at least temporarily unsatisfactory as well, I'm forced to conclude that the measures taken have not achieved their intended goal, at least not 100% of it.
As mentioned before, this was one of the last options known to me that can be implemented without a considerable rewrite of the system and/or expensive external help which we can hardly afford at this point in time. We will evaluate our remaining options over the coming days and keep you posted. The last silver lining I can see is that only 4 game worlds seem to be currently affected, all of which are among the most active ones. So maybe, as a last resort, throwing more hardware (aka money) at the problem will solve it once more. But we're at the upper limit of hardware expenses we can afford anyhow, so I can't promise anything.
Would it be possible to test with one world to see if hardware improvement does the job?
I cannot speak for others, but If I was playing on the affected server and I had a big airline running, I would prefer to pay slightly more in credits to keep the world running on a dedicated hardware. I am sure you already checked this possible cause, but a bad piece of hardware (e.g. memory) could be causing havoc on the server. You did have your colo facility have run a thorough hardware diagnostics, right? (please don't take this a wrong way, I am just suggesting options ... I would hate to be a player on one of the affected servers)