The International Simutrans Forum

 

Author Topic: poor game flow in large maps  (Read 24059 times)

0 Members and 1 Guest are viewing this topic.

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
poor game flow in large maps
« on: September 01, 2009, 10:50:53 AM »
I've got a game going on a 1024x1024 map  with several large towns and 8 fast trains, (4 Northbound 4 Southbound) Multiple trams running in all full growth towns (15000 or over). The game has become extremely laggy with no seeming cause. It can happen at the start of a month, two days in or halfway. All not needed processes are shut down and the graphics card is running at 'Turbo'.

The same game played in Experimental suffers no lagginess at all.

Maybe a look should be taken at Knightlys 'New Path Search' to see if it can be incorporated into Standard? Or maybe a new type of search path could be written to suit Standard?

Offline Isaac.Eiland-Hall us

  • Benevolent Dictator
  • Administrator
  • *
  • Posts: 3649
  • PanamaCityPC.com/support/
    • Facebook Profile
  • Languages: EN
Re: poor game flow in large maps
« Reply #1 on: September 01, 2009, 11:08:30 AM »
I suspect this is not a pak64-specific issue...

Meanwhile, have you compared the performance of the GDI executable vs. SDL?

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
Re: poor game flow in large maps
« Reply #2 on: September 01, 2009, 12:10:10 PM »
I suspect this is not a pak64-specific issue...

Meanwhile, have you compared the performance of the GDI executable vs. SDL?

Yes I have, It doesn't make any noticable difference. I haven't tried it with PAK128 as I only have PAK128 Britain which is a EXP game. I would try PAK128 but it takes weeks to get to the stage where I have PAK64.  But you could be right it may be a Simutrans-Standard problem.

Offline gerw

  • Coder/patcher
  • *
  • Posts: 618
Re: poor game flow in large maps
« Reply #3 on: September 01, 2009, 12:23:01 PM »
Can you post your savegame?

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
Re: poor game flow in large maps
« Reply #4 on: September 01, 2009, 07:09:11 PM »
Can you post your savegame?
Ok posted to http://simutrans-germany.com/files As Colin's Standard 2611.sve

Offline whoami

  • Devotees (Inactive)
  • *
  • Posts: 693
Re: poor game flow in large maps
« Reply #5 on: September 01, 2009, 08:34:28 PM »
(Moving to Development...)

Direct link to the savegame is http://simutrans-germany.com/files/upload/Colin%27s%20Standard%202611.sve; (EDIT:) it is loadable with plain pak64/r175.
« Last Edit: September 02, 2009, 07:48:59 PM by whoami »

knightly

  • Guest
Re: poor game flow in large maps
« Reply #6 on: September 02, 2009, 08:57:08 AM »
I have just tried Colin's save game.

There was a brief period of reduced responsiveness immediately after loading, but soon afterwards the game ran smoothly. When I tried to add a halt or adjust certain convoy's schedule, responsiveness was reduced again shortly afterwards for quite some time.

So, probably the reduced responsiveness is caused by the re-routing of all existing ware packets in one single world step after set_schedule_counter() is triggered by actions like halt construction/deletion or schedule changes.

Offline gerw

  • Coder/patcher
  • *
  • Posts: 618
Re: poor game flow in large maps
« Reply #7 on: September 02, 2009, 09:04:10 AM »
There was a brief period of reduced responsiveness immediately after loading, but soon afterwards the game ran smoothly. When I tried to add a halt or adjust certain convoy's schedule, responsiveness was reduced again shortly afterwards for quite some time.

So, probably the reduced responsiveness is caused by the re-routing of all existing ware packets in one single world step after set_schedule_counter() is triggered by actions like halt construction/deletion or schedule changes.
I've noticed the same, while trying Colin's game. So it's the rerouting of goods, which makes the game laggy...

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
Re: poor game flow in large maps
« Reply #8 on: September 02, 2009, 09:06:30 AM »
I have just tried Colin's save game.

There was a brief period of reduced responsiveness immediately after loading, but soon afterwards the game ran smoothly. When I tried to add a halt or adjust certain convoy's schedule, responsiveness was reduced again shortly afterwards for quite some time.


Reduced by quite a lot and when building a game, because one is continually adding halts or new schedules, it happens with repeated monotony. Every few game days actually, and not counting Auto Save which I've extended to every 3 months. Brave little Aussie fellow that I am. ;D ;D ;D

knightly

  • Guest
Re: poor game flow in large maps
« Reply #9 on: September 02, 2009, 09:13:59 AM »
I've noticed the same, while trying Colin's game. So it's the rerouting of goods, which makes the game laggy...

Gerw, is it possible to spread the load of re-routing ware packets across a few steps?

Reduced by quite a lot and when building a game, because one is continually adding halts or new schedules, it happens with repeated monotony. Every few game days actually, and not counting Auto Save which I've extended to every 3 months. Brave little Aussie fellow that I am. ;D ;D ;D

Colin, I understand how poor the responsiveness would be when you continue to add/remove halts or adjust schedules, because I have a similar PC specifications as yours ;) The game almost freezes during such global re-routing.


Offline gerw

  • Coder/patcher
  • *
  • Posts: 618
Re: poor game flow in large maps
« Reply #10 on: September 02, 2009, 10:47:49 AM »
Gerw, is it possible to spread the load of re-routing ware packets across a few steps?
It's currently spread across 32 steps, see spieler_t::step. But maybe this number should depend on the time, the rerouting needs.

knightly

  • Guest
Re: poor game flow in large maps
« Reply #11 on: September 02, 2009, 11:34:41 AM »
It's currently spread across 32 steps, see spieler_t::step. But maybe this number should depend on the time, the rerouting needs.

Thanks a lot for pointing out. :) It is quite incredible that after spreading re-routing across 32 steps, the game is still so unresponsive.

Offline gerw

  • Coder/patcher
  • *
  • Posts: 618
Re: poor game flow in large maps
« Reply #12 on: September 02, 2009, 01:08:44 PM »
Thanks a lot for pointing out. :) It is quite incredible that after spreading re-routing across 32 steps, the game is still so unresponsive.
But one can think of a worst case, where all the 'expensive' stations (with many waiting goods) are rerouted in one step...

Offline z9999

  • Devotees (Inactive)
  • *
  • Posts: 848
Re: poor game flow in large maps
« Reply #13 on: September 02, 2009, 02:22:24 PM »
At least, there wasn't this problem in 99.17.1.
But 100.0 and after versions have this problem.

I heard many people saying 100.0 and after versions are too heavy.

Offline whoami

  • Devotees (Inactive)
  • *
  • Posts: 693
Re: poor game flow in large maps
« Reply #14 on: September 02, 2009, 07:40:40 PM »
My observations with Colin's savegame:
1) After loading of the savegame, and also after deployment of a new schedule or a new goods category for an existing schedule, ST becomes unresponsive for some seconds, with down to 2(!) fps and 100% cpu/core utilization. => ST seems to recalculate routing for all goods packets, not just for the goods categories served by the new schedule/convoy. This should be fixable by filtering before re-routing.
2) The previous observation especially applies to the change of routing in an isolated network, which seems to still lead to re-calculation of routing on the whole map. For isolated freight lines/networks, it may be useful to limit the refresh, e.g. by walking through the graph of stations *and* connected convoys, also filtering(!) by goods category (see above), marking the stations (also convoys due to their contents?) for refresh, and refreshing only the marked ones in the next loop (removing the marker here again).
1)+2) could be changed as described (or by other means), but a large, fully interconnected P+M network will still not benefit directly from such changes, only indirectly by optimized routing for certain other cases. Further spreading of re-routing calculation may help.
3) side-note: station coverage setting in the savegame is 3 (=> 49 tiles to check at P+M+G generation), which doesn't matter for re-routing, but the additional calculation effort for newly generated passengers+mail increases overall load.
« Last Edit: September 02, 2009, 10:16:29 PM by whoami »

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
Re: poor game flow in large maps
« Reply #15 on: September 03, 2009, 08:29:17 AM »
I have just read whoami's post and it seems to me that he has hit the nail on the head, as they say. If his suggestions can be implemented we should see a, not vast, but quite substantial in crease in game flow. (Incidentally my FPS went down to 1)

I could set the station coverage back to 2 but surely that would lead to an increase in Trams/Tram lines/Stations, creating even more route finding?

I have question re this particular game.

modnote by whoami: this part is now discussed in http://forum.simutrans.com/index.php?topic=3203.msg31352#msg31352:
 I have a passenger train (Not shown in the uploaded game)  Which travels between two stations only. It leaves one station with over 3000 pasengers and arrives at it's destination where they all disembark. It makes a loss of over 41 thousand (not in travel costs) when the passengers leave. It returns home with over 300 passengers and makes a profit of over 22k when the passengers leave. Have I got a setting in wrong in the config.tab or is this another bug?
« Last Edit: September 09, 2009, 10:51:19 AM by whoami »

Offline whoami

  • Devotees (Inactive)
  • *
  • Posts: 693
Re: poor game flow in large maps
« Reply #16 on: September 03, 2009, 11:22:18 AM »
As I wrote, my suggestions (if they matter at all) will not help (much) in the case of the passenger/mail network.

I could set the station coverage back to 2 but surely that would lead to an increase in Trams/Tram lines/Stations, creating even more route finding?
Directly, it will only increase routing calculation effort to a smaller extent, because of the larger number of stations ("hops"). But usually, you will have more different lines, more interconnection points and more transfers, and that makes a big difference.

Quote
It makes a loss of over 41 thousand (not in travel costs) when the passengers leave. It returns home with over 300 passengers and makes a profit of over 22k when the passengers leave. Have I got a setting in wrong in the config.tab or is this another bug?
(I assume that you don't refer to ST-experimental.)
pay_for_total_distance = 2 can lead to such a result.
Also, cost deduction at switching of the month might be involved.
« Last Edit: September 03, 2009, 11:26:25 AM by whoami »

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
Re: poor game flow in large maps
« Reply #17 on: September 03, 2009, 06:11:00 PM »
As I wrote, my suggestions (if they matter at all) will not help (much) in the case of the passenger/mail network.
Directly, it will only increase routing calculation effort to a smaller extent, because of the larger number of stations ("hops"). But usually, you will have more different lines, more interconnection points and more transfers, and that makes a big difference.

Do you mean that if I make multiple smaller Tram networks in my cities instead of one large network it would improve the flow of the game?

In Experimental the train made $721 profit on the outward journey with 3078 passengers as apposed to a $41k loss in Standard.

Additional info re-train behaviour.

Train travels A-B-A.
From A to B
journey 1. 3078 passengers, loss on arrival at B -$41k plus.
Journey 2. 3078 passengers, loss on arrival at B -$42k plus.
Journey 3. 3078 passengers, loss on arrival at B -$43k plus.
From B to A
journey 1. 325 Passengers, profit on arrival at A $22k plus.
Journey 2. 476 Passengers, profit on arrival at A $16k plus.
Journey 3. 124 passengers, profit on arrival at A $4k plus.

This doesn't make sense and it doesn't matter if I set the pay_for_total_distance to 0, 1, or 2. I realise that these settings may not affect a current SVE game. But it doesn't detract from the nonsensical profit and loss that is being generated.
« Last Edit: September 03, 2009, 08:42:12 PM by Colin »

Offline whoami

  • Devotees (Inactive)
  • *
  • Posts: 693
Re: poor game flow in large maps
« Reply #18 on: September 04, 2009, 07:39:38 PM »
Do you mean that if I make multiple smaller Tram networks in my cities instead of one large network it would improve the flow of the game?
More different lines means more schedules and transfers to check, so goods routing calculation will need more CPU time. From this aspect, a larger station coverage can support in reducing calculation effort.
The performance-wise effect of the station coverage setting depends, however, also on your network layout: The more tiles with houses are covered by several stations, the slower the game may become, because a generated passenger will check more than one station for suitable connections. But the developers can provide a better, substantiated analysis on this complex problem.

As your savegame doesn't contain an extremely large network, there may be a performance bug. Additionally, Z9999 may be right when he said that ST has become "too heavy".

Regarding the obscure revenue: It's unrelated to the program performance problems, so could you please open a new topic for that, also providing the savegame with the train? After that, I will move my own replies and any remaining related text there.

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
Re: poor game flow in large maps
« Reply #19 on: September 05, 2009, 11:01:16 AM »
As your savegame doesn't contain an extremely large network, there may be a performance bug. Additionally, Z9999 may be right when he said that ST has become "too heavy".

Oh dear! I thought my SVE contained a fairly massive network. I must be really naive.

Regarding the obscure revenue: It's unrelated to the program performance problems, so could you please open a new topic for that, also providing the savegame with the train? After that, I will move my own replies and any remaining related text there.

Ok I'll upload my SVE with the latest additions and then open a new topic.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9459
  • Languages: De,EN,JP
Re: poor game flow in large maps
« Reply #20 on: September 05, 2009, 11:38:36 AM »
The station coverage is not important.

The problem in your savegame are not the size of the map but the shear amount of waiting persons. You have several stations with more than 300000 people waiting. Updating such a monster will of course take some time.

I will look into the code to make it smoother, although no promises ...

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
Re: poor game flow in large maps
« Reply #21 on: September 06, 2009, 01:08:36 PM »
The station coverage is not important.

The problem in your savegame are not the size of the map but the shear amount of waiting persons. You have several stations with more than 300000 people waiting. Updating such a monster will of course take some time.

I will look into the code to make it smoother, although no promises ...

Thank You Prissi, I will look forward to any improvements you can make.

 In my next game (which may be awhile away due to this one being nowhere near completed) I will reduce the Passenger Level.

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
Re: poor game flow in large maps
« Reply #22 on: September 07, 2009, 08:17:52 PM »
@ Prissi,

Does nightly 2640 contain your changes?

knightly

  • Guest
Re: poor game flow in large maps
« Reply #23 on: September 07, 2009, 11:36:01 PM »
Hi Colin :)

According to the source code update history, Prissi's changes for speeding up re-routing have already been incorporated. So, does it improve the responsiveness of your game?

Knightly

Offline Colin

  • Devotees (Inactive)
  • *
  • Posts: 663
  • Certa Cito
Re: poor game flow in large maps
« Reply #24 on: September 08, 2009, 04:07:40 AM »
Hi Colin :)

According to the source code update history, Prissi's changes for speeding up re-routing have already been incorporated. So, does it improve the responsiveness of your game?

Knightly


No, if anything it's getting worse. I'm sorry to say.

Offline z9999

  • Devotees (Inactive)
  • *
  • Posts: 848
Re: poor game flow in large maps
« Reply #25 on: September 08, 2009, 05:52:54 AM »
No, if anything it's getting worse. I'm sorry to say.

Same here.
What we want is not to solve jumping convoi, but to solve unresponsive.
In recent simutrans, idle time is zero during long time even though convoi keep moving well.

knightly

  • Guest
Re: poor game flow in large maps
« Reply #26 on: September 08, 2009, 06:35:05 AM »
Quote
void spieler_t::step()
{
   if(  halt_list.get_count()>0  ) {
 
      uint32 it = halt_iterator_start;
      slist_iterator_tpl <halthandle_t> iter( halt_list );
      while(  it>0  &&  iter.next()  ) {
         it--;
      }
      if(  it>0  ) {
         halt_iterator_start = 0;
      }
      else {
         uint32 units_handled = 0;
         while(  units_handled<8192  ) {
            if(  !iter.next()  ) {
               halt_iterator_start = 0;
               break;
            }
            halt_iterator_start ++;
            // iterator until 8192 passengers were handled
            units_handled += iter.get_current()->sum_all_waiting_goods();
            iter.get_current()->step();
         }
         INT_CHECK("simplay 156");
      }
   }
}

After looking at Prissi's changes, I think one of the reasons why the unresponsiveness problem has aggravated lies in the fact that >= 8192 ware (not ware packets) are re-routed in each step. For hub stations with large-sized ware packets (i.e. large menge), this will indeed help to prevent too much re-routing work in one step; but with non-hub halts, they usually have only small-sized ware packets (please see the attached image), so it needs to process many non-hub halts before the limit of 8192 can be reached.

IMHO, re-routing workload depends not on packet size (menge), but on packet count. More packets, more re-routing work, and vice versa. So, I would suggest to change from counting ware menge to counting the number of packets processed. Of course, the limit of 8192 also needs to be reduced accordingly.

« Last Edit: September 08, 2009, 06:50:23 AM by Knightly »

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9459
  • Languages: De,EN,JP
Re: poor game flow in large maps
« Reply #27 on: September 08, 2009, 08:47:12 PM »
The problem in Colin save are that there are several stations with 300000 passengers. Updating (rerouting) them takes nearly a second on my computer. During this no user interaction is possible, since such can be only done outside a step. The only other way would be that the stations themselves only handle 8192 each step, and indicate the result then. But this would require more complex logik, as changes of the passenger numbers must be also taken into account (i.e. something like a dirty flag for ware in stations.) I will look into this next week.

knightly

  • Guest
Re: poor game flow in large maps
« Reply #28 on: September 08, 2009, 10:04:32 PM »
Edit : Post removed as the provided screenshot was wrong.
« Last Edit: September 09, 2009, 06:44:39 AM by Knightly »

Offline HeinBloed

  • *
  • Posts: 79
  • Languages: DE, EN, FR
Re: poor game flow in large maps
« Reply #29 on: September 09, 2009, 12:23:14 AM »
Knightly,

I really only stumbled accross this thread randomly, but looking at your screenshot, I strongly suspect that you fell victim to a display bug: the sum of passengers of the packages displayed is never 300k, not even close. I guess lots of destinations with letter >B are missing.

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9459
  • Languages: De,EN,JP
Re: poor game flow in large maps
« Reply #30 on: September 09, 2009, 06:31:25 AM »
Display works only with integer coordintes with 16bit; therefore I think the scrollbar just messed up. It is very easy to time those stations and they have like 4000-30000 packets and cn take up to 800ms or rerouting (on this computer here).

knightly

  • Guest
Re: poor game flow in large maps
« Reply #31 on: September 09, 2009, 06:42:19 AM »
Prissi,

Really sorry! I didn't realise that the list is incomplete. I loaded Colin's save game again and looked at the same station, and the list kept growing gradually when I tried to scroll down.

Nevertheless, I think it is more logical to count the number of packets instead of the sum of menge. Hope you can consider this suggestion when you look at the code later.

Knightly



Offline Dwachs

  • DevTeam, Coder/patcher
  • Administrator
  • *
  • Posts: 4564
  • Languages: EN, DE, AT
Re: poor game flow in large maps
« Reply #32 on: September 09, 2009, 07:15:08 AM »
[offtopic]
this:

I strongly suspect that you fell victim to a display bug: the sum of passengers of the packages displayed is never 300k, not even close. I guess lots of destinations with letter >B are missing.

should be fixed in revision 2642.
[/offtopic]

Offline gerw

  • Coder/patcher
  • *
  • Posts: 618
Re: poor game flow in large maps
« Reply #33 on: September 18, 2009, 02:19:31 PM »
In the current implementation, if the goods of a station are rerouted, simutrans does a BFS for each packet. Since no heuristic is used, it is possible to do this in only one BFS, until the target for each packet is found.

I will try to implement this, if there is any free time next week...

Offline prissi

  • Developer
  • Administrator
  • *
  • Posts: 9459
  • Languages: De,EN,JP
Re: poor game flow in large maps
« Reply #34 on: September 19, 2009, 09:47:48 PM »
Please check again with curretn nightly. Should be much smoother (only lagging on my eeePC).