News:

Simutrans Sites
Know our official sites. Find tools and resources for Simutrans.

Dr. Memory report with freelist debug

Started by jamespetts, July 21, 2013, 07:26:18 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

jamespetts

I have been trying to work out why the server crashes from time to time, so I have enabled DEBUG_FREELIST on my local debug build, and run Dr. Memory. I get the following output:

Edit: It seems that the full output was too big to fit on here. I have made some alterations to the code to try to fix the error: I have not fixed it, but narrowed it down as follows:


Error #1: UNADDRESSABLE ACCESS: reading 0x06154faa-0x06154fac 2 byte(s) within 0x06154faa-0x06154fae
# 0 convoi_t::set_next_stop_index               [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\simconvoi.cc:5331]
# 1 aircraft_t::calc_route                      [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\vehicle\simvehikel.cc:4738]
# 2 convoi_t::calc_route                        [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\simconvoi.cc:798]
# 3 convoi_t::drive_to                          [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\simconvoi.cc:1183]
# 4 convoi_t::step                              [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\simconvoi.cc:1547]
# 5 karte_t::step                               [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\simworld.cc:4221]
# 6 karte_t::interactive                        [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\simworld.cc:7306]
# 7 simu_main                                   [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\simmain.cc:1258]
# 8 sysmain                                     [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\simsys.cc:703]
# 9 WinMain                                     [c:\users\james\documents\development\simutrans\simutrans-experimental-sources\simsys_w.cc:731]
Note: @0:05:22.462 in thread 6764
Note: next higher malloc: 0x06154fb0-0x06156684
Note: instruction: mov    (%eax) -> %ecx


This error occurs in the following piece of code:


// set next stop before breaking will occur (or route search etc.)
// currently only used for tracks
void convoi_t::set_next_stop_index(uint16 n)
{
    // stop at station or signals, not at waypoints
   if(n == INVALID_INDEX)
   {
       // find out if stop or waypoint, waypoint: do not brake at waypoints
       bool reverse_waypoint = false;
       const koord3d route_end = route.back();
       if(fahr[0]->get_typ() != ding_t::aircraft)
       {
           const int count = fpl->get_count();
           for(int i = 0; i < count; i ++)
           {
               const linieneintrag_t &eintrag = fpl->eintrag[i];
               if(eintrag.pos == route_end)
               {
                    reverse_waypoint = eintrag.reverse;
                    break;
               }
           }
       }

       grund_t const* const gr = welt->lookup(route_end);
       if(  gr  &&  (gr->is_halt() || reverse_waypoint)  )
       {
           n = route.get_count()-1;
       }
   }
    next_stop_index = n+1;
}


The relevant line in which the error is generated (5331 in the message above) is:


const koord3d route_end = route.back();


This is only ever generated with aircraft. This does not crash the client, but a gdb backtrace from a few days ago suggested that it did crash the server at one point.

It is not clear to me what is happening here, and why calling route.back() should produce an unaddressable access (sometimes) with aircraft, and why, despite that, the program does not crash. This could explain, however, the phenomenon of aircraft seeming to get stuck sometimes. Any assistance in tracking this down would be much appreciated.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

prissi

Since route is a vector and not a list, it does not use the freelist code at all. Hence route.back() cannot cause invalid memory access (if that is the meaning of the error message). Dr. Memory must complain about something happened earlier.

(NB: Unaddressable access is a very cryptic error message. It does not inspire great confidence in Dr. memory at all.)

jamespetts

Hmm - the meaning of "unaddressable access" is explained here. It's not clear to me what is going on either, but this is not just a spurious result from Dr. Memory - the server crashed at one point because of an error with this code, which I found with a backtrace, so there is something real here, but it is hard to work out what.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

isidoro

With the information you've given, I would say that what happens is the following:

       
  • The route of the airplane happens to be empty (whether that is possible or it is a bug, I can't say)
  • The method route.back() tries to get the last element of the route and, since 1., get the element just one position before the beginning of the vector, and Dr. Memory complains.
Just to be sure, try to write assert(route.get_count()>0); just before that line and see if that assertion is not fulfilled some times (those in which the application would fail, btw)...

jamespetts

#4
The server is currently crashing reliably with this issue. Running it with GDB, I get the following (the first part being the debug output and the second part a backtrace):


Message: binary_heap_tpl():     initialized
Message: route_t::append_straight_route():      start from (1388,461) to (243,484)
Message: route_t::append_straight_route():      to (243,484) found.
Message: route_t::append_straight_route():      start from (243,484) to (263,484)
Message: route_t::append_straight_route():      to (263,484) found.
Message: binary_heap_tpl():     initialized
Message: route_t::append_straight_route():      start from (933,555) to (991,561)
Message: route_t::append_straight_route():      to (991,561) found.
Message: route_t::append_straight_route():      start from (991,561) to (1011,561)
Message: route_t::append_straight_route():      to (1011,561) found.
Message: aircraft_t::find_route_to_stop_position():     no free prosition found!
Message: binary_heap_tpl():     initialized
Message: route_t::append_straight_route():      start from (1178,950) to (1020,954)
Message: route_t::append_straight_route():      to (1020,954) found.
Message: route_t::append_straight_route():      start from (1020,954) to (1041,954)
Message: route_t::append_straight_route():      to (1041,954) found.
Message: binary_heap_tpl():     initialized
Message: route_t::calc_route(): No route from 1549,622 to 1026,948 found
Message: aircraft_t::calc_route():      search runway start near (1026,948,-1)
Message: binary_heap_tpl():     initialized
Message: aircraft_t::calc_route():      failed

Program received signal SIGSEGV, Segmentation fault.
0x000000000065111a in operator== (a=..., b=...) at boden/../dataobj/koord3d.h:84
84              return ((a.x-b.x)|(a.y-b.y)|(a.z-b.z))==0;
(gdb) backtrace
#0  0x000000000065111a in operator== (a=..., b=...) at boden/../dataobj/koord3d.h:84
#1  0x00000000006667bd in convoi_t::set_next_stop_index (this=0x28cdaee0, n=65530) at simconvoi.cc:5336
#2  0x0000000000727e02 in aircraft_t::calc_route (this=0x28cddf70, start=..., ziel=..., max_speed=885, route=0x28cdaf78)
    at vehicle/simvehikel.cc:4737
#3  0x0000000000654c84 in convoi_t::calc_route (this=0x28cdaee0, start=..., ziel=..., max_speed=885) at simconvoi.cc:798
#4  0x0000000000655edc in convoi_t::drive_to (this=0x28cdaee0) at simconvoi.cc:1183
#5  0x000000000065761e in convoi_t::step (this=0x28cdaee0) at simconvoi.cc:1547
#6  0x00000000006f5c3e in karte_t::step (this=0x1ab8900) at simworld.cc:4221
#7  0x0000000000703993 in karte_t::interactive (this=0x1ab8900, quit_month=2147483647) at simworld.cc:7264
#8  0x00000000006a5ead in simu_main (argc=17, argv=0x7fffffffe4b8) at simmain.cc:1258
#9  0x00000000006b5b51 in sysmain (argc=17, argv=0x7fffffffe4b8) at simsys.cc:703
#10 0x00000000007591d2 in main (argc=17, argv=0x7fffffffe4b8) at simsys_posix.cc:147


I am trying the assert now...

Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

prissi

If trying assert, be sure you have not defined NDEBUG and instead defined DEBUG or else no asserts will be active at all.

jamespetts

#6
Prissi - thank you for the reminder. I do make sure that I do not define NDEBUG with my debug builds, and have managed to get the assert to work.

The assert fails a single time when loading this with the Windows debugger. Looking on the map, the route 1549,622 to 1026,948, which the Linux server reports as failed, is interesting: it is a route between the apron of one airport and a tile just outside another airport. That fact, that it is a tile just outside another airport rather than the actual apron of that airport or even the runway, I think, is deliberate, as the aircraft has to circle if it cannot land straight away. However, on this particular occasion, that tile happens to be a piece of grass taxiway, otherwise unconnected to the airport, used by the player to fence off the airport. Looking at that tile, there is an aircraft immediately above it, in the air, but showing "no route" and not moving.

Edit: Modifying the relevant part of the code to this:


// set next stop before breaking will occur (or route search etc.)
// currently only used for tracks
void convoi_t::set_next_stop_index(uint16 n)
{   
    // stop at station or signals, not at waypoints
-  if(n == INVALID_INDEX)
+  if(n == INVALID_INDEX && !route.empty())
   {
       // find out if stop or waypoint, waypoint: do not brake at waypoints
       bool reverse_waypoint = false;
       const koord3d route_end = route.back();
       if(fahr[0]->get_typ() != ding_t::aircraft)
       {
           const int count = fpl->get_count();
           for(int i = 0; i < count; i ++)
           {
               const linieneintrag_t &eintrag = fpl->eintrag[i];
               if(eintrag.pos == route_end)
               {
                    reverse_waypoint = eintrag.reverse;
                    break;
               }
           }
       }

       grund_t const* const gr = welt->lookup(route_end);
       if(  gr  &&  (gr->is_halt() || reverse_waypoint)  )
       {
           n = route.get_count()-1;
       }
   }
    next_stop_index = n+1;
}


prevents the "UNADDRESSABLE ACCESS" errors in Dr. Memory - I have not yet tried this on the Linux server to see whether it prevents crashes. However, it strikes me that there is still something going wrong such that aircraft cannot find a route, as there is nothing in principle which should stop them being able to find a route to a place in the sky.

Edit 2: Checking, this indeed does not prevent the problem with the aircraft stuck in the sky with "no route", which, after hovering for a while, eventually teleports to the depot. Is it significant that the aircraft is on the very last tile of its route when loading?

Edit 3: I have now fixed the no route issue - it was caused by the aircraft not recognised as being in flight when trying to route from flying over a taxiway tile.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.

prissi

The backtrace would be indeed interesting, as this convois is usually only called from vechicles waiting at signals. Especially airplanes should not call this at all.

jamespetts

I think that it was Bernd who altered this to cause taxiing aircraft to call this, but there was a bug whereby flying aircraft would call it if they were over a taxiway: I have fixed that last bug by adding a check to see whether the aircraft is flying.
Download Simutrans-Extended.

Want to help with development? See here for things to do for coding, and here for information on how to make graphics/objects.

Follow Simutrans-Extended on Facebook.