The International Simutrans Forum

 

Author Topic: Lightweight private car routing  (Read 2579 times)

0 Members and 1 Guest are viewing this topic.

Offline freddyhayward au

  • *
  • Posts: 117
  • Languages: EN
Re: Lightweight private car routing
« Reply #70 on: February 04, 2020, 12:28:50 AM »
I got a segfault with this backtrace on deleting a road on this branch:

Code: [Select]
Thread 1 "simutrans-exten" received signal SIGSEGV, Segmentation fault.
0x000055555562c98f in slist_tpl<hashtable_tpl<koord, koord3d, koordhash_tpl<koord> >::node_t>::begin (this=0x4b0) at boden/wege/../../tpl/slist_tpl.h:393
393 iterator begin() { return iterator(head, NULL); }
(gdb) bt full
#0  0x000055555562c98f in slist_tpl<hashtable_tpl<koord, koord3d, koordhash_tpl<koord> >::node_t>::begin (this=0x4b0) at boden/wege/../../tpl/slist_tpl.h:393
No locals.
#1  0x00005555558ce3df in hashtable_tpl<koord, koord3d, koordhash_tpl<koord> >::set (this=0x60, key=..., object=...) at boden/wege/../../tpl/hashtable_tpl.h:335
        iter = {ptr = 0x1ffffba00, pred = 0x5555ab2fe968}
        end = {ptr = 0x7fffffffba33, pred = 0x555559cba0b0}
        bag = <error reading variable>
        node = {key = {x = -17869, y = -1, static invalid = {x = -1, y = -1, static invalid = <same as static member of an already seen type>, static north = {x = 0, y = -1,
                static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>, static south = {x = 0, y = 1,
                  static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                  static south = <same as static member of an already seen type>, static east = {x = 1, y = 0, static invalid = <same as static member of an already seen type>,
                    static north = <same as static member of an already seen type>, static south = <same as static member of an already seen type>,
                    static east = <same as static member of an already seen type>, static west = {x = -1, y = 0, static invalid = <same as static member of an already seen type>,
                      static north = <same as static member of an already seen type>, static south = <same as static member of an already seen type>,
                      static east = <same as static member of an already seen type>, static west = <same as static member of an already seen type>, static nsew = {{x = 0, y = -1,
                          static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                          static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
                          static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                          static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                          static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}, {x = 0, y = 1,
                          static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                          static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
                          static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                          static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                          static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}, {x = 1, y = 0,
                          static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                          static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
                          static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                          static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                          static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}, {x = -1, y = 0,
                          static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                          static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
                          static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                          static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                          static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}},
                      static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                      static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>},
                    static nsew = <same as static member of an already seen type>, static neighbours = <same as static member of an already seen type>,
                    static second_neighbours = <same as static member of an already seen type>, static from_ribi = <same as static member of an already seen type>,
                    static from_hang = <same as static member of an already seen type>}, static west = <same as static member of an already seen type>,
                  static nsew = <same as static member of an already seen type>, static neighbours = <same as static member of an already seen type>,
                  static second_neighbours = <same as static member of an already seen type>, static from_ribi = <same as static member of an already seen type>,
                  static from_hang = <same as static member of an already seen type>}, static east = <same as static member of an already seen type>,
                static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>},
              static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
              static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
              static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
              static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>},
            static north = <same as static member of an already seen type>, static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
            static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
            static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
            static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}, value = {x = 32767, y = 0, z = -80 '\260',
            static invalid = {x = -1, y = -1, z = -1 '\377', static invalid = <same as static member of an already seen type>}}}
#2  0x00005555558c9159 in stadt_t::process_private_car_routes (this=0x5555a2b5c960) at simcity.cc:6139
--Type <RET> for more, q to quit, c to continue without paging--
        previous_tile = {x = 115, y = 517, z = 0 '\000', static invalid = {x = -1, y = -1, z = -1 '\377', static invalid = <same as static member of an already seen type>}}
        gr = 0x5555ab2fe968
        road_tile = 0x0
        route = @0x5555a96e0560: {key = {x = 114, y = 518, static invalid = {x = -1, y = -1, static invalid = <same as static member of an already seen type>, static north = {x = 0, y = -1,
                static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>, static south = {x = 0, y = 1,
                  static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                  static south = <same as static member of an already seen type>, static east = {x = 1, y = 0, static invalid = <same as static member of an already seen type>,
                    static north = <same as static member of an already seen type>, static south = <same as static member of an already seen type>,
                    static east = <same as static member of an already seen type>, static west = {x = -1, y = 0, static invalid = <same as static member of an already seen type>,
                      static north = <same as static member of an already seen type>, static south = <same as static member of an already seen type>,
                      static east = <same as static member of an already seen type>, static west = <same as static member of an already seen type>, static nsew = {{x = 0, y = -1,
                          static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                          static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
                          static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                          static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                          static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}, {x = 0, y = 1,
                          static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                          static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
                          static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                          static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                          static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}, {x = 1, y = 0,
                          static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                          static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
                          static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                          static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                          static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}, {x = -1, y = 0,
                          static invalid = <same as static member of an already seen type>, static north = <same as static member of an already seen type>,
                          static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
                          static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                          static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                          static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}},
                      static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                      static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>},
                    static nsew = <same as static member of an already seen type>, static neighbours = <same as static member of an already seen type>,
                    static second_neighbours = <same as static member of an already seen type>, static from_ribi = <same as static member of an already seen type>,
                    static from_hang = <same as static member of an already seen type>}, static west = <same as static member of an already seen type>,
                  static nsew = <same as static member of an already seen type>, static neighbours = <same as static member of an already seen type>,
                  static second_neighbours = <same as static member of an already seen type>, static from_ribi = <same as static member of an already seen type>,
                  static from_hang = <same as static member of an already seen type>}, static east = <same as static member of an already seen type>,
                static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
                static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
                static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>},
              static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
              static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
              static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
              static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>},
            static north = <same as static member of an already seen type>, static south = <same as static member of an already seen type>, static east = <same as static member of an already seen type>,
            static west = <same as static member of an already seen type>, static nsew = <same as static member of an already seen type>,
            static neighbours = <same as static member of an already seen type>, static second_neighbours = <same as static member of an already seen type>,
            static from_ribi = <same as static member of an already seen type>, static from_hang = <same as static member of an already seen type>}, value = {data = 0x7fffa816f410, size = 205,
            count = 190}}
        once1__6117 = false
        iter__6117 = {bag_i = 0x5555a2b600c8, bag_end = 0x5555a2b605f0, node_i = {ptr = 0x5555a96e0558}}
        end__6117 = {bag_i = 0x5555a2b605f0, bag_end = 0x5555a2b605f0, node_i = {ptr = 0x0}}
        break__6117 = true
--Type <RET> for more, q to quit, c to continue without paging--
        container___6117 = @0x5555a2b5fc78: {<hashtable_tpl<koord, vector_tpl<koord3d>, koordhash_tpl<koord> >> = {bags = {{head = 0x7fff741d6518, tail = 0x7fff741d1b78, node_count = 2}, {
                head = 0x5555ad752ff8, tail = 0x7fff741d1718, node_count = 2}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fff741d2738, tail = 0x7fff741d2738, node_count = 1}, {
                head = 0x7fff601011f8, tail = 0x7fff601011f8, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fffa8098308, tail = 0x555555f6e068, node_count = 2}, {
                head = 0x5555ad74e1d8, tail = 0x5555ab7f6f08, node_count = 3}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fff741d65b8, tail = 0x7fff741d65b8, node_count = 1}, {head = 0x0,
                tail = 0x0, node_count = 0}, {head = 0x7fff741d61f8, tail = 0x5555a8ae6db8, node_count = 3}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fff600fc4f8, tail = 0x7fff600fc4f8,
                node_count = 1}, {head = 0x7fff600fc618, tail = 0x7fff600fc618, node_count = 1}, {head = 0x5555ad754e58, tail = 0x7fff741d1758, node_count = 3}, {head = 0x0, tail = 0x0, node_count = 0},
              {head = 0x5555ab162828, tail = 0x5555ab162828, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0,
                node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x555555f68fe8, tail = 0x555555f68fe8, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {
                head = 0x5555ad754b98, tail = 0x7fffa01d7668, node_count = 2}, {head = 0x7fff741d1898, tail = 0x7fff741d1898, node_count = 1}, {head = 0x5555ad7553f8, tail = 0x555555f6ab08,
                node_count = 2}, {head = 0x5555ad754198, tail = 0x5555ad754198, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0,
                tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fff741d67d8, tail = 0x7fff741d67d8, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {
                head = 0x7fff741d1a58, tail = 0x5555aaa09f98, node_count = 4}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fff600fc9f8, tail = 0x7fffa8098808, node_count = 4}, {head = 0x0,
                tail = 0x0, node_count = 0}, {head = 0x5555a37dd698, tail = 0x5555a37dd698, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {
                head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x555555fdca38, tail = 0x555555fdca38, node_count = 1}, {head = 0x0, tail = 0x0,
                node_count = 0}, {head = 0x5555ad7544b8, tail = 0x5555a3367258, node_count = 2}, {head = 0x5555a96e0558, tail = 0x5555a96e0558, node_count = 1}, {head = 0x7fff741d2378,
                tail = 0x7fff741d2378, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fff741d2438, tail = 0x7fff741d2438, node_count = 1}, {head = 0x5555ad7526f8,
                tail = 0x5555ad7526f8, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0,
                tail = 0x0, node_count = 0}, {head = 0x5555ac3691a8, tail = 0x5555ac3691a8, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x5555a8f0edc8, tail = 0x555555fba8e8,
                node_count = 2}, {head = 0x5555ad7537b8, tail = 0x555555fbb648, node_count = 2}, {head = 0x7fff600fc698, tail = 0x7fff600fc698, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0},
              {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fff600ff078, tail = 0x7fff600ff078, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fff741d1df8,
                tail = 0x7fff741d1df8, node_count = 1}, {head = 0x7fff741d64f8, tail = 0x7fff741d64f8, node_count = 1}, {head = 0x7fffa8099488, tail = 0x7fffa8099488, node_count = 1}, {head = 0x0,
                tail = 0x0, node_count = 0}, {head = 0x5555ad754898, tail = 0x5555ad754898, node_count = 1}, {head = 0x5555ad752718, tail = 0x5555ad7546d8, node_count = 2}, {head = 0x5555ad753458,
                tail = 0x5555ad753458, node_count = 1}, {head = 0x7fff741d2598, tail = 0x7fff741d2598, node_count = 1}, {head = 0x555555f6b488, tail = 0x555556005708, node_count = 2}, {
                head = 0x5555a8e232d8, tail = 0x5555a8e232d8, node_count = 1}, {head = 0x7fff741d5fd8, tail = 0x7fff741d5fd8, node_count = 1}, {head = 0x0, tail = 0x0, node_count = 0}, {
                head = 0x5555ad7527b8, tail = 0x555555f6ab28, node_count = 2}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fff600fc3f8, tail = 0x7fff600fc3f8, node_count = 1}, {head = 0x0,
                tail = 0x0, node_count = 0}, {head = 0x555555f694e8, tail = 0x7fff60100738, node_count = 8}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x7fffa80998c8, tail = 0x7fff741d66b8,
                node_count = 2}, {head = 0x5555ad74db58, tail = 0x5555ad74ea98, node_count = 2}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x555555f6afa8, tail = 0x555555f6afa8, node_count = 1},
              {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x5555a8f4ffb8, tail = 0x5555ad5919a8,
                node_count = 3}, {head = 0x7fffa8096e48, tail = 0x555555fb86c8, node_count = 3}, {head = 0x7fff741d63b8, tail = 0x7fff600fc5d8, node_count = 2}, {head = 0x7fffa80982e8,
                tail = 0x7fffa80982e8, node_count = 1}, {head = 0x5555ad753ab8, tail = 0x5555ad593288, node_count = 2}, {head = 0x5555ad74e438, tail = 0x5555ad74e438, node_count = 1}, {
                head = 0x555555f6a0c8, tail = 0x555555f6a0c8, node_count = 1}, {head = 0x5555ad752a18, tail = 0x7fffa01de648, node_count = 2}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0,
                tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}, {head = 0x0, tail = 0x0, node_count = 0}}, count = 95}, <No data fields>}
        once0__6117 = false
#3  0x00005555558bab58 in stadt_t::step_heavy (this=0x5555a2b5c960) at simcity.cc:2692
        error = 0
        __PRETTY_FUNCTION__ = "void stadt_t::step_heavy()"
#4  0x00005555559aa7c6 in karte_t::step (this=0x555559cba0b0) at simworld.cc:5529
        i = 0x5555a2b5c960
        once1__5521 = false
        iter__5521 = {ptr = 0x5555a4131480}
        end__5521 = {ptr = 0x5555a41315f0}
        break__5521 = true
        container___5521 = @0x555559cbb0c0: {nodes = 0x5555a4131480, size = 24, count = 23, total_weight = 84476}
        once0__5521 = false
        time = 65459
        delta_t = 100
        check_city_routes = false
        season_change = false
        snowline_change = false
        step_cities_count = 0
        dt = 0
#5  0x00005555559be61f in karte_t::interactive (this=0x555559cba0b0, quit_month=2147483647) at simworld.cc:10549
        time = 65458
        hashes_ok = {data = 0x0, size = 0, count = 0}
        ms_difference = 0
--Type <RET> for more, q to quit, c to continue without paging--
#6  0x000055555594b69b in simu_main (argc=1, argv=0x7fffffffe608) at simmain.cc:1382
        pause_after_load = false
        welt = 0x555559cba0b0
        view = 0x555559cc47a0
        eventmanager = 0x555559cc46b0
        resolutions = {{640, 480}, {800, 600}, {1024, 768}, {1280, 1024}, {704, 560}}
        disp_width = 704
        disp_height = 560
        fullscreen = 0
        quit_month = 2147483647
        path_sep = 0x555555a7501e "/"
        pak_diagonal_multiplier = 724
        pak_tile_height = 8 '\b'
        pak_height_conversion_factor = 2 '\002'
        found_settings = false
        found_simuconf = true
        multiuser = true
        simuconf = {file = 0x0}
        path_to_simuconf = "config/simuconf.tab\000\000\000\000"
        version = 0x555555a750f0 "Simutrans version 120.2.1 Extended Nightly development build 14.9 from Feb  3 2020 #d1e3688\n"
        cli_syslog_enabled = false
        cli_syslog_tag = 0x0
        file = {mode = 4, saving = false, buffered = false, curr_buff = 0, buf_pos = {0, 0}, buf_len = {0, 0}, ls_buf = {0x0, 0x0}, version = 0, extended_version = 0, extended_revision = 0, ident = 0,
          pak_extension = '\000' <repeats 255 times>, filename = "", fd = 0x555555d0ff40, static save_mode = loadsave_t::zipped, static autosave_mode = loadsave_t::zipped}
        xml_filename = "settings-extended-debug.xml\000\000\000\000"
        xml_settings_found = false
        obj_conf = "{dir}/simutrans/simuconf.tab"
        themes_ok = true
        parameter = {0, 0}
        new_world = false
        loadgame = ""
#7  0x0000555555960520 in sysmain (argc=1, argv=0x7fffffffe608) at simsys.cc:825
        buffer2 = 0x0
        buffer = "{dir}/Simutrans-Extended-Complete/simutrans/simutrans-extended-debug", '\000' <repeats 24 times>, "\020\000\000\000\000\000\000\000R\345td\004\000\000\000\310\t\002\000\000\000\000\000\310\031\002\000\000\000\000\000\310\031\002\000\000\000\000\000\070\006\000\000\000\000\000\000\070\006\000\000\000\000\000\000\001\000\000\000\000\000\000\000\004\000\000\000\020\000\000\000\005\000\000\000GNU\000\002\000\000\300\004\000\000\000"...
        length = 88
#8  0x0000555555a3e1aa in main (argc=1, argv=0x7fffffffe608) at simsys_s2.cc:792
No locals.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #71 on: February 04, 2020, 10:51:07 AM »
I believe that I may have fixed the above crash - I should be grateful if you could re-test.

Offline Matthew gb

  • *
  • Posts: 316
    • Japan Railway Journal
  • Languages: EN, some ZH, DE & SQ
Re: Lightweight private car routing
« Reply #72 on: February 04, 2020, 03:42:21 PM »
Thank you for that. I did want to load the 1960s game, but when I click the link, I get only a blank page; is there some complexity here?

Apologies, there was a typo in the BBcode. Fixed now.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #73 on: February 05, 2020, 12:37:13 AM »
Apologies, there was a typo in the BBcode. Fixed now.

Splendid, thank you: I have now had a chance to look at this. A performance profiler suggests that ~8% of CPU time is being spent on processing private car routes in this map, of which ~4% is on clearing existing private car routes before processing - but  there is very little else computationally intensive going on, since there are no player transport routes save for a single 'bus line, so all the other CPU intensive activities, such as the path explorer, convoy routing, passenger and mail generation, vehicle physics and route unreserving are not applicable here, such that private car routing is the only CPU intensive activity apart from the graphics.

It is possible that a more sophisticated design might allow route processing to be faster; one possibility is to use a fixed size array of hashtables in each tile (rather than having a single hashtable), with one for reading the other for writing, swapping between them periodically; but this is likely to be very complex to implement, and it is not clear to me how deleting routes would work in such a system. Freahk suggested a system of this sort, so he might have a clearer idea of how to do this.

Another possibility is having parallel multi-threading of this. At present, the processing of private car routes is done in a single thread. This is now set to run concurrently with the passenger and mail generation threads. In a developed game, the passenger and mail generation threads are very CPU intensive, so take a long time; running this concurrently with them, especially on a computer with many cores, will allow this to be more time efficient.

However, on a map which is nearly all private car routing, the passenger and mail generation takes so little time that this has no benefit. Running it in parallell, however, might help a little: what this would do is, for each route to process, have one thread process the first part of the route, another thread process the second part of the route and so on, depending on the number of threads assigned to this task. This is the same system as the multi-threaded route unreservation system for trains. This would be more efficient no matter how much that the passenger and mail generation threads were being used. However, this would take a significant amount of work to code, as multi-threading is always complex and difficult work.

I should be interested to hear from other people as to the performance with Matthew's 1960 era map; this would be most useful in determining the significance of this issue. I also notice that Matthew's computer is rather limited in RAM - this saved game alone when running takes ~1.2Gb of RAM, which is a significant proportion of 4GB. If Matthew were running low on RAM, this would potentially impact performance significantly.

One other thing to note is that one can set "assume_everywhere_connected_by_road" to disable private car route searching entirely, which will reduce any performance impact.

In any event, thank you for testing, and I should be interested in any feedback on performance that any of you might have.

Offline Phystam jp

  • *
  • Posts: 379
  • Pak256.Ex developer
    • Pak256 wiki page
  • Languages: JP, EN, EO
Re: Lightweight private car routing
« Reply #74 on: February 05, 2020, 07:18:54 AM »
Sometimes I observe that a private car stops at the front of a junction, such as an exit of the highway. It looks that the car is searching for his direction where he wants to go. It may cause severe traffic congestion behind the highway exit.
In order to prevent that phenomenon, I suggest guide signs. A guide sign acts as a distant signal on railways, and it tells private cars the direction of the next junction (or the next next junction, when the junction has multi tiles). If he finds which direction to go, he does not need to stop at the junction.

Offline Phystam jp

  • *
  • Posts: 379
  • Pak256.Ex developer
    • Pak256 wiki page
  • Languages: JP, EN, EO

Offline Freahk

  • *
  • Posts: 577
  • Languages: DE, EN
Re: Lightweight private car routing
« Reply #76 on: February 05, 2020, 04:50:11 PM »
Finally I had compiled and tested this on my dev device (i7-6700HQ, 16GB ram, opensuse, stephenson-siemens save, compiled with OPTIMISE=1 setting, sdl2)

Performance aspects
First, the performance aspect as it was discussed quite a lot in the last few posts. I could not find any significiant perfromance issues yesterday evening, so I let the game run the whole night and today I still get more or less constant 30 Fps out of it. It varies in between 27-32 Fps. It's fine and without looking at the Fps number, I did not notice these little variations.

To directly compare this without digging deep in profilers and stuff like that (James already did this quite a lot, so I don't think such observations are quite helpful), I just launched both self compiled binaries (master branch and private-car-routing-lightweight branch) at the same time with the same save from stephenson-siemens and let both running for several hours.
I should mention that both games did not seem to be bottlenecked by any ressources. Both were running at ~30 Fps whenever I had a quick look at them and after 4 hours, both games were still at the same ingame time +/- 2s.

The private car binary consumed ~3200 mb additional memory, which is a fairly small amount.
Further, the private car binary consumed slightly more CPU, however the main thread consumes even a little bit less CPU on average. I had observed some main thread CPU spikes pushing the updated routes to the roads) in the main thread but could not measure significiant spikes, so it is either to fast to show as at the given sampling rate or it is bandwith limited rather than cpu limited.

I will continue observations on Mathews posted map soon.

Gameplay aspects
Now let's get to the gameplay aspects of this feature. How well does it solve our issue of cities jammed by randomly stucking around drivers?
I hav observed this on the mentioned stephenson-siemens game and the latest save from my server, which is from December.
I am quite sorry, I have not packed it into a nice story this time but I have a quite huge backlog of stuff and packing this into a story without missing the points will simply take a lot more time to write.

I would say the core functionality of this patch works great, which is a well-working long-distance car routing, thus also less jamming within cities. City bypasses work just well and the time to update routes is quite fast, maybe even too fast with complaints about perfomance in mind. Imho it's not an issue if it takes a few minutes to update a route as long as the old route was not detected broken, in which case we need a more or less immediate update of course.

In the first place everything was just stuck but after buildung a few bypasses, traffic flow was fine and it is quite impressive to watch the cars moving. However, due to the limitations of simuroads we will need to be extremely careful when building intersections. Avoid diagonal intersections and non-motorway like connections to adjacent parallel roads. Both will greatly decrease roads throughput, the latter will even quite likely cause gridlocks. With these rules in mind, one can build a relatively well performing intercity road network.

However, there are a few more aspects than long-distance journeys:
- What happens within cities?
- What happens to car traps?
-What happens if a bypass has multiple exists for the same city?
- What happens to cities being partly an island?

What happens within cities?
At first, thanks to a little patch, there is no random mode anymore, I guess it's fine now calling it heuristics mode.
As expected, how fast cars will find a guided road depens on two factors:
1. How many different roads within the town are guided?
The more, the better, in terms of quickly finding a guided route to the destination. Towns with a bypass road will quite often only have one route through the city towards many destinations, so each car in the city has to find that road to leave the town.

2. How well does the road network (and giuded road network) match to the guesses of the heuristics?
If a car spawns in an area with many actual and city border implied dead ends, the heuristics will make quite bad guesses so it will take quite a lot of time to find a guided way out of the town.
Another point is private cars spawning e.g. in the south of a city when their destination is also south of the city but the road towards that city is in the north.
The heuristics will be quite bad in that case, so it will take quite a lot of time to find a guided way out of the town.

What happens to car traps?
In short, they won't work outside of cities but within cities they still work quite "well" when built around corners where the heuristics fails. In general, the closer to the border of a map, the more destinations will be in the same direction, the easyer it is to catch cars towards that direction before they reach the guided road.
Especially if one designs such guided roads to have as few roads connected as possible, the easyer it will be to trap cars.

What happens if a bypass has multiple exists for the same city?
If the bypass itself is within city borders, cars will be in heuristics mode, so it's more or less all right. Heuristics mode does not really like motorway exits/slip roads as cars will most often need to drive in the wrong direction for a short time.
If the bypass is outside of city borders, any traffic towards the same city (without monuments) will use the same exit, the one that reaches the city center best, although it might be a much worse route for many people in that city than another exit.
This is the "enter city" equivalent to the above described "leave city" issue. If the destination is not close to the city center, it might drive strange detours in the city, although it could just take another exit very close to their door.
So we get to the last point where this behavior gets even worse.

What happens to cities being partly an island?
Either the city center is on the island, in which case this city will only spawn private cars within the city or the city center is not on the island, in which case anyone in that city considers travelling by car.
In the first case Inhabitants that could use their private car to get to their destination won't use it. It's odd but not a huge issue.
Further there will be cars spawned within the city that can never reach their destination because it's on the other side of the water. Sometimes there might even be a road connection outside of the city borders but private cars will never find that route because it's outside of the city. Whilst also odd, this ususally is not a huge issue as there won't be that many private cars spawned for journeys within the city to have a huge impact on city traffic.
It get's much worse when the city center is connected to the road network but parts of the city are not!
In that case there will spawn a lot more cars on the island and towards the island that will cause very much congestion.

Conclusion
One can argue about the performance, so I will only consider the gameplay aspects here.
In many situations the new feature will be a huge improvement. It unstucks nonsensely stucked cities and will create a lot of traffic along main routes leading to much more realistic congestions than before if not properly counterplayed by bypasses.
However, entering and leaving guided mode could require some improvement and partly disconnected cities will be an issue.
Iirc on the Bridgewater-Brunel game there were quite a lot cities that jumped across the ocean. On Stephenson-Siemens this is not an issue.

If you are interessted in a possible solution, please let me know. It's not too complicated, essentially remaining with one route per city (including attractions) but adding sub-routes for chunks of the city.'

It is possible that a more sophisticated design might allow route processing to be faster; one possibility is to use a fixed size array of hashtables in each tile (rather than having a single hashtable), with one for reading the other for writing, swapping between them periodically; but this is likely to be very complex to implement, and it is not clear to me how deleting routes would work in such a system. Freahk suggested a system of this sort, so he might have a clearer idea of how to do this.
In short, this is called a transactional data structure.
There are multiple ways to ensure this but all use the same idea: Prepare an operation (e.g. insert, remove,...) But do not immediately show these changes to the outside before we get told the transaction is complete. In that case use the prepared data to quickly perform the actions.

The easyest approach is maintaining two identical sets of data, one being used to read from, the other one used to write to.
Once a write transaction is done, we will flush that information to to read dataset by simply swapping the datasets.
When swapping only the read dataset will be valid.
Before we can start the next transacrion, we will first need to get it in sync with the read dataset.

In our case, the cleanest way should be implementing this directly into the hash map, however, if we don't want to change the maps implementation, we could also go the less clean (and slightly less performant) way of implementing this in the route calculation logic. I will describe the latter as the description can easily be adapted to implement it directly to the hash map.

The whole progress of calculating and updating would be the following:
1. In parallel select a route to be recalculated and calculate the new route.
Calculate the union set of both routes.
For each tile in the union, prepare the inactive signpost by synching it to the used signpost.
For each tile in the old route, remove the routing information from the inactive sign.
For each tile in the new route, add the routing information to the inactive sign.
Add the (origin,destination,new_route,union_tiles) 4-tuple to the route update queue (transaction done, please flush)

2. In sync select a route from the route update queue.
For each tile in the union tiles swap the active/inactive signpost.
Replace the (origin,destination) route in origin towns route list with the new route.

We pay that ability to quickly flush our changes obviously by additional memory and some additional administration effort (does one say this?)
« Last Edit: February 05, 2020, 06:19:22 PM by Freahk »

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #77 on: February 05, 2020, 05:24:54 PM »
Thank you all for your feedback, and to Freahk for the detailed feedback especially.
Phystam: I have also observed that issue. I traced it back to the overtaking code; the car seems to think that another car is blocking its route, but I did not get as far as finding out why. It is not clear whether this is an issue specific to this code or whether it existed before but was harder to detect. Thank you for the pakset and saved game! It is very interesting to see the cars all using the freeway/highway/motorway rather than the local roads: it is most satisfying to watch.
I see that the stopping at junction bug does affect this map badly, so I may well need to look into this.

As to performance, I note that this seems to cause some issues on less powerful computers. I will have to consider implementing the parallel/sequential multi-threading for route processing: this is the only performance improvement that is likely to be feasible, I think.
I am very glad that bypasses work as intended (I hope that everyone enjoyed the old film about them above) - there is much to be said for a good bypass.

In relation to the gameplay issues, the issues identified are likely to be unsolveable without a more sophisticated algorithm. I note that Frahk suggests one (something similar to the route fragments idea that I had imagined long ago but dismissed as too complex), but I am concerned about the complexity and performance impact of coding this. However, the issues that remain are no worse than they would be without the existing lightweight code on this branch (although they were perhaps less easily observed previously when cars moved randomly).

I will also need to look into recalibrating the visitor demand of sports stadia, as these seem to be out of proportion with other items in the game: but this will need some detailed data and computation.
In any event, thank you again all for your feedback: it is most helpful.

Offline Freahk

  • *
  • Posts: 577
  • Languages: DE, EN
Re: Lightweight private car routing
« Reply #78 on: February 05, 2020, 07:02:55 PM »
It is not clear whether this is an issue specific to this code or whether it existed before but was harder to detect.
I had previously observed busses "waiting for clearance" when the road was entirely free, but could not further investigate thus, so this might have existed before already.

However, the issues that remain are no worse than they would be without the existing lightweight code on this branch (although they were perhaps less easily observed previously when cars moved randomly).
This is absolutely true. The routing improves the game in any case. I had just listed any considerations of what (related) issues the routing solves and which will remain.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #79 on: February 07, 2020, 12:49:20 AM »
I have spent some time this evening investigating the issue with cars waiting unnecessarily at junctions. The code in this respect is extremely complex (and I did not write this myself, so I have no idea what was intended), and it is very difficult to trap the necessary behaviour in a debugger. However, I have managed to alter the behaviour somewhat by commenting out a part of the code. This seems to make private cars less cautious at junctions generally. I cannot find any significant adverse side effects to this, but I should be grateful if people could re-test in case there are any serious side effects of this that I have missed, and also check whether this in fact solves the problem.

Offline Phystam jp

  • *
  • Posts: 379
  • Pak256.Ex developer
    • Pak256 wiki page
  • Languages: JP, EN, EO
Re: Lightweight private car routing
« Reply #80 on: February 07, 2020, 01:35:20 AM »
Excellent, the heavy congestion on my map has gone after the modification. Probably THLeaderH, who is the developer of OTRP patch, knows about that. I will ask him.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #81 on: February 07, 2020, 02:04:09 AM »
I have now added multi-threading for the processing of the private car routes in order to improve performance. I should be grateful if people could re-test to see whether performance is improved and whether the feature still works correctly. Thank you.

Offline Phystam jp

  • *
  • Posts: 379
  • Pak256.Ex developer
    • Pak256 wiki page
  • Languages: JP, EN, EO
Re: Lightweight private car routing
« Reply #82 on: February 07, 2020, 02:46:32 AM »
First the performance was very good.
However, the FPS suddenly fell down to ~ 5 fps. the very low FPS kept until shutting down the game. And I observed very high CPU usage up to ~30 %.


I tried to reload the save data but failed, with a message:

Offline wlindley us

  • Devotee
  • *
  • Posts: 992
    • Hacking for fun and profit since 1977
  • Languages: EN, DE
Re: Lightweight private car routing
« Reply #83 on: February 07, 2020, 03:40:10 AM »
Multi-threading now does spread across cores and the game functions properly, but still with very long spells of extremely slow fast-forward per Phystam.

Offline THLeaderH jp

  • Coder/patcher
  • Devotee
  • *
  • Posts: 340
  • Languages: JP,EN
Re: Lightweight private car routing
« Reply #84 on: February 07, 2020, 03:47:41 PM »
Excellent, the heavy congestion on my map has gone after the modification. Probably THLeaderH, who is the developer of OTRP patch, knows about that. I will ask him.

The indicated code is written to prevent private cars from stacking in an intersection. When the intersection tiles are in a row, private cars should confirm that there is no car in the all intersection tiles. However, a private car knows coordinates of only two tiles ahead, so the code inspects only two successive tiles.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #85 on: February 07, 2020, 03:57:57 PM »
The indicated code is written to prevent private cars from stacking in an intersection. When the intersection tiles are in a row, private cars should confirm that there is no car in the all intersection tiles. However, a private car knows coordinates of only two tiles ahead, so the code inspects only two successive tiles.

Interesting, thank you. Commenting out the check for the second tile does appear to make traffic flow rather better and alleviates the unnecessary stoppage bug. If there be a way of fixing the latter without removing this, then this may be considered, but it is not clear at present that it is better to have this check than not.

Incidentally, as to the performance, I have pushed a further change to improve performance, but I am noticing some performance issues that are somewhat mysterious in that the information in the performance profiler does not seem to correspond in a way that I can understand to the observed performance in the game, the times of lower in-game performance corresponding to a considerably smaller proportion of CPU time being registered as being taken by the private car routing features than at times of apparently higher performance.

The saved game issue I will have to look into - thank you for the report.

Edit: I have now fixed the loading/saving issue, although this breaks compatibility with saved games from this branch (but not the master branch). This can be overcome in the debugger by stopping execution at line 9544 of simworld.cc and manually moving execution to line 9548 when loading an earlier saved game.

Edit 2: As to the performance, can I check whether this is apparently any worse than the earlier version without multi-threading?

Edit 3: Further testing shows that the multi-threaded route processing does appear to make performance worse - I suspect that the thread synchronisaiton overhead for the freelist mutex is the culprit. With this enabled, processing all the private car routes on the Pak256 testing map posted by Phystam above took circa 250ms. With this disabled, it fluctuated between circa 70ms and circa 180ms, but was usually circa 100ms. I have thus disabled this feature, but I have not at this juncture removed it: it can be re-enabled by defining the preprocessor symbol MULTI_THREAD_ROUTE_PROCESSING so that others can conduct more thorough comparative testing if desired. I should note that this change also stops route processing from being concurrent to the passenger generation algorithm, which may not be an efficient change; but that may need further consideration in time.
« Last Edit: February 07, 2020, 04:42:36 PM by jamespetts »

Offline Qayyum

  • *
  • Posts: 158
Re: Lightweight private car routing
« Reply #86 on: February 07, 2020, 05:53:26 PM »
The newsystem is like allELEMENTSin1BOX

Offline Phystam jp

  • *
  • Posts: 379
  • Pak256.Ex developer
    • Pak256 wiki page
  • Languages: JP, EN, EO
Re: Lightweight private car routing
« Reply #87 on: February 07, 2020, 06:51:13 PM »
Still I have a problem when loading the save data with the multi-threaded process...

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #88 on: February 07, 2020, 11:17:58 PM »
Phystam: does that only happen when you enable the multi-threading for route processing using the preprocessor directive?

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #89 on: February 08, 2020, 01:33:57 AM »
I have undertaken some more performance testing this evening with somewhat inconclusive results. The Bridgewater-Brunel saved game from December 2019 (game year 2003) seems to report similar fast forward speeds and framerates with multi-threaded route processing both on and off, although the fast forward speeds always seem to be slightly higher shortly after loading a saved game than otherwise. On the Stephenson-Seimens saved game (game year 1987), the framerate and fast forward speeds remain respectable with the single threaded route processing no matter how much time has elapsed.

With the Pak.256 saved game, I notice that this is an older saved game that contains records of routes to individual city buildings, the creation of which I prevented a few days ago. This may well affect performance by very significantly increasing hashtable sizes on the road tiles.

I have not been able to reproduce the error with the saved game data after I pushed the fix for this described above.

Qayyum - are you referring to all the destinations being stored in a single hashtable on each road tile? I did wonder about this, but I am not sure whether splitting them into different hashtalbes is likely to help, nor into what different hashtables to split them.

It would be very helpful if anyone could attempt comparisons with and without the multi-threaded route processing. Also, if anyone can produce a reliable reproduction case for the reported saved game issue (with loading and saving in the latest commit on this branch), that would be very helpful.

Thank you again all for testing - this has been most useful.

Offline Qayyum

  • *
  • Posts: 158
Re: Lightweight private car routing
« Reply #90 on: February 08, 2020, 04:46:30 AM »
Jamespetts - I thought I am thinking, when me see the new system, as comprising of a list of destinatinns from where to where, ALL in one SINGLELIST. For two-way rpads, two entries of destination, and one-way road, one entry of destination. Entry of destination means entry of destinations. I would like to keep the routing of private car operation in one thread, akin to one person being the leader of public road system

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #91 on: February 08, 2020, 11:19:22 AM »
Jamespetts - I thought I am thinking, when me see the new system, as comprising of a list of destinatinns from where to where, ALL in one SINGLELIST. For two-way rpads, two entries of destination, and one-way road, one entry of destination. Entry of destination means entry of destinations. I would like to keep the routing of private car operation in one thread, akin to one person being the leader of public road system

Is your suggestion to have one hashtable per direction? It is difficult to see how this would help; road tiles have a maximum of four directions (for a crossroads), so private cars would have to check each one.

As to putting private car routing operations in a single thread, again, it is difficult to see how this would help; there is very little that actually processing the routes can be concurrent with. Unlike finding the routes, processing the routes cannot be concurrent with sync_step.

Offline Phystam jp

  • *
  • Posts: 379
  • Pak256.Ex developer
    • Pak256 wiki page
  • Languages: JP, EN, EO
Re: Lightweight private car routing
« Reply #92 on: February 08, 2020, 11:49:33 AM »
I could reload the deta which is saved in single-threaded version.
I show the compatibility table between the single- and multi-threaded versions.

save -> load
single -> single: OK
single -> multi: OK
multi -> single: failed
multi -> multi: failed

This means something wrong in the multi-threadead version when saving the game data.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #93 on: February 08, 2020, 11:53:18 AM »
Splendid, that is helpful - thank you.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #94 on: February 08, 2020, 01:05:12 PM »
More research has found the immediate, but not the ultimate, cause of this saved game corruption. What appears to happen is that, somehow, the individual threads get out of sync with the barrier waits and multiple different routes end up being processed at once. This, in turn, causes race conditions at individual road tiles, causing corrupt hashtables, which in turn corrupt the save data when the game is saved.

It is not clear whether it is worth fixing this, since testing has not shown any clear performance advantage to this, since adding data to the hashtables, the most CPU-intensive task in the route processing, is in any event forced to be effectively single threaded by the freelist mutex.

I have now separated the preprocessor definition for the main multi-threading of the processing with the preprocessor definition (#defined in the code) used for making the route processing run concurrently with the passenger generation code, which might well allow this to run slightly more efficiently than running entirely singly at least.

I think that this feature is now ready for inclusion - if anyone thinks that there are any major problems to resolve before I include it, please let me know before this evening. Thank you again for all your testing and feedback. I shall look forward to seeing bypasses and motorways in all the online games!

Offline Phystam jp

  • *
  • Posts: 379
  • Pak256.Ex developer
    • Pak256 wiki page
  • Languages: JP, EN, EO
Re: Lightweight private car routing
« Reply #95 on: February 08, 2020, 02:45:18 PM »
I tested the latest version for a short time. The result is very well. It always keeps ~30 fps, and the CPU usage is less than 7 percents. Reloading the multi-threaded save data works correctly, too. QUITE stable!!!

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #96 on: February 08, 2020, 03:51:49 PM »
Thank you very much for testing. I have pushed another performance improvement - instead of using multi-threading, this limits the number of routes in every city processed per step to 8 (by default: this can be changed in simuconf.tab), spreading the workload between steps.

Hopefully, this will address the performance issue reported by Matthew. I should be grateful if people could re-test.

Offline Matthew gb

  • *
  • Posts: 316
    • Japan Railway Journal
  • Languages: EN, some ZH, DE & SQ
Re: Lightweight private car routing
« Reply #97 on: February 08, 2020, 05:58:13 PM »
Splendid, thank you: I have now had a chance to look at this. A performance profiler suggests that ~8% of CPU time is being spent on processing private car routes in this map, of which ~4% is on clearing existing private car routes before processing

James, thank you for your continued work on this, especially since so much of it is for the benefit of low-spec gamers like me. It's frustrating that multi-threading has turned out to be a dead end. However, that effort wasn't totally in vain: it has taught me the importance of performance profilers to make sure I'm not just imagining a problem. I have spent several hours learning to use a performance profiler on Linux (including dead ends of my own - there is a severe lack of comprehensible tutorials) so hopefully I can give more helpful replies in future.

I have tested the latest patch (commit #035705f; 1960; pak128.Britain-Ex) and it is like night and day compared to some earlier versions of this branch! Frameskips have completely disappeared and I get a steady 30fps for the first 10 mins of a game, at least. However, there is an oddity:



I don't seem to be getting any passengers at all....? No visitors at all for sports stadia seems particularly odd. Perhaps this will clear itself as I play on though.

Quote
I should be interested to hear from other people as to the performance with Matthew's 1960 era map; this would be most useful in determining the significance of this issue. I also notice that Matthew's computer is rather limited in RAM - this saved game alone when running takes ~1.2Gb of RAM, which is a significant proportion of 4GB. If Matthew were running low on RAM, this would potentially impact performance significantly.

One other thing to note is that one can set "assume_everywhere_connected_by_road" to disable private car route searching entirely, which will reduce any performance impact.

Though it was a good guess, I don't think I am memory-limited, as this top report shows:



Sim-Ex is only using 1.5 GB and even my biggest games are <2GB so far. I've never been able to see Bridgewater-Brunel because of its size, but I think the issue with this branch is was CPU, as far as I can tell.
« Last Edit: February 08, 2020, 10:58:00 PM by Matthew »

Offline Phystam jp

  • *
  • Posts: 379
  • Pak256.Ex developer
    • Pak256 wiki page
  • Languages: JP, EN, EO
Re: Lightweight private car routing
« Reply #98 on: February 08, 2020, 07:15:31 PM »
I found very wierd behavior:
https://cdn.discordapp.com/attachments/665986412641779712/675780704067649569/Simutrans_120.2.1_Extended_Nightly_development_build_14.9_035705f_2020-02-09_04-08-42.mp4
This issue happened at the edge of the city. Probably it is similar behavior as I have already reported about oneway motorway. And, as you can see in the video, this may finally causes a small dead lock.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #99 on: February 08, 2020, 07:56:22 PM »
Thank you both for your feedback: that is very helpful. Matthew - I cannot reproduce any reduction in passenger traffic. When I run the saved game from the Stephenson-Seimens server, Freahk's company's passenger numbers increase.

Phystam - it is likely to be difficult to debug that issue without a reproduction case. However, this is probably marginal enough not to prevent it from being worthwhile merging this now, so this will be in to-morrow's nightly build.

Thank you to everyone who has tested this so far: this is most helpful.

Offline Matthew gb

  • *
  • Posts: 316
    • Japan Railway Journal
  • Languages: EN, some ZH, DE & SQ
Re: Lightweight private car routing
« Reply #100 on: February 09, 2020, 08:00:59 AM »
EDIT: As the private-car-routing is now included in the nightly, I have tried the Bridgewater-Brunel executable and confirmed that I've somehow failed to compile the game properly  :-[ :-[ :-[ :-[ Apologies for the unintentionally unhelpful report about passenger generation.

Thank you both for your feedback: that is very helpful. Matthew - I cannot reproduce any reduction in passenger traffic. When I run the saved game from the Stephenson-Seimens server, Freahk's company's passenger numbers increase.

I can't access the Stephenson-Siemens game due to the ongoing pakset mismatch issue. But I've tested this for another couple of hours and I still get very odd passenger generation in this branch. When I generate new maps, I get no passengers at all. On old maps (from master), passenger numbers fall off a cliff:



I think it's fairly obvious when I switched the game to the this branch.  :D

Almost all omnibus lines are at or just above 0% occupancy. There are a few passengers appearing at stops, but I wonder how many of them have walked from other stops as part of a longer route, rather than starting a new journey. I can't explain why this is happening though. perf shows that generate_passengers_or_mail is running.

But it's very, very possible



(though the source code appears to be correct).
« Last Edit: February 09, 2020, 08:33:55 AM by Matthew »

Offline Freahk

  • *
  • Posts: 577
  • Languages: DE, EN
Re: Lightweight private car routing
« Reply #101 on: February 09, 2020, 10:45:54 AM »
I can't access the Stephenson-Siemens game due to the ongoing pakset mismatch issue.
Indeed an ongoing issue :/
You can access it loading the "save" net:list.extended.simutrans.org

Offline killwater

  • *
  • Posts: 222
Re: Lightweight private car routing
« Reply #102 on: February 18, 2020, 09:32:05 PM »
First of all I wanted to say that this is a truly epic improvement. Thank you James and everyonee who contributed.
I have a few questions:
1. Do cars use attractions and industry outside cities as spawning point and destination?
2. If they do how far can the road be from the attraction/industry?
3. Do you need a stop for the attraction/industry to generate and accept private cars?
Also I noticed that cars do not want to route through wooden bridges over deep water.

Offline jamespetts gb

  • Simutrans-Extended project coordinator
  • Moderator
  • *
  • Posts: 19273
  • Cake baker
    • Bridgewater-Brunel
  • Languages: EN
Re: Lightweight private car routing
« Reply #103 on: Yesterday at 01:57:31 AM »
First of all I wanted to say that this is a truly epic improvement. Thank you James and everyonee who contributed.
I have a few questions:
1. Do cars use attractions and industry outside cities as spawning point and destination?

Private cars certainly use these as destinations. They should also use them as spawn points for the return journies, but I have not seen this happening (the code for the generation of private cars has not changed - only the code for where they go once they have been generated).
Quote
2. If they do how far can the road be from the attraction/industry?

The road needs to be immediately adjacent to the spawn tile, which, if I recall correctly, is the North-West corner.
Quote
3. Do you need a stop for the attraction/industry to generate and accept private cars?

No - private cars are entirely independent of player stops.

Quote
Also I noticed that cars do not want to route through wooden bridges over deep water.

On the face of it, this is odd as there is no means of testing when a car is about to drive over the bridge whether it is over deep water or not. If you are able to test and confirm this (and specifically that it is confined (1) to wooden bridges; and (2) that are over deep water), then I should be grateful if you could post a full bug report with a reproduction case.