Script API: return planned way routes to scripts

[victor_18993]

Following Andarix's suggestion, I am opening a separate thread for the technical discussion about exposing the internal way planner to scripts.
The discussion started here:
https://forum.simutrans.com/index.php/topic,23967.0.html
In that thread, Prissi suggested separating route planning from construction: first let the script request a route and receive the materialized result, and only later consider a second API that could build from a route, possibly after the script has inspected or modified it.
I investigated that direction and prepared a first cut limited strictly to planning.
The proposed API is:

Code Select Expand

way_planner_x.find_route(pl, start, end, way [, straight, keep_city_roads, bridge, tunnel])It returns an

Code Select Expand

array<coord3d> from start to destination.
The

Code Select Expand

way_builder_t only exists during the call; the returned route is copied into independent script data. Planning does not build anything, charge money, terraform or reserve anything.
For this first cut, coordinates appear sufficient to represent the planner result. Bridges and tunnels are not stored as separate objects in the internal route either: they are represented by gaps between route coordinates together with the bridge/tunnel descriptors supplied as input.
I would not treat that as deciding the contract for a future

Code Select Expand

build(route), though. That second step may need the descriptors again, additional metadata, or a richer route representation. I would rather design that separately before assuming the planning representation is also sufficient for construction.
The API uses the same internal route planner, although tool-specific preprocessing can still make the result differ from

Code Select Expand

command_x.build_way in some situations.
Current validation:

• 8 new tests
• 245/245 tests passing
• no new compiler warnings attributable to the patch
• invalid API arguments raise a Squirrel error
• a valid request with no route returns an empty array
• repeated calls return independent results
• planning does not modify the world or player finances

I have attached the current diff for review.
Before going further with a possible second cut, I would like to confirm whether this minimal route representation looks like a reasonable basis for the script planning API.

[prissi]

I think the prefer parallel flag may be also a good addition for the route planning of a second track.

[Andarix]

Tunnels cannot currently be built using script AI. According to Dwachs, this is because field testing doesn't work underground.

This means that if the route includes a tunnel, it cannot be built and the route is incomplete.

[victor_18993]

I looked into the

Code Select Expand

prefer_parallel suggestion, and it turned out to need less than I expected: the flag already exists.

Code Select Expand

way_builder_t already has

Code Select Expand

prefer_parallel, and

Code Select Expand

tool_build_way_t::calc_route() derives

Code Select Expand

assume_parallel from the tiles around the two clicks and passes it through

Code Select Expand

set_prefer_parallel(). The planning API simply never set it, so it always planned with

Code Select Expand

false.
That shows up immediately in the case you mentioned. With a road already running from

Code Select Expand

(2,5) to

Code Select Expand

(13,5), asking for a route from

Code Select Expand

(2,6) to

Code Select Expand

(13,6) returned:

Code Select Expand

(2,6) (3,6) (4,6) (4,5) (5,5) ... (12,5) (12,6) (13,6)So it climbs onto the existing road for nine tiles instead of staying beside it, because an existing way costs

Code Select Expand

way_count_straight while empty ground costs

Code Select Expand

way_count_no_way.
With

Code Select Expand

prefer_parallel enabled, it returns all twelve tiles of row 6, which is also what the normal way tool builds when dragging between those same two tiles. I get the same behaviour with rail.
So the API now only exposes the existing flag:

Code Select Expand

way_planner_x.find_route(pl, start, end, way [, straight, keep_city_roads, bridge, tunnel, prefer_parallel])It defaults to

Code Select Expand

false and only calls

Code Select Expand

bob.set_prefer_parallel(). There is no second heuristic: the way tool infers the flag from the surroundings, while a script can state it explicitly because it knows whether it is trying to build a second parallel track.
Omitting the argument preserves the previous planning behaviour exactly; one of the two new tests checks that explicitly.
The suite is now 247/247, with no new compiler warnings from the change.
There is one API detail I would rather leave to you. I currently put

Code Select Expand

prefer_parallel at the end so the signature I already posted remains unchanged. That means a script has to pass

Code Select Expand

null for

Code Select Expand

bridge and

Code Select Expand

tunnel just to reach it.
Conceptually, it probably belongs next to

Code Select Expand

straight and

Code Select Expand

keep_city_roads, and since none of this is integrated yet we can still move it without breaking anything.
Would you prefer it there, or should I keep it at the end?

[victor_18993]

Tunnels cannot currently be built using script AI. According to Dwachs, this is because field testing doesn't work underground.
This means that if the route includes a tunnel, it cannot be built and the route is incomplete.

About the tunnels: I tried to reproduce it and I think I found something, but not quite what I
expected, so I would rather check with you before going further.

Scripts can build tunnels through the tunnel tool. tests/tests/test_way_tunnel.nut in trunk does
it, both as a single click on the slope and as two clicks between two underground coordinates, and
I reproduced that from scratch on r12134.

What I did find is that command_x.build_way refuses interior tunnel ground: any coordinate whose
ground is a tunnelboden and is not the map ground. On a three tile tunnel the two portals are
accepted and the tile in between is not, so a call fails whether that tile is the start or the
end. The refusal also carries no text, it arrives as an empty string, which makes it look like
nothing happened at all. Meanwhile find_route() plans straight through those same tiles, so a
planned route can contain coordinates that command_x.build_way will not take.

Is that your case? To be sure I am chasing the same thing, could you tell me:

  - which API you call (command_x.build_way, the tunnel tool, or something else)
  - the start and end coordinates you pass
  - whether either of them is underground
  - whether you get an empty string back or some other error

I have not changed anything yet. I would rather fix what actually stops you than what I guessed.

[Andarix]

There are 3 Script APIs.

Script AI - tunnels not allowed
Script Scenarios - tunnels allowed and working (autostep in tutorial)
Script Tools - not work

The script tool I wrote to upgrade tunnels isn't working.

Code Select Expand

            local tool = command_x(tool_build_tunnel)
            err = tool.work(pl, wayline[i-1], wayline[i], way_new)


[victor_18993]

Andarix, thank you for the reproducer, that was exactly what was needed. Your tool builds
tunnels once one line is changed, and the cause is not the script tool context.

In upgrade_tunnel/tool.nut:

Code Select Expand

way_new = find_object(pl, "tunnel", wt, speed)  // returns a tunnel_desc_x object
err = tool.work(pl, wayline[i-1], wayline[i], way_new)

The fourth argument of command_x::work is the tool's default_param, that is a string, but
find_object() hands you a descriptor object. Pass way_new.get_name() instead and it works.
The same tool is already used that way in the trunk, script/hm_lib/hm_tunnel_tl.nut passes
desc_name.

It is not a script tool problem: it fails identically in a scenario. I reproduced it in the
automated test scenario with the same map, the same player and the same tool, changing only
the type of the fourth argument:

  work(pl, start, end, desc.get_name())  ->  null, tunnel built
  work(pl, start, end, desc)            ->  throws "Error during initializing tool"

Two things made that hard to see, and I would like to fix the first one.

1) The type error is raised and then dropped. param<const char*>::get() calls sq_raise_error()
and returns NULL, but command_work() does not look at the result, so default_param silently
becomes NULL. tunnel_builder_t::get_desc(NULL) returns NULL, tool_build_tunnel_t::init()
returns false, and what the script finally sees is "Error during initializing tool", which
does not mention the parameter at all.

2) It throws instead of returning, so the "if (err != null)" branch never runs and nothing is
printed. From the outside the tool simply does nothing.

The attached patch is five lines in api_command.cc plus two tests. A non-string tool parameter
is now rejected with a message that names the actual problem.

Limits, stated up front:

- The check is only for calling convention (3), work(pl, pos, pos2, param). Convention (2)
  cannot be checked the same way, because work(pl, pos, pos2) without a parameter is in use:
  test_halt_move_stop_invalid_param does exactly that, and checking there breaks it.
- An explicit null is still accepted, as before.
- The regression test runs as a scenario, since the test suite is a scenario. There is no test
  harness for script tools at all, which is why this gap was never covered.

On the underground tiles I asked about in my previous post: that problem is real, but it is not
yours. tool_build_tunnel does accept interior tunnel tiles. Measured on a three tile road
tunnel, the two portals are is_ground() while the middle tile is is_tunnel() but not
is_ground(), and upgrading RoadTunnel (80 km/h) to FastRoadTunnel (130 km/h) tile by tile
across that middle tile works. That is the second test in the patch. The refusal I described
applies to command_x.build_way only, the two do not share the validation.

Script AI is not blocked from building tunnels either. export_commands() is not gated by script
type, so command_x is the same in scenarios, in AI and in script tools, and no tool is on a
white or black list. The only difference is that a scenario runs with PLAYER_UNOWNED and
therefore gets WFL_NO_CHK, which is only consulted for scripted scenario rules and for the
network password check. I checked by giving the scenario harness the script tool player
binding: all tunnel tests still pass, and only test_scenario_rules_allow_forbid_way_tool_rect
fails, which is exactly what that flag governs.

Tested with pak64 on a headless gcc/MinGW build on Windows: 239/239, that is the 237 before
plus the two new ones. The patch applies to a clean checkout of r12138 without fuzz, builds
from scratch and adds no compiler warning in the file it touches. I have not built it with MSVC.

If you would rather not have the extra check in command_work, say so and I will drop it. The
script side is a one line change either way.

[Andarix]

works

thanks

[prissi]

Then lets incorporate it.

[victor_18993]

The patch has now been incorporated into trunk as r12139.
Thanks for testing and confirming the issue.