Modernisation of the Squirrel Script API

[victor_18993]

Simutrans already exposes a substantial part of the engine through the Squirrel Script API, but while working with it I have found some places where information already available in the C++ engine is either lost or only partially exposed to scripts.
Rather than redesigning the Script API, I would like to use this thread for small, incremental and backwards-compatible improvements where such gaps can be clearly demonstrated.
First change:

Code Select Expand

way_planner_x.get_step_cost()

Code Select Expand

way_builder_t::is_allowed_step() already does two things internally:

• determines whether a step is allowed;
• calculates the pathfinding weight for that step.

However, the existing

Code Select Expand

way_planner_x.is_allowed_step() binding only exposes the boolean result and discards the calculated weight.
As a result,

Code Select Expand

sqai and

Code Select Expand

sqai_rail currently have to estimate these weights again in

Code Select Expand

astar.nut.
This patch adds:

Code Select Expand

way_planner_x.get_step_cost(from, to)It returns:

• an integer containing the pathfinding weight already calculated by
  
  Code Select Expand
  way_builder_t;
• Code Select Expand
  null when the step is rejected or cannot be resolved.

This is a pathfinding weight, not a construction price.
The change is completely additive.

Code Select Expand

is_allowed_step() is unchanged, existing scripts do not need any modification, and this patch does not modify

Code Select Expand

sqai,

Code Select Expand

sqai_rail or

Code Select Expand

astar.nut.
Five new

Code Select Expand

way_planner_x tests have been added, covering:

• untouched terrain versus an existing way;
• slopes;
• rejected steps returning
  
  Code Select Expand
  null;
• unusual/invalid inputs;
• consistency between
  
  Code Select Expand
  get_step_cost() and
  
  Code Select Expand
  is_allowed_step().

Test results:

• 5/5 new tests passed
• previous suite: 238/238
• total: 243/243
• Code Select Expand
  none and SDL2 builds successful
• no new compiler warnings

Patch attached:

Code Select Expand

squirrel-api-01-step-cost.diffI would like to keep any further changes in this area similarly small and independent, so each improvement can be reviewed on its own merits.

[victor_18993]

The descriptor initialization fix is now in trunk as r12144.

Code Select Expand

way_builder_t now initializes

Code Select Expand

desc,

Code Select Expand

bridge_desc and

Code Select Expand

tunnel_desc to

Code Select Expand

NULL, so an unconfigured

Code Select Expand

way_planner_x now behaves deterministically instead of depending on uninitialized memory.
For an unconfigured planner:

• Code Select Expand
  is_allowed_step() returns
  
  Code Select Expand
  false
• Code Select Expand
  get_step_cost() returns
  
  Code Select Expand
  null

Configured builders are unchanged.
A regression test was added for this case.
Final validation:

• 244/244 tests passed
• Code Select Expand
  none and SDL2 builds successful
• no new compiler warnings

This fix is now complete in trunk.

[victor_18993]

The tram/elevated planner fix is now in trunk as r12145.

Code Select Expand

way_planner_x.set_build_types() now preserves the same tram and elevated way-type semantics as the normal way tool:

• tram descriptors use
  
  Code Select Expand
  schiene_tram;
• elevated descriptors use
  
  Code Select Expand
  elevated_flag;
• air ways and runways remain non-elevated.

This does not change the Script API signature, and existing road and rail scripts are unaffected.
Final validation:

• 247/247 tests passed, including three new
  
  Code Select Expand
  way_planner_x regression tests
• Code Select Expand
  none and SDL2 builds successful
• no new compiler warnings

This change is now complete in trunk.

[victor_18993]

The tunnel planner addition is now in trunk as r12146.
A new

Code Select Expand

tunnel_planner_x.find_end() method is now available to Squirrel scripts. It exposes the existing native tunnel endpoint search without building anything.

• returns the far tunnel portal when one can be found;
• returns an invalid coordinate on failure;
• accepts only a single direction;
• uses the same unrestricted search length as the native tunnel endpoint search;
• does not expose
  
  Code Select Expand
  full_tunnel or the currently incomplete tunnel error messages.

This is an additive Script API change. It does not modify the existing tunnel tool or the AI implementations.
Final validation:

• 252/252 tests passed, including 5 new tunnel planner regression tests;
• endpoint parity with the normal tunnel tool tested;
• rotated-map behaviour tested;
• Code Select Expand
  none and SDL2 builds successful;
• no new compiler warnings.

This gives scripts a native way to query a tunnel endpoint before deciding whether to build it.

[victor_18993]

API-04A is now in trunk as r12149.

Code Select Expand

command_x.build_bridge_at() now accepts an optional

Code Select Expand

max_length parameter:

Code Select Expand

command_x.build_bridge_at(pl, pos, bridge_desc [, max_length])

The existing 3-argument behaviour is unchanged and still defaults to a maximum search length of 10.

• Code Select Expand
  max_length > 0 uses the supplied limit;
• Code Select Expand
  max_length <= 0 searches as far as the bridge descriptor and
  
  Code Select Expand
  way_max_bridge_len allow;
• Code Select Expand
  min_length remains internal;
• flat ends are not enabled by this change.

The Script API search is capped at 254 tiles because the native bridge endpoint search uses an 8-bit counter internally.
Final validation:

• 256/256 tests passed;
• 4 new API-04A regression tests;
• 17/17 tests in
  
  Code Select Expand
  test_way_bridge.nut;
• 27/27 across the bridge test family;
• API-03 tunnel and way planner regressions still pass;
• headless and SDL2 builds successful;
• no new compiler warnings.

This is an additive API change. Existing scripts using the 3-argument form retain the previous behaviour.

[victor_18993]

EXEC-05 has now been integrated in r12150.

Code Select Expand

command_x.build_way() and

Code Select Expand

command_x.build_road() now accept an optional trailing

Code Select Expand

terraform argument:

command_x.build_way(pl, start, end, way, straight [, terraform])
command_x.build_road(pl, start, end, way, straight, keep_city_roads [, terraform])

The default remains

Code Select Expand

false, so existing scripts keep exactly the previous behaviour.
When

Code Select Expand

terraform=true, the command may use the native way-builder slope adjustment while constructing the way. This does not enable automatic bridges or tunnels.
The change also preserves the terraform policy through network tool execution, so the same command semantics are used on the authoritative side in multiplayer.
Regression coverage was added for the legacy/default behaviour, slope crossing,

Code Select Expand

build_road, adjacent-step construction, scenario restrictions, no automatic bridge construction, and tool-state reuse.
Full automated suite after the change: 264/264 passed.
This completes the executor side of the terraforming work. The next step will be the corresponding

Code Select Expand

way_planner_x change, so scripts can plan and execute using the same policy.

[victor_18993]

API-05 planner terraforming has now been integrated in r12151.

Code Select Expand

way_planner_x.set_build_types() now accepts an optional trailing

Code Select Expand

terraform parameter:

set_build_types(way [, terraform])

The default remains

Code Select Expand

false, so existing scripts keep the previous behaviour unchanged.
When enabled, the planner considers the same slope-changing capability exposed to

Code Select Expand

command_x.build_way() /

Code Select Expand

build_road() in r12150. This means scripts and AIs can now plan across certain slopes that the native way builder can handle through terraforming, instead of discovering that capability only when attempting construction.
The existing way-type composition is preserved, including tram and elevated ways, and enabling terraforming does not enable bridges or tunnels.
The planner remains a planning/query interface: an accepted step is not a guarantee that a later construction command will succeed, since the executor may still reject the operation for other reasons.
Regression coverage was added for:

• legacy behaviour with the parameter omitted or
  
  Code Select Expand
  false;
• slope cases requiring terraforming;
• planner/executor parity on the same geometry;
• ensuring terraforming does not bypass unrelated restrictions;
• non-mutating planner queries;
• clearing the flag when the planner is reconfigured;
• preserving tram/elevated way types.

The complete automated suite passes 271/271.
This closes the planner/executor terraforming work. The next Script API work will focus on auditing how failures and error results are currently exposed to scripts before proposing any further API changes.

[Andarix]

I believe the function 'command_x::build_bridge_at()' is no longer usable in its current form, since Simutrans no longer builds bridges with just a single click.

Are you also incorporating your changes into the Doxygen documentation?

[victor_18993]

Yes, I understand what you mean now.

build_bridge_at() still works by taking a starting tile, finding a suitable end and then building the bridge, but your point is whether that is still the right API for a Script AI now that bridge construction is naturally defined by two endpoints.

We already have bridge_planner_x.find_end() and command_x.build_bridge(start, end, ...), so I will review this part specifically and check which interface should be considered the proper one for scripts rather than assuming that the old one-click abstraction is still the best choice.

And yes, I am updating the Doxygen documentation together with the API changes. I want the final API behaviour and its documentation to stay in sync.

[victor_18993]

A small update on the Script API work.
Two more changes have now been integrated into trunk:

• r12152 clarifies the Doxygen documentation for
  
  Code Select Expand
  command_x return values and the planner/executor boundary. In particular, tool calls return
  
  Code Select Expand
  null on success and a string on failure, including the possibility of an empty string when no useful message is available.
• r12153 fixes
  
  Code Select Expand
  get_tool_key(), which was returning Ctrl and Shift prefixes reversed for some keys. This was reproduced at runtime before changing the code and then verified again after the fix.

I am now looking at the next part from the point of view of actual Script AI usage, especially the terrain/route-building problems mentioned by Andarix.
The terraform support added in r12150/r12151 gives scripts the ability to ask the planner to consider slope changes and to let the executor perform them, but the current Script AIs do not use those new arguments yet.
One interesting finding from the current audit is that the C++

Code Select Expand

way_builder_t already has a fairly clean separation between calculating a route, calculating its monetary cost and actually building it. The native AI already uses this to compare alternatives before changing the map.
Script AI, however, currently has much less information available before construction. It can ask whether individual steps are possible, but it cannot yet inspect a complete engine-calculated route together with its real construction cost in the same way.
I am still investigating this before proposing any new API. I want to distinguish carefully between limitations of the Script API and things that are simply AI strategy problems, rather than adding functions that may not actually be needed.
I am also checking the existing automatic bridge/tunnel behaviour of the way tool, since there may already be useful engine functionality available through a less obvious path.
As before, the idea is to keep the changes small and incremental: first understand and reproduce the limitation, then decide whether an API change is actually justified.