There is definitely a bug with passengers staying on a vehicle. This can be exploited to amusing effect as this save game shows:
http://simutrans-germany.com/files/upload/rail_loop.sve(pack britain exp 0.84)
Linking to stops within a station coverage of 3 of significant passenger generators seems to stop this, although I think this includes all buildings, not just factories as I'd thought before. It seems to include tourist attractions - in one town I build a horse carriage route with stuck passengers, and I can drain this by connecting the nearby church and cathedral, after which it builds up again. I then connect the nearby industries and they drain away, and stays drained, and then I disconnect the church and cathedral and it builds up again.
I haven't been able to get passenger build ups on intercity routes, and suspect it occurs when a passenger packet gets to its final destination, is not within 3 squares of the destination, but has a stop available that is within the 16 square range of its final destination and switches to this stop. And then when it gets to the new stop it switches again with no limit.