As usual, I find that, in these situations, the most sensible thing to do is to consider how it would work in reality. That, after all, is what players are implicitly given to expect will happen from the very fact that this is a simulation game. Anything that departs from reality significantly and in a non-obvious way is likely to be confusing.
Looking at the reality of things, with passengers, they generally have a limited tolerance and will only wait so long for a train before considering abandoning their journey, or looking for other modes of transport. Passengers will tend to avoid stations that are known to be very overcrowded, especially if there is a possibility that they will not be able to fit on a train for quite some time. Although passengers cannot predict how crowded that stations down a line are on an individual journey, if a station becomes notoriously overcrowded, the same will apply.
So, a sensible solution for passengers is to have a quality of service metric based on average waiting times per line per station, and average crowding levels per station. Passengers would choose, where there is a choice, a route that had origin and transfer stops with higher, rather than lower, quality of service metrices. If a station ever became excessively crowded, such that no more passengers could fit into it at all, then newly arriving passengers should effectively be discarded en route (abandoning their journey or continuing it by taxi). Furthermore, if passengers have to wait too long at any station, they should also abandon the journey. Passengers abandoning a journey should record themselves as "unhappy" at the station at which they so abandon.
Mail and goods are different: they are not sentient and do not therefore have a tolerance level. Some goods are more urgent than others, so, if, for example, mail is not picked up for some time, the people in charge of the postal system will become concerned and potentially withdraw it. Likewise, food will perish if not collected before a certain time. So, as with passengers, certain goods, if they have to wait too long at a station, should be discarded, and lost to the player. The situation is different for, say, coal or iron ore, the delivery of which is not urgent, per se, but, even then, suppliers will not want to have their valuable product sitting unused in a station's store for ever. Eventually, that, too, would be withdrawn and private transport arranged. So, all goods should have an expiry time: a wait time (in months) beyond which they will abandon the journey. That could be specified in simuconf.tab for each different type of transported item.
Similarly, if goods arrive in a station that is already too full, there is nowhere to put them. For a while, they can be piled up in the car park or in the stationmaster's office, but there is a limit even to that. In reality, operators would know that a certain station is becoming overcrowded, and would avoid routing via it (remember: with goods, the transport company decides the route which they take; passengers decide their own route). However, there might be no alternatives. It would make sense for the transport company to hold back the goods to prevent them from being lost at the transfer; but, after a while, they would be withdrawn in any event.
With all that in mind, here are some summarised suggestions:
(1) each type of goods/passengers should have its own maximum waiting time at each transfer, beyond which the journey will be abandoned;
(2) there should be a quality of service rating per station per line, which determines relative routing preferences;
(3) goods/passengers unloaded at very overcrowded stations should be discarded; and
(4) if a vehicle comes and the next transfer is overcrowded, don't load the packet unless it has been waiting at the station for more than, say, two thirds of its maximum wait time (passengers get desperate, and take the next train that will take them to their destination somehow or other; goods get routed to avoid the possibility of abandonment).
Hopefully, the above set of solutions will add realism and interest to the gameplay and avoid deadlocks all at the same time, whilst maintaining a realistic response to the issue of station overcrowding: players should be penalised for it to some extent, but not in the form of irresolvable deadlocks. The capacity of a station should also be somewhat elastic, such that some goods can unload when it is overcrowded, but only a few.