The pak size only tells you how large a tile is, not how large objects on that tile should be. That becomes especially visible with pak64.H0, which has trains the same size as pak192.comic. (H0 original thread:
https://forum.simutrans.com/index.php?topic=6428.0)
If your vehicles have the same size as they do in any 128 pak, then there is nothing wrong with using the same scale for buildings as well. If your vehicles are twice* as big, then it makes sense that buildings become twice as big as well. However, in almost all paksets the scale of buildings to vehicles is skewed anyway, since both are supposed to fit on a tile neatly. If you were to scale up vehicles (that are already oversized) and scale down buildings (which are typically undersized), nobody will look at it and think of it as a "real" cityscape anymore. Doesn't mean it's bad, just stylized - compare to the original settlers 2 with oversized people in comparison to their houses.

Next question is: Are you just gonna take graphics from other paksets? COULD you scale them up and still have them look good? It's a style question as well. If you look at the japanese 256 pak, their graphics are so crisp that an upscale will look out of place. But if you go for a foggy, less detailed style it might work for you. Depends on whether you want to go with a foggy Chicago or clear sunshine in Arizona.
If you are creating your own graphics anyway, even if you don't consider scale, couldn't you make a clay pit as larger or small as you want anyway, since there are clay pits of different sizes? As such, it wouldn't matter how large they are in other paksets at all.
My suggestion: If you have created any buildings at all so far, check their size, and make new buildings scaled accordingly / scale foreign buildings to fit YOUR scale. If you have nothing yet,
do the same scale as pak256.Japan. That way, it can become a standard for future pak256 paksets, such that they can all be mixed and matched by players.
*twice = onedimensional doubling. As in, if it was 1x1 before, it's 2x2 then, even though thats four times the area.