Forum attachments discussion

[IgorEliezer]

Just some thoughts...

- We could set some permissions in certain boards of forum. In some boards, attachments would be allowed for any member. In other boards (if needed so), for some membergroups. And in other boards else, attachments would be disallowed.

- If I need a help or reporting a bug, I don't need to keep images and sve attached in forum forever. I could simply host such files in an online image/file storage (e.g. www.4shared.com and www.imageshack.us), I copy the link and paste in post, and anyone is able to see or click on link to download. Such stored files will be deleted by these services after ~1 year of inactivity.

- I fear of self-deletion of old attachments because of pak files.

[vilvoh]

We could set some permissions in certain boards of forum...

I support it but I don't know if SMF is complet enough to have such an option like that, is it?

If I need a help or reporting a bug, I don't need to keep images and sve attached in forum forever...

I think sve files from bug reports are not downloaded too many times. We can avoid that deleting the bug report topic once it has been solved, attached files included. In order to prevent duplicates, we could create a log which includes all bug reports that have been solved. This way you safe disk space and you make the search easier,as you just have to do a little search in the log.

[Isaac Eiland-Hall]

There are two relevant permissions for each membergroup:
- View attachments
- Post attachments

The main issue is that once you make each board use custom permissions, it becomes a pain to make any changes, as you have to make those changes for each board... Granted that once it's set up, it should be rare to change, in practice, this isn't always true.

It's certainly doable, but I'm not sure it's worth our time. It'll be hard enough to remember to remove old attachments every few weeks as it is - if we do that (I thought that was the consensus plan upon moving to this new forum - essentially no guaranteed attachments, i.e. they can be removed at any time...)

[vilvoh]

Uhm, I see. In that case, I would only allow txt, zip, rar, tab, dat, diff, patch, sve file types and I would leave gif, jpg and png files hosted in external sites.

[Colin]

Uhm, I see. In that case, I would only allow txt, zip, rar, tab, dat, diff, patch, sve file types and I would leave gif, jpg and png files hosted in external sites.

I agree with this. I still think a time should be set for files to remain on the forum. The old forum was bloated with files dating back to 2005. I know Isaac's server could handle the load, but it's a bit silly really. Maybe they could be archived off and stored off line, to be accessed only if someone has a specific question that needs to be referenced back to old files.

Actually I fail to see why anyone, (I know I'll be shot down in flames here) would want to access information regarding PAK's and .exe's beyond the one previous to the latest release. Surely anyone who really plays the game would be using either the latest PAK's (currently 64-100) or at least the immediate predecessor.

[vilvoh]

The last statement is fairly true. Pak developers, at least me, usually use old versions. In fact, I do pak tests with 99.04 and I spend more time developing than playing... :

[Colin]

The last statement is fairly true. Pak developers, at least me, usually use old versions. In fact, I do pak tests with 99.04 and I spend more time developing than playing... :

I suppose I was refering more to players than developers. I would have assumed that developers would have copies of older PAK files on there own computer anyway. I was never a developer but I used to keep all the PAK files right back to when Hajo was doing all the developing, until I realised that no one had any use for them any more. I still have some on DVD going back to early 2006.

[wipi35]

Or merged with topic "Allowed file types for upload"

[vilvoh]

Yep, I know you were refering to players. One doubt, what do you mean with pak? Are you refering to objects or to paksets?  ....You must realize that if there aren't major changes, old pak files from objects can work with newer versions of Simutrans. There's quite retrocompatibility. It doesn't happen the same with pakset files, that need to be updated every time that there's a new release.

[Ashley]

Having a complete and unedited record is important, if you start deleting specific kinds of topic/attachment you begin to lose context and crosslinking of information. And there is nothing more frustrating than coming across some link to or reference of a topic that doesn't exist anymore, or has been moved. Broken links are the single most annoying thing on the internet, and I mean that in both the technical and contextual sense. That's why I was so eager to preserve everything on the old forum, topics, boards and attachments, because I certainly couldn't think of any good way to decide what stays and what goes and keeping a record of the past is important.

I'd say think very carefully about the legacy of this forum before forcing all images to be hosted off-site, the context of a conversation can be completely lost if everyone in the thread is referring to an image which is no longer available, and that thread then becomes meaningless. Images certainly are small compared to .sve's, especially if only .jpeg/.png are allowed and limited to a sensible size. Same for addons, or pretty much any other kind of file - remember that external hosting services are frail and changable, and once something is gone it is usually gone forever.

To conserve bandwidth you could use apache to disable hotlinking, or turn on compression, or even just design a forum skin which is a little bit lighter (for example, each page I made on the archive reduced the page size by some 80% on average, there's a lot of bloat there which costs bandwidth, especially without compression).

If necessary I am sure I could produce some kind of incremental archival system, to backup/cull old threads in a static form (like the backup I did for the old forum). It could even run automatically I'm sure, with admin rights, to copy/delete old threads. With some creative addition of apache rewrite rules it could co-exist with the current live forum, so that links to the old archived threads are of the same format as they were when live.

As a thought, the total size of *all* the attachments on the old forum (remember it was active for around 3 years) only amounted to ~3gig.

[vilvoh]

I'm curious to know what is the percentage of images, paks, sve's, zip in those 3 GB of attachments.

[wipi35]

We can ask the users to remove no longer needed attachments. That might help a bit, too.

Many attachments are screenshots to illustrate a bug report. They can be removed as soon as bug is solved.
Is this a practical suggestion?

[IgorEliezer]

I'm going to merge three topics with the same issue into one: "Forum Screen Save problems." by Colin, "Unable to upload images." by wipi35, and "allowed file types for upload" by ansgar... and the topic title will be "Forum attachments discussion" to make it clear.

I won't change every message's subject in order to remind the origin of each post.

Coming soon....

EDIT: Done!

EDIT 2: Please, upon replying, verify if message's subject is "Re: Forum attachments discussion". If not, change it to new topic's subject.

[wipi35]

I'm going to merge three topics with the same issue into one: "Forum Screen Save problems." by Colin, "Unable to upload images." by wipi35, and "allowed file types for upload" by ansgar... and the topic title will be "Forum attachments discussion" to make it clear.

A very wise decision indeed. 

[Ashley]

I'm curious to know what is the percentage of images, paks, sve's, zip in those 3 GB of attachments.

.zip 899 files, 370MB
.rar 106 files, 80MB
.sve 682 files, 1.4GB

.png 7551 files (including thumbnails), 767MB
.jpg 3986 files (including thumbnails), 747MB
.gif 775 files (including thumbnails), ~80MB
.bmp 78 files (including thumbnails), 53MB

.pdf 7 files, 4MB
.pak 922 files, 83MB

total: 15538 files, 3,667,064KB

The real space hog is the save files, and most of those are from the recent fad for collaborative games, on a quick look the collaborative game save files alone account for:

.sve (collab) 334 files, 1.04GB (out of 1.4GB)

That's more than a quarter the entire size of the archive! These are just the same save game posted over and over with only minor modifications, I suggest that the best way to save space on the forum would simply be to ban the uploading of such save games, but still permit the uploading of .sve's for testing purposes (most .sve's posted for bug reports etc. are very small files, as a lot of people recreate the situation in a smaller game, and having one-off postings of 2meg files is fine, it's just the repetition of a 3meg save 40 or 50 times which adds up...)

[prissi]

Also, if someone would add a network mode to simutrans (or even an automated check in) then things would be not needed anymore for collab games ... )

[vilvoh]

Nice way to ask it... implement multiplayer mode to save server disk space...p0wned!!....
AFAIK, there's a collaborative game that been playing through a gmail account. We will have to ask them how it goes...

So, watching those figures, I think the main problem are images (png and jpg), and we could avoid it applying the solutions proposed above (mainly external hosting) I think the issue it's quite clear, isn't it? In reference to savegames, I would only allow them at Bug reports and Help requests boards.

[Isaac Eiland-Hall]

Ah, you can allow or disallow *attachments* for a particular board (which is the aforementioned pain), but you cannot allow/disallow *types* on a board-level basis. That's forum-wide.

[Ashley]

Well that's ~1.5GB of images over the course of 3 years, even if an archival was run only once every 6 months the maximum you could reasonably expect to accrue would be around 250-500MB/6 months. TBH I think we are trying to solve a problem here which doesn't exist yet, why not worry about restricting attachment space once it becomes limited (and even then, it would be much better to simply archive old posts and attachments to a seperate, cheap, static server...)

Maybe we should try to have as few restrictions as possible, monitor disk/bandwidth usage over the course of 3 months or longer and then make a decision based on actual utilisation and need rather than on guesswork - trying to anticipate how the forum will be used may end up stifling growth and fragmenting resources. The only restriction I would be in favour of at present is a sensible limit on max attachment size (e.g. 2-3MB) and a general rule forbidding the repeated posting of savegames for collaborative games. Sensible moderation policies can take care of any other potential problems, and the community retains unrestricted, centralised resources.

At this point I feel we ought to be discussing resiliancy/redundancy and backup/archival (in case this sort of situation arises again) than trying to guess the future...

Also, if someone would add a network mode to simutrans (or even an automated check in) then things would be not needed anymore for collab games ... )

Hehe, well don't look at me, if Simutrans was written in python maybe...

[vilvoh]

That's true, atm we don't have clear figures to guess what will happen. It's also true that this new stage of the forum has many less activity than the old one...you know, we're still setting up a lot of things, so we can relax for a few months. But do not relax too much or we could find with another annoying surprise, folks!

Hehe, well don't look at me, if Simutrans was written in python maybe...

Are you saying that if we port it to python, you code the multiplayer? Ok, it's worthwhile...uhmm,let's see, somebody knows where's my C++-to-Python converter?.....

[Ashley]

Well it'd be quicker to rewrite it from scratch than port it =P I've made one or two abortive efforts along those lines actually, both in 2 and 3 dimensions (it isn't until you try and start making a game like Simutrans that you realise just how much effort has gone into it, and you get a feeling of awe towards those that have developed it).

Multiplayer would be awesome though... :(

[vilvoh]

Port it, make it from scratch, it doesn't matter. We would do almost anything to get the multiplayer... :

[IgorEliezer]

We would do almost anything to get the multiplayer... :

Is vilvoh supposed to code anything for multiplayer mode? 

*pun*

(We are about to start a new subject in this topic... heh)

I'd like to see multiplayer mode running in Simutrans too, but as I don't know anything about C++, I can't help in it.

It makes me remember something: hadn't we discussed in old forum that we could get a "space" in TTD-foruns for Simutrans development? Who knows we could earn some help from there.

Coming back to original subject: Once I suggested to delete General Discussion Category, because there are "some" non-Simutrans-related topics (e. g. "The Topic Killer's Worst Nightmare Thread (AKA "Last PostWins") - Oh No!", with 273 post; and "Test" with 431 posts). But.... as I wasn't too much frequent there, I don't know if deleting such stuff will cause undesirable loss for community. :-\

[vilvoh]

Is vilvoh supposed to code anything for multiplayer mode? 

You talkin' to me?.... :

Once I suggested to delete General Discussion Category, because there are "some" non-Simutrans-related topics ......I don't know if deleting such stuff will cause undesirable loss for community. :-\

Well, IIRC, the General Discussion board from the old forum consist of a lot ot topics about transportation, amazing images of buildings and vehicles, news about transportation, jokes, topics blaming blender or canada, some interesting polls (i.e: where are you from? How much time do you really play Simutrans?), a lot of topics with tons of screenshots (Mustang versus Civic) and several happy new year greeting topics. So, we won't miss it too much, but from that moment we won't know what to do when we're fooling around in the forum....

[Spike]

Port it, make it from scratch, it doesn't matter. We would do almost anything to get the multiplayer... :

Give me a few extra years of life, then I port the needed parts to Java and use my Sonnheim server offer a multiplayer Simutrans.

I would like to do that, but I don't have the time, and it'd take months or even years :/

Maybe I can try to start a very simple transport simulation in Java, and get it running in multiplayer, but most people would be disapointed, since it'd lack many features compared to Simutrans. Who knows. Maybe some day I get to do it.

[Frank]

...
Then we have a new policy about attachments here.

I think we should save space and bandwidth from/against files that we attach in forum for temporary uses (screenshots and saves just to show a bug).

Then "where to host a savegame or a screeshot when I want to show a bug or ask for a help?"

I leave here a small list of services dedicated to image/file hosting:

http://www.sendspace.com
http://www.4shared.com
http://www.photobucket.com
http://www.imageshack.us

I would ask for everyone to use such services in order to avoid attaching files that become useless after a time. I can't guarantee reliability of all services above, because I only use and know photobucket, I have to say I'm happy with it.

...

alternatively files.simutrans-germany.com/

no script, no cookie, no registration

Translation into other languages possible

[IgorEliezer]

alternatively files.simutrans-germany.com/

What?!

I didn't know this site until now!

Frank, can you say me more info about this site, like capacity, who is maintainer, how long files are kept there, etc?

This site could be our official site for attachments.

[DirrrtyDirk]

I'll translate Frank's PM (on his request):

The server simutrans-germany.com belongs to me.

HD-space is more than sufficient - as is traffic per month. Both really should be no issue.

file lifetime:

- without username: 1 day
- with username, but without password: 30 days
- with both, depending on setting: up to 999 days - eventually even permanent (when the field is left blank, but that shouldn't be used excessively.). Default is 60 days.

The site (and the server) are relatively new and it was meant as an uncomplicated way to exchange files. It was created because of the small attachment limits here on the forum (and on the German forum it's not much better either). And sending by email, putting things on a server, link to, and then remember to delete were a bit complicated in the long run. And also it can be a pain to register with several different file services, that have their own limits and regulations, sometimes don't have a multi-language interface (only English)... far from perfect.

There is no user management though.
Username and password are stored in an ini file for each file - so they don't have to match every time.
In fact username and password are only meant to enable people to filter files for their own, so that they can handle/delete them more easily.