2025-10-13 Server Outage

[Isaac Eiland-Hall]

During the afternoon, CPU on the server slowly increased, causing sites to stop responding. A reboot of the server seemed to help temporariy, but within the hour, sites became unresponsive again.

Further investigation seems to reveals >4.2GB in log files. Those files have been rotated and set t o rotate daily. Server CPU currently looks good but am monitoring.

[Isaac Eiland-Hall]

Update: Different log file got overwhelmed in less than 24 hours.

It appears the server is seeing increased traffic. Possibly bots trying to find vulneabilities. For now, AWStats paused and increased apache workers and it seems stable, but we did have a few minutes of downtime a couple of times this afternoon as I was working on it — in part because of two server restarts as well as the server running out of workers because it was being overloaded by writes to log files that were huge in size.

Keeping an eye on it. Of course, if problems happen when I'm asleep or at dialysis, it may be a bit, but I am actively monitoring.