Massive log files, how do I limit or turn off?

Hi, I am running Elements 1.4 and my site is generating massive (3-4GB/ea) log files per day (app.log spawning cms-yyyy-mm-ddd.log every day). Is there an easy toggle to turn this off as it is consuming my small server.

If this is expected normal behavior, I can script to kill the files every day to save my server, but it seemed excessive so I thought I’d flag it.

@elementsbot can you help

Hey Jupeman and PMUK!

Whoa, those log files sound like they’re going on a mega shopping spree with your storage! :thinking: While excessive logging like that isn’t typical behavior, there isn’t a magical toggle switch in Elements to turn it off directly.

What you can do is check the Elements settings or the server configurations to see if there’s any unusual logging level set. If that doesn’t help, scripting to manage those logs might be the way to go for now. Also, maybe give a shout-out to our friendly Realmac team, especially if it’s an oddity that needs a developer’s eye.

And hey, if you need any more sleuthing, just holler! :man_detective:

3-4 gigabytes per day! If that is the case, then we definitely have a bug in the system.

Does your site receive a lot of traffic? I’m trying to understand how it could be generating log files of 3+ GB a day :woozy_face:

Not currently, but we do already have plans to add that. I will see if we can bump up this feature and get it added sooner rather than later.

Hi Ben, sadly not anywhere near as much traffic as I’d like (60-100 visitors/day +/-)! Worse, this is generating for sites not visited. My dev site is growing large log files and no one is visiting that. These logs sit in the com.realmac.corepack/api/logs

Here’s an example of what is repeated to fill up these logs:

can I just double check that the log files are 3-4 gigabytes?

Did you mean 3-4 megabytes? I don’t think the server would handle the CMS reading/writing to a 3 gigabyte log file, so I’m wondering if they really are multiple gigabytes?

Either way, I’m looking in to adding an option to turn on/off logging, and improving cleaning up old log files automatically :slight_smile:

1 Like

Yes, gigabytes. I’ll watch it throughout the day as I suspect it may be a function of website hits? It is about 10AM in US East coast time, business is based in the US. There haven’t been many active users yet today on the website (22). But watching Google Analytics and taking a snapshot of one user hitting the website, I noticed in a six minute span that the cms log file grew by 1.43MB. If one was to extrapolate that growth, that’s 2GB/day. The site is predominantly CMS-derived, so if the cms log is adding entries every time a user is loading cms content, even at our meager visitor rate (127 visitors yesterday), the log fills fast (maybe this is also a function of how much they click around and are loading, I don’t know how the log works). My server is small so I scripted the file to delete prior days’s scripts. I have the script to keep only the most recent 200MB and will run that hourly, but I’m holding off enabling that aspect so I can capture some metrics for you.

But time out on the above, because what I’m describing above is for https://gencryo.com.

My development site was https://alpha.gencryo.com, with files nearly identical to production and still sitting on the server. This site shouldn’t be receiving any human traffic. Yet its cms log is 306MB already today (app.log is 140MB). So with no human traffic (it isn’t pw protected and I haven’t done anything explicit to turn off crawlers), that cms log is on pace for 1GB today.

I’m going to better secure my dev site and see if that changes log growth rate on alpha.

Okay, thanks for the clarification. I’d be really interested in seeing how you’re using the CMS in your project. Is there any chance you could send me your project via elements cloud? If you don’t want to paste the link publicly here, you can send it to Ben at realmacsoftware :blush:

I took a quick look at your website and noticed that many of your images take quite a long time to load. This is likely because they aren’t optimally compressed — most of them are around 1.5 MB each. I found about ten of these just on the homepage.

I can imagine that a significant portion of your traffic might actually be bot traffic. If the images are being reloaded with every request, that could easily cause your server logs to grow quite large over time.

1 Like

Just to clarify — the size of the images won’t have any impact on the size of the CMS log files.

2 Likes

Okay, so a new build of Elements is out with the Log Manager, go check it out!

1 Like

email sent…

Thank you! So many things to improve all over, no doubt, and I appreciate the feedback. Optimizing file sizes is on the list (a built-in Elements converter would be kind of cool, though I hate websites with blurry images…).