The network environment of the cloud server in the United States is stable and has rich computing resources. However, with the increase of visits to the cloud server in the United States, the log files generated by Nginx may be expanding at a rate of tens of GB per day. Unmanaged logs will occupy a lot of storage space, slow down the efficiency of query and analysis, and cause disk overflow service interruption. Managing Nginx access logs in the US cloud server is critical!

The disadvantages of a single log file are especially serious when the service scale is expanding. Troubleshooting is difficult: a single 100GB log file cannot be opened using a conventional text editor, and the grep query may take more than 30 minutes. Skyrocketing storage costs: AWS S3 standard storage costs $0.023/GB/ month, and uncompressed year-round logs can incur thousands of dollars in additional costs; Compliance risk: GDPR, CCPA and other regulations require that the log retention cycle be controllable, and mixed storage may violate the terms of data lifecycle management; Monitoring failure: ELK (Elasticsearch, Logstash, Kibana) and other monitoring systems index large files inefficiently, and real-time alarm is delayed.

With log splitting, the complex access.log can be split into separate files by time (such as daily) or by size (such as 500MB), significantly improving manageability. The following are the comparison and implementation details of the two mainstream solutions.

Logrotate automatic cutting is a stable and easy to use system level tool. The standard log management tool of the Linux ecosystem combines with Cron to implement scheduled tasks and is suitable for non-invasive modification of Nginx processes.

Create the nginx configuration file in the /etc/logrotate.d/ directory:

sudo nano /etc/logrotate.d/nginx

Write the following (for Ubuntu/Debian systems) :

/var/log/nginx/.log {
daily
missingok
rotate 14
compress
delaycompress
notifempty
create 0640 wwwdata adm
sharedscripts
postrotate
[ f /var/run/nginx.pid ] && kill USR1 cat /var/run/nginx.pid
endscript
}

Analysis of key parameters:

daily: Cut by day. You can choose weekly/monthly or size 500M.

rotate 14: retains the latest 14 archive files.

compress: Uses gzip to compress old logs (default level 6).

delaycompress: indicates that the previous log file is compressed to avoid interrupting data writing.

create 0640 wwwdata adm: Set the permission and owner of the new log file, which must be the same as the Nginx user.

The US cloud server may use UTC by default. If you need to cut logs based on local time (such as EST/PST), change the Logrotate execution time:

Sudo/etc/mv cron. Daily/logrotate/etc/cron. Hourly/instead of cutting by the hour
sudo nano /etc/crontab

Add a scheduled task (for example, execute it every day at 00:00 Eastern Time) :

0 0    root /usr/sbin/logrotate /etc/logrotate.conf

Manual trigger test:

logrotate vf /etc/logrotate.d/nginx

Check the new log file:

ls lh /var/log/nginx/access.log

Nginx native time variable cutting - millisecond precision control. For scenarios that require finer granularity segmentation (such as hourly) or avoid potential risks to USR1 signals, this can be achieved by modifying the Nginx configuration.

Timestamp formatting output Defines the log format variable in the http block of nginx.conf:

nginx
http {
map $time_iso8601 $logdate {
'~ ^ (? <ymd>\d{4}\d{2}\d{2})' $ymd;
default 'datenotfound';
}
access_log /var/log/nginx/access$logdate.log;
}

The advantage is that day-level logs are generated in UTC time, such as access20240830.log. Reduce dependencies without external tools.

Split the advanced configuration by hour. To cut by hour, combine Nginx's lua module (ngx_http_lua_module needs to be compiled and installed) :

nginx
http {
lua_shared_dict log_rotate 1m;
init_worker_by_lua_block {
local delay = 3600 Executed every hour
local handler = function()
os.execute("mv /var/log/nginx/access.log /var/log/nginx/access$(date +'%Y%m%d%H').log")
os.execute("kill USR1 $(cat /var/run/nginx.pid)")
end
local ok, err = ngx.timer.every(delay, handler)
}
}

Storage tuning strategies help with performance tuning and troubleshooting. Logs older than 7 days are automatically stored on AWS S3 Glacier ($0.004/GB/ month) or Google Cloud Coldline. Use s3cmd to synchronize:

s3cmd sync /var/log/nginx/access.gz s3://yourbucket/nginxlogs/

Upgrade the compression algorithm, replace gzip with Zstandard (compression rate increased by 30%) :

compresscmd /usr/bin/zstd
compressext .zst

Common troubleshooting About permission issues. Ensure that the Nginx user (such as wwwdata) has write permission to the log directory:

chown R wwwdata:adm /var/log/nginx

When you run kill USR1 to reopen the log file, ensure that the PID file path is consistent with the Nginx configuration:

nginx
pid /var/run/nginx.pid

Monitoring integration pushes logs to ELK or cloud services (such as Amazon CloudWatch) in real time via Filebeat:

yaml
Filebeat configuration
filebeat.inputs:
type: filestream
paths:
/var/log/nginx/access.log
processors:
decode_json_fields:
fields: ["message"]
target: "json"
output.elasticsearch:
hosts: ["youresendpoint:9200"]

Log splitting is not a simple file splitting, but a system engineering involving storage cost, query efficiency and compliance. In the US cloud server environment, Logrotate solution is recommended for small and medium-sized businesses to reduce maintenance costs. High concurrency scenarios can be combined with Nginx native configuration and Zstandard compression to maximize performance gains. When terabyte logging becomes the norm, data can only evolve from a burden to an asset through automated toolchains and intelligent layering strategies.