Okta Access Gateway Log Download Fails Due to Size or Timeout Limits

Okta Access Gateway (OAG) log downloads fail when the log files exceed default size limits or when the polling process times out. The configuration and UI logging messages were added with the 2023.4 version. Administrators must increase the maximum allowed log size or the polling timeout values in the configuration file to resolve this issue. When downloading logs from the OAG Admin Console, Okta displays one of the following errors:

minimum.disk.space.not.available=Not enough disk space available on the server to generate log file

disk.space.not.available=Log File is too large, not enough disk space available on the server to generate log file

failed.logs.not.available.for.given.date=Log files are not available for download for the given date

could.not.prepare.tar.gz.file=Failed to prepare log file

failed.remote.log.preparation.event.creation=Failed to complete log download task on worker node(s)

failed.remote.log.file.download.failed=Failed to download logs from worker node(s)

failed.remote.log.preparation.task.timeout=Log file size in worker node(s) too large, failed to download logs from worker(s)

• Okta Identity Engine (OIE)
• Okta Classic Engine
• Okta Access Gateway (OAG)
• Log Download

The log files exceed the default maximum allowed log size or disk space requirements, or the polling process times out before the worker nodes finish generating the logs.

The file size or timeout limits require adjustments in the configuration file.

Verify system utilization permits larger files, open the configuration file in the OAG shell, and increase either the file size or the timeout limits depending on the encountered error.

1. Check the system utilization to ensure space permits by running the following commands:
   

   sudo df -h

   ls -lh /opt/oag/logs/all/ics_all.log*

   ls -lh /opt/oag/logs/archive/all/

   The archive for a specific date on an appliance will contain all logs from every directory under /opt/oag/logs/. A rough estimation can be done by checking the directory size using : 

   du -sh /opt/oag/logs/*

   NOTE: 

   • It is recommended to use log forwarders and keep minimal logging on the cluster. For more information, refer to Manage log verbosity.
   • The workaround mentioned below should only be used as a temporary solution, and the changes must be reverted once the log download is successful.
2. Sign in to the OAG shell.
3. Open the configuration file using a text editor:
   

   sudo vi /opt/oag/bin/logPrepareConfig.sh

4. For file size errors on all OAG nodes, increase the MAX_ALLOWED_LOG_SIZE and DISK_SPACE_REQUIRED values to 20000000 (20GB).
5. Timeout errors will only be seen on the Admin node, increase the MAX_ATTEMP_TO_GET_REMOTE_STATUS value to 150 (a 5-minute timeout).
   NOTE: Increasing the SLEEP_BETWEEN_ATTEMPTS value is not recommended.
   NOTE: Increasing the MAX_ATTEMP_TO_GET_REMOTE_STATUS  on the worker nodes changes nothing.

How are the default settings restored?

Run the following command on any node to revert the configuration file to the default settings.

How is the log preparation process troubleshooted?

Set the global log level to info in the Admin Console, trigger the log download, and monitor the log preparation process using the tail command.

1. Navigate to the Backups and logs tab, select Log storage, and set the Global verbosity to info.
    
2. Run the following command to view the logs:
   

   tail -F /opt/oag/logs/all/ics_all.log | grep LOG_PREPARE

3. Prompt the log download from the Admin Console.
4. Observe the log messages to verify the polling status.
   

   LOG_PREPARE_OPERATION SERVICE LOG_PREPARE_OPERATION INFO Polling log file on node myserver.lab, status=2

   NOTE: A status of 2 indicates that the log file generation on the target server has not completed. The process continues until it reaches the MAX_ATTEMP_TO_GET_REMOTE_STATUS value or observes a status of 0. A status of 0 indicates completion. If the process fails with a Failed.remote.log.preparation.task.failed error, access the last polled node to investigate the failure.