Okta Access Gateway Applications Do Not Sync Changes Pushed Through the Admin UI

Okta Access Gateway (OAG) applications fail to sync changes pushed via the Admin UI when the incrond service stops. Moving old events to a backup folder, restarting the incrond service, and resending the changes from the UI resolves this issue. When this issue occurs, the NGINX service reports no errors, but the updated configurations are not reflected when users access the applications.

• Okta Identity Engine (OIE)
• Okta Classic Engine
• Okta Access Gateway (OAG)
• High Availability
• incrond

When Okta processes changes from the Admin UI, the Admin node receives an event to create a specific configuration. The incrond service processes these events and reloads NGINX on the Admin node. After successful processing, Okta pushes the events to all workers, where incrond generates the NGINX-specific configuration and reloads the service. If the incrond service stops running, Okta fails to reflect the changes, and NGINX continues serving the last successful configuration. If the issue occurs on the Admin node, the worker nodes fail to receive the updated configurations. The service might stop during an upgrade or after a reboot.

How is the incrond service status and event processing validated?

Execute the following commands to validate the incrond service status, check for stuck events, and review the upgrade logs.

1. Validate the service status by executing the systemctl command.
   

   sudo systemctl status incrond

2. Check the /opt/oag/events location for unprocessed events.
   

   sudo ls -ltr /opt/oag/events

   NOTE: If no issue exists, the location remains empty because Okta processes events within a few seconds.

3. Check the /opt/oag/events_processed location to view the last successful synchronization time.
   

   sudo ls -ltr /opt/oag/events_processed | tail

4. Check the timestamp in the yumUpdateOutput.log file to determine when the upgrade completed.
   

   sudo ls -l /opt/oag/upgrades/current/yumUpdateOutput.log

5. Review the cron log file for errors using the timestamp from the previous step.
   

   sudo less /var/log/cron

    

How is the incrond service restarted to resolve the synchronization issue?

If the incrond service is stopped, move the old events to a backup folder, start the service, and resend the changes from the Admin UI.

1. Create a folder to hold the old events.
   

   sudo mkdir /tmp/old_events

2. Move the old events to the newly created folder to prevent them from affecting the configurations during processing.
   

   sudo mv /opt/oag/events/* /tmp/old_events

3. Verify that the old events reside in the backup location.
   

   sudo ls -ltr /opt/oag/events
   sudo ls -lrth /tmp/old_events

4. Start the incrond service after confirming the /opt/oag/events directory is empty.
   

   sudo systemctl start incrond

5. Edit and save the application in the Admin UI to resend the changes. Okta generates a new event and sends it to the Admin node for processing.
6. Verify that the events process successfully and do not remain stuck in the events directory.
   

   sudo ls -ltr /opt/oag/events

7. Review the Okta Access Gateway High Availability options to check the status of all nodes after the synchronization.
8. Perform a sync from admin to retrieve all events from the Admin node if the issue occurs on a worker node.