Event IDs

Overview

Each service emits a number of log statements when certain events within that service occur. When these events are logged they contain the relevant amount of written data and values to help you understand and trace that log but they also contain an event id as part of the log statement. This event ID allows you to find a series of log statements or use queries to help better understand what events have occurred within a service.

You can collect and centralise the logs of Glasswall Halo using a log aggregation and monitoring tool, such as Elk stack, Splunk, or Datadog. These tools provide powerful search and visualisation capabilities, allowing you to query logs based on event IDs, timestamps, and other attributes.

Log levels

Log levels indicate the severity or importance of a log entry. This will help you prioritise and filter log messages. here's how to understand and utilise log levels effectively:

Log level hierarchy

Log levels typically follow a hierarchy, with each level indicating increasing severity. common log levels include:

• Information: provides general information about the application's operation.
• Warning: highlights potential issues or unexpected behaviour that may not be critical.
• Error: indicates errors that require immediate attention.
• Critical: reserved for the most severe issues that may lead to application failure.

Consumer actions

When you consume the log data you can take specific actions based on log levels:

• Info: these logs are valuable for monitoring the regular operation of an application. operations teams may use them to ensure the system is running smoothly.
• Warning: potential issues or irregularities are highlighted. Operations teams can set up alerts to proactively address these.
• Error: immediate attention is needed. operations teams should investigate and address errors promptly to prevent service disruptions.
• Critical: indicates a severe issue that requires immediate action. operations teams should be alerted, and incident response processes should be initiated.

The events omitted by Glasswall Halo are as follows:

Global event IDs

Event	ID	Description	Log level
MongoDB-healthcheckfailed	100100	When the MongoDB database health check fails	Error
MongoDB-documentsretrieved	100101	When a query to the MongoDB database successfully returns documents	Information
MongoDB-documentinserted	100102	When documents are successfully inserted into the MongoDB database	Information
MongoDB-documentsdeleted	100103	When documents are successfully deleted from the MongoDB database	Information
MongoDB-sumofcalculated	100104	When a sumof query is successfully executed against the MongoDB database	Information
MongoDB-documentsupdated	100105	When documents are successfully updated in the MongoDB database	Information
MongoDB-documentsmatched	100106	When an exists query is successfully executed against the MongoDB database	Information
CosmosDB-healthcheckfailed	100150	When the cosmos database health check fails	Error
CosmosDB-itemsretrieved	100151	When a query to the cosmos database successfully returns items	Information
CosmosDB-iteminserted	100152	When items are successfully inserted into the cosmos database	Information
CosmosDB-itemsdeleted	100153	When items are successfully deleted from the cosmos database	Information
CosmosDB-sumofcalculated	100154	When a sumof query is successfully executed against the cosmos database	Information
CosmosDB-itemsupdated	100155	When items are successfully updated in the cosmos database	Information
CosmosDB-itemsmatched	100156	When an exists query is successfully executed against the cosmos database	Information

Sync API

Event	Id	Description	Log level
Receivedauthresponse	5	When the API received the response from the Engine service	Information
Suscribedtoqueue	6	When the API he subscribed to the response queue	Information
Createdqueue	7	When the API creates the correct response queue	Information
Sentmessage	8	When the API successfully submits a message to the rabbit mq	Information
Filewritten	9	When the API successfully writes a file to the file storage	Information
Fileread	10	When the API successfully reads the rebuilt file or analysis from storage	Information
Messagepollerstarted	11	When the rate limiting polling has started	Information
Incorrectresponsemessage	12	When the API received a response from the Engine service but there is no task waiting to return the response to the user	Warning
Operationfailed	13	When an operation within the API fails	Warning
Retryingpolicy	14	When an operation retries based on the defined retry policy	Warning
Failedrebuildarchiveentry	15	When the API fails to rebuild the clean files sections of an archive response	Warning
Failedanalysisarchiveentry	16	When the API fails to rebuild the analysis section of an archive response	Warning
Ratelimited	17	When a request is not allowed due to rate limiting being reached	Warning
Errorhandlingrpcresponse	18	When the API fails to handle a response message from the Engine	Error
Failedtosendtransactionevent	19	When the API fails to send transaction events used for dashboards	Error
Requestfailed	20	When the request to the API fails for a known reason	Error
Unknownrequestfailure	21	When the request to the API fails for an unknown reason	Error
Pipelinestarted	22	When an action within the API begins	Information
Pipelinecompleted	23	When an action within the API has completed	Information
Starterror	24	When the API fails to start up	Information
Storagenotfound	25	When the API is unable to find the mounted storage location	Information

Engine

Event	Id	Description	Log level
Metricsoutput	2	When the Engine service logs metrics to stdout	Information
Subscribedtoqueue	3	When the Engine service has subscribed to the request queue	Information
Createdqueue	4	When the Engine service has created the request and logging queue	Information
Sentmessage	5	When the Engine service has sent a message to the response or report queue	Information
Completedmessage	6	When the Engine service has completed the request message from the API	Information
Rejectedmessage	7	When the Engine service rejects the request message from the API	Rejectedmessage
Getstream	8	When the Engine service gets stream of a file stored on the mounted disk	Information
Writtenfile	9	When the Engine service has written the rebuilt file to the mounted storage	Information
Servicestopping	10	When the Engine service stops running	Information
Messagereceivedduringshutdown	11	When the Engine service receives a message during its shutdown process	Information
Handlingmessage	12	When the Engine service begins to handle a request	Information
Writeanalysisfailed	14	When the Engine service fails to write the analysis report to the mounted storage	Warning
Writecleanfilefailed	15	When the Engine service fails to write the clean file to the mounted storage	Warning
Retryingpolicy	16	When the Engine service retries an operation based on a defined retry policy	Warning
Engineversionfailure	17	When the Engine service fails to get the version from the embedded Engine	Warning
Rabbitmqhealthfailure	18	When the RabbitMQ health check fails	Warning
Errorhandlingmessage	20	When the Engine service fails to handle a request message	Error
Messageerror	21	When the Engine service has an issue processing a message from the request queue	Error
Servicedisconnected	22	When the Engine service disconnects	Critical
Storagenotfound	23	When the Engine service cannot find the mounted storage	Information
Starterror	24	When the Engine service fails to start	Information
Trackeractionfinished	25	When the Engine service completed tracking an event within the service	Warning
Servicestarting	26	When the Engine service starts up	Information
Storagehealthfailure	27	When the storage health check fails	Warning
Writeexportfilefailed	28	When the Engine service fails to write the export file to the mounted storage	Warning
Detectingpathfiletype	2001	When detectfiletypefromfile is called	Information
Detectingmemoryfiletype	2002	When detectfiletypefrommemory is called	Information
Nolicensesetatsessioncreation	2003	When no active license was found. in this case, no CDR occurs	Critical
Enginelicenseerror	2004	An active license was found, but it is invalid or expired	Critical

Report extractor

Event	Id	Description	Log level
Unabletoconnecttomessagebroker	1	When the service is unable to connect to the message broker at start up	Critical
Unabletoconnecttomessagebrokerqueue	2	When the service is unable to connect to the response queue at start up	Critical
Unexpectedmessagebrokerexception	3	When the API has an unexpected error connecting to the message broker or queue at start up	Critical
Storagehealthfailure	4	When the storage health check fails	Error
Readanalysisreport	5	When the report extractor has read the analysis report from the mounted drive	Information
Failedtoreadanalysisreport	6	When the report extractor fails to read the analysis report from the mounted drive	Error

API access

Event	Id	Description	Log level
Anonymousauth	1	When a request is allowed via anonymous auth	Information
Missingauthheader	2	When a request is missing the required auth header	Information
Validorganisation	3	When a request is made with the correct auth org id	Information
Invalidauthheader	4	When a request is made with invalid auth header	Information
Servicefailedtostart	5	When the service fails to start	Critical
Upstreamresponse	6	When an upstream service returns a response	Information
Upstreamrequest	7	When a request to an upstream service is made	Information
Requesterror	8	When a request errors	Error

Portal access

Event	Id	Description	Log level
Upstreamresponse	1	When an upstream service returns a response	Information
Upstreamrequest	2	When a request to an upstream service is made	Information
Starterror	3	When the service fails to start	Critical
Requesterror	4	When a request errors	Error

Async API

Event	Id	Description	Log level
Servicestarting	1	When the API starts up	Information
Servicestopping	2	When the shutdown process is called for async API	Information
Messagereceivedduringshutdown	3	When the API receives a message during its shutdown process	Warning
Handlingmessage	4	When the API is handling a message from the queue	Information
Handledmessage	5	When the API has successfully handled a message from the queue	Information
Messageerror	6	When the API has an issue processing a message from the queue	Error
Servicedisconnected	7	When the API disconnects from the message broker	Critical
Subscribedtoqueue	8	When the API subscribed to receive message from the queue	Information
Sentmessage	9	When the API has sent a message for processing	Information
Filewritten	10	When the API successfully writes a file to the file storage	Information
Fileread	11	When the API successfully reads the rebuilt file or analysis from storage	Information
Incorrectstatusstored	12	When the status for a stored item is not the expected status	Warning
Operationfailed	13	When an operation within the API fails	Critical
Retryingpolicy	14	When an operation retries based on the defined retry policy	Warning
Requestfailed	16	When the request to the API fails for a known reason	Error
Unknownrequestfailure	17	When the request to the API fails for an unknown reason	Error
Pipelinestarted	18	When an action within the API begins	Information
Pipelinecompleted	19	When an action within in the API has completed	Information
Starterror	20	When the API fails to start up	Information
Storagenotfound	21	When the API is unable to find the mounted storage location	Information
Hashingerror	22	When the API is unable to generate hashes for the file	Error
Deletefailed	23	When the API is unable to delete the file from the mounted storage location	Error
Unabletoconnecttomessagebroker	24	When the API is unable to connect to the message broker at start up	Critical
Unabletoconnecttomessagebrokerqueue	25	When the API is unable to connect to the response queue at start up	Critical
Unexpectedmessagebrokerexception	26	When the API has an unexpected error connecting to the message broker or queue at start up	Critical
Databasehealthfailure	27	When the API's database health check fails to connect to the database	Warning
Messagebrokerhealthfailure	28	When the API's message broker health check fails to connect to the message broker	Warning
Databasefailure	29	When the API is unable to complete an operation on the database after maximum number of retries	Error
Databaseitemretrieved	30	When the API has performed a retrieval on an item in the database	Information
Databaseitemstored	31	When the API has stored an item in the database	Information
Databaseitemupdated	32	When the API has updated an item in the database	Information
Handlesinglefileresponse	33	When the API has received a response relating to a single file	Warning
Handlearchivecontentresponse	34	When the API has received a response relating to an archive content item	Warning

Tally accumulator

Event	Id	Description	Log level
Unabletoconnecttomessagebroker	1	When the API is unable to connect to the message broker at start up	Critical
Unabletoconnecttomessagebrokerqueue	2	When the API is unable to connect to the response queue at start up	Critical
Unexpectedmessagebrokerexception	3	When the API has an unexpected error connecting to the message broker or queue at start up	Critical
Starterror	4	When the API fails to start up	Critical
Processtransactioneventstart	5	When a supported transaction is received and processing has started	Information
Processtransactioneventend	6	When a supported transaction is received and processing has finished successfully	Information
Licensetransactionstoreupsertstart	7	When an upsert is about to happen against the license transaction store	Information
Licensetransactionstoreupsertsuccess	8	When an upsert against the license transaction store has finished successfully	Information
Tallystoreupsertstart	9	When an upsert is about to happen against the tally store	Information
Tallystoreupsertcomplete	10	When an upsert against the tally store is successful	Information
Tallystoregetusagestart	11	When license usage is about to be retrieved from the tally store	Information
Tallystoregetusagesuccess	12	When license usage is retrieved from the tally store successfully	Information
Unexpectednegativefilesize	13	When a license transaction event for a file has a negative file size	Warning
Unexpectedrequestexception	14	When an unexpected exception is raised by the service that will result in a 500 status code for a given API request	Error
Unsupportedentitlementlimitname	15	When the supplied filter for the usage endpoint is not a valid entitlement limit	Warning
Databasehealthfailure	16	When the service health checks fail to contact the database instances	Warning
Erroredfilereceived	17	When an error occurs upstream. this will result in the tally not being incremented	Warning

Metrics collation (deprecated)

As of version 2.6.2 of Halo, the metrics collation service is no longer required.

Event	Id	Description	Log level
Unabletoconnecttomessagebroker	1	When the service is unable to connect to the message broker at start up	Critical
Unabletoconnecttomessagebrokerqueue	2	When the service is unable to connect to the response queue at start up	Critical
Unexpectedmessagebrokerexception	3	When the service has an unexpected error connecting to the message broker or queue at start up	Critical
Storagehealthfailure	4	When the storage health check fails; this is usually a problem with database connectivity	Error
Writingeventtomongostorage	5	When an incoming event is about to be persisted in the database	Information
Mongowriteerror	7	When an incoming event fails to be persisted in the database	Error
Jsonexceptionerror	8	When the message listener fails to deserialize an event	Error
Unknownfailedtoprocess	10	When an unhandled exception occurs while processing an incoming event	Critical
Failedtoimporttracedata	11	When the trace provider fails to import trace data from an incoming event header	Warning
Storagehealthcheckstartup	12	When the service performs a storage health check at startup	Information

Metrics projection

Event	Id	Description	Log level
Starterror	1	When the service fails to start up	Critical
Storagehealthfailure	2	When the storage health check fails; this is usually a problem with database connectivity	Warning
Changeprocessorstart	5	When the change processor is about to setup a watcher for database changes	Information
Changeprocessorstarted	6	When the change processor has successfully setup a watcher for database changes	Information
Changeprocessorstopping	7	When the change processor is about to stop watching for database changes	Information
Changeprocessorstopped	8	When the change processor has successfully stopped watching for database changes	Information
Changeprocessorstopfailure	9	When the change processor fails to stop watching for database changes, i.e. it was not watching for changes	Warning
Changeprocessorbatchstart	10	When the change processor has recieved a batch of database changes and is about to start procoessing them	Information
Changeprocessorbatchend	11	When the change processor has finished processing the current batch of database changes	Information
Changeprocessorunhandledexception	13	When the change processor encounters a problem while processing a change; this may cause inconsistencies in data	Error
Transactioneventprojected	14	When an event (change) has been successfully mapped to a projection	Debug
Changehandlerstarted	15	When a change handler is about to start processing changes after being called upon by the change processor	Information
Changehandlerfinished	16	When the change handler has finished processing changes	Information
Changehandlerfailure	17	When the change handler has failed to process changes	Critical
Changeprocessorstartfailure	19	When the change processor fails to start watching for database changes	Error
Changeprocessorwatchstart	20	When the change processor has started actively watching for database changes (after setting up a watcher)	Information
Daywisesummaryincremented	21	When a day-wise summary has been successfully incremented based on a completed file	Information
Daywisesummarynotincremented	22	When a day-wise summary will not be incremented based on a (usually incomplete) file	Debug
Retryingpolicy	23	When an operation retries based on the defined retry policy	Warning
Materializedviewpersisted	24	When a materialized view (projection) has been persisted in the database after being projected upon	Information

License management

Event	Id	Description	Log level
Starterror	1	When the service fails to start up	Critical
Licenseprocessingerror	2	When a request fails for an unforeseen reason	Error
Deletelicensestart	3	When the service begins deleting the current active license	Information
Deletelicenseend	4	When the service has finished deleting the current active license	Information
Updatelicensestart	5	When the service begins updating the current active license	Information
Updatelicenseend	6	When the service has finished updating the current active license	Information
Getlicensestart	7	When the service begins retrieving the current active license	Information
Getlicenseend	8	When the service has finished retrieving the current active license	Information
Licensenotfound	9	When the current active license cannot be found during update or retrieval	Error
Licenseexpired	10	When the uploaded license has expired, causing the update to fail	Error
Fetchinglicenseusagestart	11	When the service begins retrieving the current active license and usage data	Information
Fetchinglicenseusageend	12	When the service has finished retrieving the current active license and usage data	Information

ICAP server

Event	Id	Description	Log level
Backgroundservicestarting	1	When the ICAP listener is starting	Information
Backgroundservicestarted	2	When the ICAP listener has started	Information
Backgroundservicestopping	3	When the service shuts down and the ICAP listener is stopping	Information
Backgroundservicestopped	4	When the service shuts down and the ICAP listener has completed stopping	Information
Starterror	5	When the service fails to start	Critical
Halocallfailure	8	When a file cannot be built via the Halo API	Warning
Profilestoremaintainerstarting	9	When the background thread starts listening for ICAP profile changes to persist locally	Information
Profilestoremaintainerstarted	10	When the background thread listening for ICAP profile changes has started successfully	Information
Profilestoremaintainerstopping	11	When the service shuts down and the profile background thread stops listening for profile updates	Information
Profilestoremaintainerstopped	12	When the service shuts down and the profile background thread completes its shutdown process	Information
Profilestoremaintainerheartbeaterror	13	When profiles are queried and cannot be persisted due to an error	Error
Profilestoremaintainerheartbeatsuccess	14	When profiles are queried successfully	Information
Profileusagefailure	15	When the profile specified in an ICAP request is not found	Warning
Errorhandled	17	When the ICAP server finishes processing an unexpected exception by sending an error response to the ICAP client	Error
Unlicensedrequest	18	When an ICAP request is received for an unlicensed ICAP server	Critical
Retryingpolicy	19	When an action is being retried, for example a failed call into the Halo API	Warning
Fileblocked	20	When an ICAP request is processed containing a file that is blocked	Information
Fileprocessingstarted	21	When starting to process a file within an ICAP request	Information
Profilefailedtofetch	22	When a profile cannot be retrieved from the policy API	Error
Rebuiltfileexceedscachesize	23	When the ICAP request contains a file too large to be stored in the cache	Warning
Fileretrievedfromcache	24	When an ICAP request is processed and an associated rebuilt file is found in the cache	Information
Fileprocessingfinished	25	When finished processing a file within an ICAP request	Information
Clearingcache	26	When running cache cleanup starts	Information
Cachecleared	27	When running cache cleanup finishes	Information
Errorclearingcache	28	When running cache cleanup errors	Error
Filebypassed	29	When a file has been bypassed during modification by the ICAP policy in use	Information
Fileprocessed	30	When a file has been processed during modification by the ICAP policy in use	Information
Fileerrored	31	When an error occurs while attempting to process a file during modification	Information
Tlsdisabled	32	When no valid certificates are provided to the ICAP server, the server stops listening on secure port 11344 and tls is disabled.	Warning
Tlsenabled	33	The ICAP server has found a valid tls certificate and is listening on the secure port 11344.	Information
Serverstarting	1000	When the ICAP server is starting	Information
Serverstarted	1001	When the ICAP server has started	Information
Clientsocketerror	1002	When an unexpected exception occurs while listening to the ICAP client	Error
Unsupportedicaprequest	1003	When the ICAP server receives a request that is not supported, for example the method is not reqmod, options or respmod	Warning
Clientconnectionhandleerror	1004	When an error occurs when processing data sent to the server	Error
Sendfailure	1005	When attempting to send a response to the ICAP client fails	Warning
Serverlistenercancelled	1006	When the ICAP listener thread is cancelled, for example on shutdown	Warning
Errorduringsocketclose	1007	When closing a connection raises an error, for example when the client has already disconnected	Information
Clientreceiveerror	1008	When listening to data on the socket sent by the ICAP client and error occurs	Warning
Requestprocesserror	1009	When an unhandled exception occurs during the processing of an ICAP request	Error