Log Source Parsing – The Foundation of ArcSight
ArcSight derives all its important forensic, correlation and reporting capabilities from its ability to parse log records into the Common Event Format (CEF). This parsing uses the Micro Focus ArcSight standard schema consisting of approximately 200 standard fields. SmartConnectors are pre-built ArcSight parsers, and they cover a wide variety of commercial network security, application and database products.
Unfortunately, SmartConnectors are not available for every product, and even when there is a SmartConnector available, it may not be compatible with the version of the software your organization uses. In addition, applications you developed in-house will not have a standard SmartConnector available. Finally, your organization may need specific fields that are not parsed by a standard ArcSight SmartConnector, or that are parsed into a format that is not compatible with your organization’s needs.
Fortunately, ArcSight provides a powerful FlexConnector framework which enables skilled developers to create parsers for logs derived from any source. Even better, ArcSight provides several other features, short of a ArcSight FlexConnector, that experienced Micro Focus ArcSight experts can leverage to modify log parsing for existing ArcSight SmartConnectors. Log parsing is a complex process, but since it attaches meaning and context to log records, it is a critical part of your ArcSight solution.
ArcSight FlexConnector Development Services
Development of an ArcSight FlexConnector is both an art and a science. Properly parsed log records facilitate incident response and empower hunt teams, while enabling you to develop cross-correlation rules, trends and all types of content and reporting. Familiarity with every aspect of the Micro Focus ArcSight feature set, as well as experience with the common uses for every field in the ArcSight schema, is necessary to create a valuable ArcSight FlexConnector. Finally, our working knowledge of more economical alternatives to FlexConnectors such as map files, second-level regex parsers, parser overrides and external mapping, can often provide your ArcSight solution with the data it needs while saving you money on consulting and ongoing maintenance costs.
|FlexConnector development for new log source||Onboard and attach meaning for new log sources for integration into the ArcSight feature set. This includes field parsing, categorization, documentation and installation support.||40|
|Parser Override||Enhance an existing SmartConnector to parse additional log source fields. Also change field mappings and formats as required.||20|
|Map Files||For any SmartConnector or FlexConnector, derive additional field values based on parsed fields.||10|
|External Database FlexConnector||Enhance existing SmartConnectors or FlexConnectors to use parsed fields to look-up data in any JDBC database. Parse and add these fields to the log record before forwarding.||40|
|Second Level Regex Parser||As an alternative to a parser override, enhance an existing SmartConnector to re-parse any fields.||10|
In general, we perform this type of work remotely.
- Develop and deliver the appropriate properties, categorization and/or map files
- Detailed installation instructions
- Installation assistance, QA and remediation of any deficiencies
- Samples of log records for all record types
- Documentation of log record format
- For syslog sources, please send the records to a Syslog Daemon SmartConnector and then forward to a Logger with “Preserve Raw Event” enabled in the Destination Runtime Parameters
- For database sources, we require the JDBC connector string and credentials necessary to read the tables or views containing the log records. The database schema, if available, is also helpful
- For Windows Event Log sources, we will need to collect event logs first and then write a key-value parser for it