Choosing how to collect and forward log messages to Splunk using syslog-ng involves understanding the evolution of support for Splunk within syslog-ng, as well as weighing the pros and cons of different solutions, both open-source and commercial.
History of Splunk Support in syslog-ng
Traditionally, syslog-ng recommended a method involving central log collection using syslog-ng, storing logs locally, and using Splunk forwarders to send them to Splunk. While effective, this approach incurred overhead by requiring installation of multiple applications and duplicating data storage.
The introduction of Splunk HTTP Event Collector (HEC) simplified log forwarding to Splunk. Initially, using the http() destination in syslog-ng was straightforward but lacked encryption and scalability. To address these shortcomings, a Python script was developed for improved security and performance when called via the program() destination.
Over time, syslog-ng’s http() destination evolved with TLS support, multi-threading, and load balancing. Syslog-ng Premium Edition (PE) introduced splunk-hec(), streamlining Splunk configuration compared to direct http() usage.
Additionally, Splunk released Splunk Connect for Syslog (SC4S), a containerized solution based on syslog-ng. SC4S enhances syslog-ng with additional message parsers, though it offers limited configuration compared to syslog-ng.
Syslog-ng Store Box (SSB), built on syslog-ng PE, offers comprehensive log lifecycle management with a Splunk destination.
Recently, syslog-ng open-source edition (OSE) integrated a Splunk destination into its configuration library (SCL), eliminating the need for custom solutions starting from version 4.2.0.
Choosing the Right Solution
The choice between syslog-ng editions depends on various factors:
- syslog-ng PE and SSB: Ideal for organizations requiring robust support and exclusive features like compliance and cloud support. They offer commercial-grade reliability and advanced capabilities like LogStore for encrypted log storage.
- syslog-ng OSE: Suitable for long-time open-source users or small-scale deployments using the free version of Splunk. It now includes built-in Splunk destination, simplifying configuration without commercial support.
- SC4S: Suitable if Splunk is the sole destination and complex filtering isn’t required. It’s based on syslog-ng open-source with added parsers, but lacks extensive configuration flexibility.
Conclusion
Both syslog-ng PE and OSE provide high-performance log collection, parsing, filtering, and versatile destination options, including Splunk. Effective message parsing and filtering reduce license costs by forwarding only relevant logs to each service. Evaluate trial versions of commercial syslog-ng variants or explore the open-source edition to determine the best fit for your environment.
For more details or trials, visit syslog-ng trials page which also provides access to the syslog-ng GitHub page for the open-source edition.
Source: https://www.syslog-ng.com/community/b/blog/posts/sending-logs-to-splunk-using-syslog-ng
Giới thiệu về DT Asia
DT Asia được thành lập vào năm 2007 với sứ mệnh rõ ràng là xây dựng bước thâm nhập thị trường cho các giải pháp bảo mật CNTT tiên phong khác nhau từ Mỹ, Châu Âu và Israel.
Ngày nay, DT Asia là nhà phân phối giá trị gia tăng khu vực về các giải pháp an ninh mạng, cung cấp các công nghệ tiên tiến cho các cơ quan chính phủ trọng điểm và các khách hàng hàng đầu thuộc khu vực tư nhân, bao gồm các ngân hàng toàn cầu và các công ty thuộc danh sách Fortune 500. Chúng tôi có các văn phòng và đối tác khắp khu vực Châu Á - Thái Bình Dương nhằm thấu hiểu rõ hơn các thị trường và cung cấp các giải pháp mang tính bản địa hóa.
