All Questions
Tagged with elasticsearch logstash
4,385 questions
0
votes
1
answer
22
views
Export all logs from ELK stack to external destination for compliance purpose
I am writing to you because I would need to export logs from inside elk to outside, like to blob in azure or any other destination pointc, for compliance purpose. Do you know any solution available?
...
- 1
0
votes
0
answers
20
views
ELASTICSEARCH - Custom Date is an unknown field
i try to use a custom Date Format but Elastic does not recognize it as a Date Field.
This is the Date from the Logs i input via Filebeat -> Logstash.
25.03.2025 15:30:01.876
I use %{DATESTAMP:...
0
votes
1
answer
30
views
converting date into date format in logstash elk stack
We are trying to ship a csv file from Filebeat > Logstash > Elasticsearch. This csv files has column by name “CreateOn” which will have the dates as below;
Thursday, March 10, 2016 3:00:23 PM ...
- 39
0
votes
0
answers
26
views
How to close a sql server db connection for logstash pipeline when elasticsearch is unreachble
I am using Logstash 8.12 My logstash pipeline reads data from sql server and send to Azure ElasticSearch.
It works fine when ES is available. When Elastic search node is unreachable, I get the error ...
- 85
0
votes
0
answers
11
views
How to Run Logstash Immediately and Maintain a Scheduled Execution?
I'm using Logstash to fetch data from a database and index it into Elasticsearch. My Logstash configuration includes a schedule to run every 50 minutes like this:
input {
jdbc {
...
0
votes
0
answers
23
views
Elasticsearch Index Contains More Documents Than MySQL Query Results
I’m encountering an issue where the number of documents in my Elasticsearch index exceeds the number of rows returned by my MySQL query. Specifically:
MySQL Query Result: 4802 rows.
Elasticsearch ...
1
vote
1
answer
27
views
Logstash field is never shown after aggregation
I have logstash version 7.8.0
Can someone tell me why the aggregation below never shown THREAD_ID field into documents please ?
My field : thread_id is added in the end of aggregation ..
Sample.log:
...
- 49
0
votes
0
answers
15
views
I am trying to connect 2 systems EAA nodes and expecting all the events get synch and displayed on to kibana1 and kibana2
I have connected 2 VApps which makes it as 2DC's system.
On DC1 I have added and Installed Elastic Search and EAA nodes and installed them, same goes with DC2. Now I want all the events from DC2 in ...
0
votes
2
answers
244
views
How to get the size of raw logs ingested into Elastic?
I want to determine the size of raw logs ingested into Elastic. Is it possible to do this directly in Elastic, or are there any third-party tools that can help? Any ideas on how to achieve this?
...
1
vote
1
answer
142
views
Filebeat - Log Processing Issues/Delay/Data Loss
We are experiencing significant challenges with log processing on three of our hosts. Each of these hosts runs nine services, generating between 30,000 to 72,000 events per minute per log file. The ...
0
votes
1
answer
26
views
Logstash import with nested type
I'm trying to import parent-child data into ES.
Here is my logstash config file:
input{
jdbc{
jdbc_driver_library => "/usr/library/postgresql-42.7.4.jar"
...
- 89
0
votes
0
answers
37
views
Logstash fails to start on Windows 11 using basic config
I am trying to launch Logstash for the first time on this Windows 11 laptop. I downloaded and unzipped version 8.15, and I created the following logstash-simple.conf config file in the logstash root
...
- 16.6k
0
votes
0
answers
37
views
Logstash Helm code for copying data from ELK to Opensearch
We were using ELK cluster for observability log monitoring in our current project and now we are moving that to opensearch tool. So we need to copy data from current ECK cluster( indices data ) to ...
- 11
0
votes
1
answer
51
views
Logstash unable to receive all events via single port using beats input
We are facing an issue where we are not able to receive the events concurrently from the servers which are sending events from different beats services like filebeat / metricbeat / winlogbeat using a ...
- 29
0
votes
2
answers
113
views
How can I check nulls in Logstash pipeline in filter plugin?
I have a source which sends data to my Logstash pipeline via Logstash http plugin.
The data model which is sent like:
{
"myArray": [
{
"myGrocery": {
"myId&...
- 934