FluentD - Log
FluentD v2.0
Installation Procedure
Code
docker compose up -d
Reference
https://www.digitalocean.com/community/tutorials/how-to-set-up-an-elasticsearch-fluentd-and-kibana-efk-logging-stack-on-kubernetes
pip install fluent-logger
from fluent import sender
from fluent import event
Function to call and Insert Record in to FluentD
#Error Log for fluent
@app.route('/ApplicationErrorLog', methods=['GET', 'POST'])
def ApplicationErrorLog():
try:
req_data = request.get_json()
serrortype = req_data['serrortype']
sprogram = req_data['sprogram']
sfnname = req_data['sfnname']
smsg = req_data['smsg']
sAppicationName = req_data['sAppicationName']
if(DBKEY == "PROD"):
sender.setup(sAppicationName, host='192.168.50.36', port=24224)
event.Event('PROD', {
'error_Type': serrortype,
'program': sprogram,
'function name': sfnname,
'error message': smsg
})
return "PROD"
else:
sender.setup(sAppicationName, host='192.168.50.36', port=24224)
event.Event('DEVL', {
'error_Type': serrortype,
'program': sprogram,
'function name': sfnname,
'error message': smsg
})
return "DEVL"
except :
#raise werkzeug.exceptions.HTTPException("Error: %s" % str("ERROR") )
errmsg = str(sys.exc_info()[1]) + ' Line: ' + str(sys.exc_info()[2].tb_lineno)
print(errmsg)
raise
return errmsg
docker build -t fluentdapp:latest .
docker run -it -d -p5015:5015 --name name_flientd fluentdapp –restart always
SNo. | Fields | Remarks |
1 | serrortype | INFO , WARNNING , ERROR |
2 | sprogram | (*) Program Name of application for example FileName of the code that we have written |
3 | sfnname | Function Name that is causing the event ( Optional ) |
4 | smsg | Error Message |
5 | sAppicationName | (*) Application name - Example LMS., JOURNEY etc |
Scope of the project.
- Run the EFK using docker compose.
- Run the middle ware and make sure it connects with EFK
- Create Sample Index in elastic search.
- Integrate Copy LE and Delete LE functionality with EFK.
FluentD v1.0
Introduction
FluentD is the centralized Log collector and format that in to standard format and send it to elastic search & kibana for visualization.
Video URL for introduction
https://www.youtube.com/watch?v=5ofsNyHZwWE
Installation ( EFK stack using Docker and kubernet )
https://www.youtube.com/watch?v=MNId4HG0wV8
EFK ( Elastic Search / Kibana / FluentD )
https://github.com/justmeandopensource/elk/tree/master/docker-efk
EFK ( Elastic Serach / Kibana / FluentD / http )
https://www.youtube.com/watch?v=Gp0-7oVOtPw