I am developing a mobile application on IOS and Android. Currently I use Firebase Crashlytic to track application crash log.
I am not quite satisfied with what I can do with Crashlytic. For example when a user reported an issue and had a video record an app crash at a specific time, I would like to see the log from the device around that time, but it's not easy with Crashlytic.
A solution popup in my mind to have the mobile application send the crash log to my AWS SQS queue and somehow pass it to Elasticsearch so that I can filter the log using Kibana.
I would like to implement something like this
retry_send_error=true in memory.retry_send_error==true in memory, try send the log again.I have been searching on the Internet to find some reference example and could not find any good one. So I doubt my solution could have something wrong.
Please help suggest if you know some good example with architecture similar to this, or if you know the reason why this solution is not so popular.
The way you want to solve this is basically not bad. But in my opinion ELK is not precisely you want.
The SQS + Lambda is a good idea to aggregate and filter the lots of public requests, but have some issues with security, or efficiency (or maybe cost). And the architecture of that will not be simple at all.
However, if I may recommend sentry.io, which is exactly the tool you need there as I understand the problem correctly.
And sentry have solution for mobile, android and iOS too.
I did use sentry for different things, and have a clearly readable interface to work with. With minimal code assigns you can tag the events, throw good Exceptions for filtering, and sentry solves the rest.
Here a good article that explain many solutions to do remote logging for mobile apps.
The best way to send your log to elastic stack is to use Hyperlog.
Hyperlog is a free and open source library to send your logs to ELK or RequestBin, service for inspecting HTTP payloads.
ELK goes for Elasticsearch, Logstash, Kibana - services for storing, collecting and monitoring logs. Despite they are mostly used for web apps, we can send logs from our apps to ELK as well. Deployment and configuring of the ELK stack is out of scope of this article, but you can find details here. You can also choose one of the ELK stack providers, there are quite a lot of them.
You can also take a look to Bugfender and Loggly that do not require the entire elastic stack to monitor and troubleshoot your apps.