cross-posted from: https://psychedelia.ink/post/513125
Quoting the origin HN post:
Hi HN, Mike and Warren here! We’ve been building HyperDX (hyperdx.io). HyperDX allows you to easily search and correlate logs, traces, metrics (alpha), and session replays all in one place. For example, if a user reports a bug “this button doesn’t work," an engineer can play back what the user was doing in their browser and trace API calls back to the backend logs for that specific request, all from a single view. Github Repo: https://github.com/hyperdxio/hyperdx
Coming from an observability nerd background, with Warren being SRE #1 at his last startup and me previously leading dev experience at LogDNA/Mezmo, we knew there were gaps in the existing tools we were used to using. Our previous stack of tools like Bugsnag, LogRocket, and Cloudwatch required us to switch between different tools, correlate timestamps (UTC? local?), and manually cross-check IDs to piece together what was actually happening. This often made meant small issues required hours of frustration to root cause.
Other tools like Datadog or New Relic come with high price tags - when estimating costs for Datadog in the past, we found that our Datadog bill would exceed our AWS bill! Other teams have had to adjust their infrastructure just to appease the Datadog pricing model.
To build HyperDX, we’ve centralized all the telemetry in one place by leveraging OpenTelemetry (a CNCF project for standardizing/collecting telemetry) to pull and correlate logs, metrics, traces, and replays. In-app, we can correlate your logs/traces together in one panel by joining everything automatically via trace ids and session ids, so you can go from log <> trace <> replay in the same panel. To keep costs low, we store everything in Clickhouse (w/ S3 backing) to make it extremely affordable to store large amounts of data (compared to Elasticsearch) while still being able to query it efficiently (compared to services like Cloudwatch or Loki), in large part thanks to Clickhouse’s bloom filters + columnar layout.
On top of that, we’ve focused on providing a smooth developer experience (the DX in HyperDX!). This includes features like native parsing of JSON logs, full-text search on any log or trace, 2-click alert creation, and SDKs that help you get started with OpenTelemetry faster than the default OpenTelemetry SDKs.
I’m excited to share what we’ve been working with you all and would love to hear your feedback and opinions!
Hosted Demo - https://api.hyperdx.io/login/demo
Open Source Repo: https://github.com/hyperdxio/hyperdx
Landing Page: https://hyperdx.io
I had a look through the comments on this HN thread the other day and came away more intrigued by https://github.com/openobserve/openobserve than hyperdx. Hyperdx is built on top of clickhouse whereas open observe has it’s own storage engines based on parquet files that can be accessed from local disk, S3, or a few other protocols.
I haven’t tried either option yet… I’m, currently using netdata for metrics and don’t do anything special for logs or tracing, but at tiny self-hosting scale I often find software with it’s own storage engines (often sqlite) to be extra hassle-free. I’m curious to kick the tires on openobserve for that reason.