The Product Problem
Social feeds can move markets before traditional news does. That is useful. It is also dangerous.
If we simply dumped every post from every finance account into a feed, users would get noise, duplicated attention, and alert fatigue. A bad signal product is worse than no signal product because it trains users to ignore the moments that actually matter.
So the design goal was not "show more tweets." The goal was:
- capture high-signal posts from curated and user-defined accounts
- route alerts only when they match user intent
- keep the ingestion layer portable
- avoid paying AI costs on every background scan
The Core Architecture
We split the system into three clear layers:
1. Source accounts
The product keeps a dedicated account registry for curated and user-added handles. That lets us distinguish between system-managed sources and custom user watchlists instead of treating X as one giant undifferentiated firehose.
2. Feed items
Each scraped post is stored as a structured record with:
- source account
- original post time
- engagement counts
- extracted tickers
- sentiment
- original URL
That matters because once the data is normalized, the frontend can filter by ticker, category, or sentiment without re-scraping anything.
3. Alert rules
This is where the real guardrail lives.
Users configure alerts per account with simple modes:
allticker_onlykeyword
An optional engagement threshold stacks on top. That means a post only routes if the main rule matches and the engagement floor is high enough. In plain language: "only interrupt me when this source says something relevant and the market is actually reacting."
Why the AI Summary Is On Demand
One easy mistake would have been to summarize every new post automatically.
We rejected that because background scanning should stay cheap and predictable. Most posts do not deserve an LLM call. So the scanner stores the raw structured item first, and the user can request an AI summary only when they care. The result is then cached on the feed item.
That gives us a cleaner economics model:
- cheap continuous ingestion
- expensive reasoning only when asked
- no repeated summary cost for the same item
Why the Scraper Is Abstracted
The scraper sits behind a small adapter interface. The first implementation can be a web-scraper path, but the surrounding product does not care whether the source later becomes an official API integration or a third-party provider.
That is not cosmetic abstraction. It is risk management.
Platforms change. Pricing changes. Rate limits change. If the rest of the system depends on a single vendor shape, the whole feature becomes brittle.
The Actual User Experience
This design supports two product surfaces at once:
- a full X/Twitter signals page with filters, engagement, ticker pills, and optional AI summaries
- a smaller Signal Lab widget that surfaces the latest items without forcing users into a dedicated feed first
Alerts then route through the existing notification system, including Telegram delivery when rules match.
What We Were Trying to Protect
The hidden design question was: what would make this a terrible product?
The answer was straightforward:
- a global stream with no user control
- expensive AI calls on every scan
- vendor lock-in at the ingestion layer
- alerts that fire so often users stop trusting them
So the final design stayed narrow and opinionated. We built for relevance, not volume.
Proof Level and Limitations
- Proof level: Internal product architecture case study.
- What this demonstrates: Signal monitoring can be designed around curated sources, account rules, engagement thresholds, and on-demand summaries instead of a global firehose.
- What this does not claim: It is not investment advice and does not claim market outperformance, perfect source coverage, or assured alert precision.
- Source anchors: Signal Lab, account filters, ticker matching, Telegram alert routing, and cached AI summary behavior.