# Technical Architecture How Stock Analysis v6.0 works under the hood. ## System Overview ``` ┌─────────────────────────────────────────────────────────────────────┐ │ Stock Analysis v6.0 │ ├─────────────────────────────────────────────────────────────────────┤ │ │ │ ┌──────────────────────────────────────────────────────────────┐ │ │ │ CLI Interface │ │ │ │ analyze_stock.py | dividends.py | watchlist.py | portfolio.py│ │ │ └────────────────────────────┬─────────────────────────────────┘ │ │ │ │ │ ┌────────────────────────────▼─────────────────────────────────┐ │ │ │ Analysis Engine │ │ │ │ │ │ │ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ │ │ │Earnings │ │Fundmtls │ │Analysts │ │Historical│ │ │ │ │ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ │ │ │ │ │ │ │ │ │ │ │ │ ┌────┴────┐ ┌────┴────┐ ┌────┴────┐ ┌────┴────┐ │ │ │ │ │ Market │ │ Sector │ │Momentum │ │Sentiment│ │ │ │ │ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ │ │ │ │ │ │ │ │ │ │ │ │ └───────────┴───────────┴───────────┘ │ │ │ │ │ │ │ │ │ [Synthesizer] │ │ │ │ │ │ │ │ │ [Signal Output] │ │ │ └──────────────────────────────────────────────────────────────┘ │ │ │ │ │ ┌────────────────────────────▼─────────────────────────────────┐ │ │ │ Data Sources │ │ │ │ │ │ │ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ │ │ │ Yahoo │ │ CNN │ │ SEC │ │ Google │ │ │ │ │ │ Finance │ │Fear/Grd │ │ EDGAR │ │ News │ │ │ │ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │ │ └──────────────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────┘ ``` --- ## Core Components ### 1. Data Fetching (`fetch_stock_data`) ```python def fetch_stock_data(ticker: str, verbose: bool = False) -> StockData | None: """Fetch stock data from Yahoo Finance with retry logic.""" ``` **Features:** - 3 retries with exponential backoff - Graceful handling of missing data - Asset type detection (stock vs crypto) **Returns:** `StockData` dataclass with: - `info`: Company fundamentals - `earnings_history`: Past earnings - `analyst_info`: Ratings and targets - `price_history`: 1-year OHLCV ### 2. Analysis Modules Each dimension has its own analyzer: | Module | Function | Returns | |--------|----------|---------| | Earnings | `analyze_earnings_surprise()` | `EarningsSurprise` | | Fundamentals | `analyze_fundamentals()` | `Fundamentals` | | Analysts | `analyze_analyst_sentiment()` | `AnalystSentiment` | | Historical | `analyze_historical_patterns()` | `HistoricalPatterns` | | Market | `analyze_market_context()` | `MarketContext` | | Sector | `analyze_sector_performance()` | `SectorComparison` | | Momentum | `analyze_momentum()` | `MomentumAnalysis` | | Sentiment | `analyze_sentiment()` | `SentimentAnalysis` | ### 3. Sentiment Sub-Analyzers Sentiment runs 5 parallel async tasks: ```python results = await asyncio.gather( get_fear_greed_index(), # CNN Fear & Greed get_short_interest(data), # Yahoo Finance get_vix_term_structure(), # VIX Futures get_insider_activity(), # SEC EDGAR get_put_call_ratio(data), # Options Chain return_exceptions=True ) ``` **Timeout:** 10 seconds per indicator **Minimum:** 2 of 5 indicators required ### 4. Signal Synthesis ```python def synthesize_signal( ticker, company_name, earnings, fundamentals, analysts, historical, market_context, sector, earnings_timing, momentum, sentiment, breaking_news, geopolitical_risk_warning, geopolitical_risk_penalty ) -> Signal: ``` **Scoring:** 1. Collect available component scores 2. Apply normalized weights 3. Calculate weighted average → `final_score` 4. Apply adjustments (timing, overbought, risk-off) 5. Determine recommendation threshold **Thresholds:** ```python if final_score > 0.33: recommendation = "BUY" elif final_score < -0.33: recommendation = "SELL" else: recommendation = "HOLD" ``` --- ## Caching Strategy ### What's Cached | Data | TTL | Key | |------|-----|-----| | Market Context | 1 hour | `market_context` | | Fear & Greed | 1 hour | `fear_greed` | | VIX Structure | 1 hour | `vix_structure` | | Breaking News | 1 hour | `breaking_news` | ### Cache Implementation ```python _SENTIMENT_CACHE = {} _CACHE_TTL_SECONDS = 3600 # 1 hour def _get_cached(key: str): if key in _SENTIMENT_CACHE: value, timestamp = _SENTIMENT_CACHE[key] if time.time() - timestamp < _CACHE_TTL_SECONDS: return value return None def _set_cache(key: str, value): _SENTIMENT_CACHE[key] = (value, time.time()) ``` ### Why This Matters - First stock: ~8 seconds (full fetch) - Second stock: ~4 seconds (reuses market data) - Same stock again: ~4 seconds (no stock-level cache) --- ## Data Flow ### Single Stock Analysis ``` User Input: "AAPL" │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ 1. FETCH DATA (yfinance) │ │ - Stock info, earnings, price history │ │ - ~2 seconds │ └────────────────────────┬────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ 2. PARALLEL ANALYSIS │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ Earnings │ │Fundmtls │ │ Analysts │ ... (sync) │ │ └──────────┘ └──────────┘ └──────────┘ │ │ │ │ ┌────────────────────────────────────┐ │ │ │ Market Context (cached or fetch) │ ~1 second │ │ └────────────────────────────────────┘ │ │ │ │ ┌────────────────────────────────────┐ │ │ │ Sentiment (5 async tasks) │ ~3-5 seconds │ │ │ - Fear/Greed (cached) │ │ │ │ - Short Interest │ │ │ │ - VIX Structure (cached) │ │ │ │ - Insider Trading (slow!) │ │ │ │ - Put/Call Ratio │ │ │ └────────────────────────────────────┘ │ └────────────────────────┬────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ 3. SYNTHESIZE SIGNAL │ │ - Combine scores with weights │ │ - Apply adjustments │ │ - Generate caveats │ │ - ~10 ms │ └────────────────────────┬────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ 4. OUTPUT │ │ - Text or JSON format │ │ - Include disclaimer │ └─────────────────────────────────────────────────────────────┘ ``` --- ## Risk Detection ### Geopolitical Risk ```python GEOPOLITICAL_RISK_MAP = { "taiwan": { "keywords": ["taiwan", "tsmc", "strait"], "sectors": ["Technology", "Communication Services"], "affected_tickers": ["NVDA", "AMD", "TSM", ...], "impact": "Semiconductor supply chain disruption", }, # ... china, russia_ukraine, middle_east, banking_crisis } ``` **Process:** 1. Check breaking news for keywords 2. If keyword found, check if ticker in affected list 3. Apply confidence penalty (30% direct, 15% sector) ### Breaking News ```python def check_breaking_news(verbose: bool = False) -> list[str] | None: """Scan Google News RSS for crisis keywords (last 24h).""" ``` **Crisis Keywords:** ```python CRISIS_KEYWORDS = { "war": ["war", "invasion", "military strike", ...], "economic": ["recession", "crisis", "collapse", ...], "regulatory": ["sanctions", "embargo", "ban", ...], "disaster": ["earthquake", "hurricane", "pandemic", ...], "financial": ["emergency rate", "bailout", ...], } ``` --- ## File Structure ``` stock-analysis/ ├── scripts/ │ ├── analyze_stock.py # Main analysis engine (2500+ lines) │ ├── portfolio.py # Portfolio management │ ├── dividends.py # Dividend analysis │ ├── watchlist.py # Watchlist + alerts │ └── test_stock_analysis.py # Unit tests ├── docs/ │ ├── CONCEPT.md # Philosophy & ideas │ ├── USAGE.md # Practical guide │ └── ARCHITECTURE.md # This file ├── SKILL.md # OpenClaw skill definition ├── README.md # Project overview └── .clawdhub/ # ClawHub metadata ``` --- ## Data Storage ### Portfolio (`portfolios.json`) ```json { "portfolios": [ { "name": "Retirement", "created_at": "2024-01-01T00:00:00Z", "assets": [ { "ticker": "AAPL", "quantity": 100, "cost_basis": 150.00, "type": "stock", "added_at": "2024-01-01T00:00:00Z" } ] } ] } ``` ### Watchlist (`watchlist.json`) ```json [ { "ticker": "NVDA", "added_at": "2024-01-15T10:30:00Z", "price_at_add": 700.00, "target_price": 800.00, "stop_price": 600.00, "alert_on_signal": true, "last_signal": "BUY", "last_check": "2024-01-20T08:00:00Z" } ] ``` --- ## Dependencies ```python # /// script # requires-python = ">=3.10" # dependencies = [ # "yfinance>=0.2.40", # Stock data # "pandas>=2.0.0", # Data manipulation # "fear-and-greed>=0.4", # CNN Fear & Greed # "edgartools>=2.0.0", # SEC EDGAR filings # "feedparser>=6.0.0", # RSS parsing # ] # /// ``` **Why These:** - `yfinance`: Most reliable free stock API - `pandas`: Industry standard for financial data - `fear-and-greed`: Simple CNN F&G wrapper - `edgartools`: Clean SEC EDGAR access - `feedparser`: Robust RSS parsing --- ## Performance Optimization ### Current | Operation | Time | |-----------|------| | yfinance fetch | ~2s | | Market context | ~1s (cached after) | | Insider trading | ~3-5s (slowest!) | | Sentiment (parallel) | ~3-5s | | Synthesis | ~10ms | | **Total** | **5-10s** | ### Fast Mode (`--fast`) Skips: - Insider trading (SEC EDGAR) - Breaking news scan **Result:** 2-3 seconds ### Future Optimizations 1. **Stock-level caching** — Cache fundamentals for 24h 2. **Batch API calls** — yfinance supports multiple tickers 3. **Background refresh** — Pre-fetch watchlist data 4. **Local SEC data** — Avoid EDGAR API calls --- ## Error Handling ### Retry Strategy ```python max_retries = 3 for attempt in range(max_retries): try: # fetch data except Exception as e: wait_time = 2 ** attempt # Exponential backoff: 1, 2, 4 seconds time.sleep(wait_time) ``` ### Graceful Degradation - Missing earnings → Skip dimension, reweight - Missing analysts → Skip dimension, reweight - Missing sentiment → Skip dimension, reweight - API failure → Return None, continue with partial data ### Minimum Requirements - At least 2 of 8 dimensions required - At least 2 of 5 sentiment indicators required - Otherwise → HOLD with low confidence