Initial commit with translated description
This commit is contained in:
122
docs/EQUITY_SHEET_FIXES.md
Normal file
122
docs/EQUITY_SHEET_FIXES.md
Normal file
@@ -0,0 +1,122 @@
|
||||
# Equity Sheet Fixes
|
||||
|
||||
## Contents
|
||||
- [NRR Column Fix](#nrr-column-column-q---range-values-fix)
|
||||
- [Conversion Rules](#conversion-rules)
|
||||
- [Fix Procedure](#fix-procedure)
|
||||
- [Impact](#impact)
|
||||
- [Related Columns](#related-columns)
|
||||
- [Prevention](#prevention)
|
||||
|
||||
## NRR Column (Column Q) - Range Values Fix
|
||||
|
||||
**Problem:** Values like "115-120%", "125%+", "N/A" in NRR column cause #VALUE! errors in MSS Score formula (columns Y/Z).
|
||||
|
||||
**Root cause:** Excel/Sheets formulas cannot perform math operations on text ranges.
|
||||
|
||||
**Solution:** Convert all NRR values to single numeric percentages.
|
||||
|
||||
### Conversion Rules
|
||||
|
||||
**Standard formats:**
|
||||
|
||||
| Original | Fixed | Calculation | Rationale |
|
||||
|----------|-------|-------------|-----------|
|
||||
| 115-120% | 117.5% | (115+120)/2 | Midpoint (conservative estimate) |
|
||||
| 120-125% | 122.5% | (120+125)/2 | Midpoint |
|
||||
| 125%+ | 125% | Use lower bound | Conservative (actual may be higher) |
|
||||
| N/A | [blank] | Leave empty | MSS formula uses IFERROR to handle blanks |
|
||||
| 110% | 110% | Already valid | No change needed |
|
||||
|
||||
**Edge cases (normalize before converting):**
|
||||
|
||||
| Variant | Normalized | Notes |
|
||||
|---------|------------|-------|
|
||||
| 115–120% (en-dash) | 115-120% | Replace en-dash with hyphen |
|
||||
| 115 - 120% (spaces) | 115-120% | Remove spaces around hyphen |
|
||||
| >=125% | 125%+ | Convert to standard "+" format |
|
||||
| 125%+ or higher | 125%+ | Strip extra text |
|
||||
|
||||
### Fix Procedure
|
||||
|
||||
**Option A: Manual fix via browser**
|
||||
1. Open sheet: https://docs.google.com/spreadsheets/d/1lTpdbDjqW40qe4YUvk_1vBzKYLUNrmLZYyQN-7HmFJg/edit#gid=0
|
||||
2. **IMPORTANT:** Select column Q header → Format → Number → Percent
|
||||
- This ensures values are stored as numbers, not text
|
||||
- If column is set to "Plain text", entering "117.5%" stores as text → still causes errors
|
||||
3. Navigate to column Q (NRR)
|
||||
4. For each range value:
|
||||
- Calculate midpoint (e.g., (115+120)/2 = 117.5)
|
||||
- Replace with single percentage: `117.5%`
|
||||
- Sheets auto-converts to numeric percentage when column is formatted correctly
|
||||
5. For "N/A" → delete content (leave blank)
|
||||
6. For "125%+" → replace with `125%`
|
||||
7. **Verify:** After editing, click cell → formula bar should show `1.175` (not `"117.5%"` with quotes)
|
||||
|
||||
**Option B: Sheets API fix (requires Sheets API enabled)**
|
||||
|
||||
**Prerequisites:**
|
||||
1. Enable Sheets API: https://console.developers.google.com/apis/api/sheets.googleapis.com/overview?project=831892255935
|
||||
2. Ensure column Q is formatted as Percent (do once before any API writes):
|
||||
- Via browser: Select column Q → Format → Number → Percent
|
||||
- Via API: Use `batchUpdate` with `repeatCell` + `numberFormat` (see below)
|
||||
|
||||
**Using gog CLI:**
|
||||
```bash
|
||||
# gog CLI uses USER_ENTERED by default (parses "117.5%" as numeric)
|
||||
gog-shapescale --account martin@shapescale.com sheets update \
|
||||
1lTpdbDjqW40qe4YUvk_1vBzKYLUNrmLZYyQN-7HmFJg \
|
||||
'Equity!Q5' '117.5%'
|
||||
```
|
||||
|
||||
**Using Sheets API directly (curl/Python):**
|
||||
```bash
|
||||
# CRITICAL: Specify valueInputOption=USER_ENTERED explicitly
|
||||
curl -X PUT \
|
||||
"https://sheets.googleapis.com/v4/spreadsheets/SHEET_ID/values/Equity!Q5?valueInputOption=USER_ENTERED" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-d '{"values": [["117.5%"]]}'
|
||||
|
||||
# Python example:
|
||||
service.spreadsheets().values().update(
|
||||
spreadsheetId=SHEET_ID,
|
||||
range='Equity!Q5',
|
||||
valueInputOption='USER_ENTERED', # Parse as Sheets would
|
||||
body={'values': [['117.5%']]}
|
||||
).execute()
|
||||
```
|
||||
|
||||
**Verify after writing:**
|
||||
- Click cell → formula bar should show `1.175` (numeric)
|
||||
- If formula bar shows `"117.5%"` with quotes → stored as text, still causes errors
|
||||
|
||||
### Impact
|
||||
|
||||
Fixing NRR ranges will:
|
||||
- ✅ Eliminate #VALUE! errors in MSS Score column (Y)
|
||||
- ✅ Eliminate #VALUE! errors in MSS Rating column (Z)
|
||||
- ✅ Allow proper numerical analysis and sorting
|
||||
- ✅ Make formulas copyable to new rows without errors
|
||||
|
||||
### How MSS Formula Handles Blank NRR Values
|
||||
|
||||
The MSS Score formula (column Y) includes `IFERROR()` wrapper to handle missing data:
|
||||
- **Blank NRR cell** → Formula treats as missing data, uses available metrics only
|
||||
- **Not treated as 0%** → Blank is excluded from calculation (doesn't penalize score)
|
||||
- **Better than text "N/A"** → Text causes #VALUE! error, blank is handled gracefully
|
||||
|
||||
**Example:** If NRR is blank but other metrics exist (Rev Growth, Rule of 40, etc.), MSS Score calculates using remaining metrics without error.
|
||||
|
||||
### Related Columns
|
||||
|
||||
Other columns that need single numeric values (not ranges):
|
||||
- **Column M (Rule of 40 Ops)**: Should be calculated value (Ops Margin + Rev Growth)
|
||||
- **Column O (Rule of 40 FCF)**: Should be calculated value (FCF Margin + Rev Growth)
|
||||
- Both can be negative for pre-profitable/turnaround companies
|
||||
|
||||
### Prevention
|
||||
|
||||
When adding new companies:
|
||||
1. Always use single percentage values in NRR column
|
||||
2. Test MSS Score formula immediately after adding row
|
||||
3. If #VALUE! error appears → check Q column for ranges/text
|
||||
212
docs/PREMIUM_SOURCES.md
Normal file
212
docs/PREMIUM_SOURCES.md
Normal file
@@ -0,0 +1,212 @@
|
||||
# Premium Source Authentication
|
||||
|
||||
## Contents
|
||||
- [Overview](#overview)
|
||||
- [Option 1: Keep It Simple (Recommended)](#option-1-keep-it-simple-recommended)
|
||||
- [Option 2: Use Premium Sources (Advanced)](#option-2-use-premium-sources-advanced)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [Alternative: Use APIs Instead](#alternative-use-apis-instead)
|
||||
- [Recommendation](#recommendation)
|
||||
|
||||
## Overview
|
||||
|
||||
WSJ and Barron's are premium financial news sources that require subscriptions. This guide explains how to authenticate and use premium sources with the finance-news skill.
|
||||
|
||||
**Recommendation:** For simplicity, we recommend using **free sources only** (Yahoo Finance, CNBC, MarketWatch). Premium sources add complexity and maintenance burden.
|
||||
|
||||
If you have subscriptions and want premium content, follow the steps below.
|
||||
|
||||
---
|
||||
|
||||
## Option 1: Keep It Simple (Recommended)
|
||||
|
||||
**Use free sources only.** They provide 90% of the value without authentication complexity:
|
||||
|
||||
- ✅ Yahoo Finance (free, reliable)
|
||||
- ✅ CNBC (free, real-time news)
|
||||
- ✅ MarketWatch (free, broad coverage)
|
||||
- ✅ Reuters (free via Yahoo RSS)
|
||||
|
||||
**To disable premium sources:**
|
||||
1. Edit `config/config.json` (legacy: `config/sources.json`)
|
||||
2. Set `"enabled": false` for WSJ/Barron's entries
|
||||
3. Done - no authentication needed
|
||||
|
||||
---
|
||||
|
||||
## Option 2: Use Premium Sources (Advanced)
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Active WSJ or Barron's subscription
|
||||
- Browser with active login session (Chrome/Firefox)
|
||||
- **Option B only:** Install `requests` library if needed:
|
||||
```bash
|
||||
pip install requests
|
||||
```
|
||||
|
||||
### Step 1: Export Cookies from Browser
|
||||
|
||||
**Chrome:**
|
||||
1. Install extension: [EditThisCookie](https://chrome.google.com/webstore/detail/editthiscookie/)
|
||||
2. Navigate to wsj.com (logged in)
|
||||
3. Click EditThisCookie icon → Export → Copy JSON
|
||||
|
||||
**Firefox:**
|
||||
1. Install extension: [Cookie Quick Manager](https://addons.mozilla.org/en-US/firefox/addon/cookie-quick-manager/)
|
||||
2. Navigate to wsj.com (logged in)
|
||||
3. Right-click page → Inspect → Storage → Cookies
|
||||
4. Copy relevant cookies (see format below)
|
||||
|
||||
### Step 2: Create Cookie File
|
||||
|
||||
Create `config/cookies.json` (this file is gitignored):
|
||||
|
||||
```json
|
||||
{
|
||||
"feeds.a.dj.com": {
|
||||
"wsjgeo": "US",
|
||||
"djcs_session": "YOUR_SESSION_TOKEN_HERE",
|
||||
"djcs_route": "YOUR_ROUTE_HERE"
|
||||
},
|
||||
"www.barrons.com": {
|
||||
"wsjgeo": "US",
|
||||
"djcs_session": "YOUR_SESSION_TOKEN_HERE"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Important:** Cookie domain must match feed URL domain:
|
||||
- WSJ feeds use `feeds.a.dj.com` (not `wsj.com`)
|
||||
- Barron's feeds use `www.barrons.com`
|
||||
- Check `config/config.json` for actual feed URLs
|
||||
|
||||
**Note:** Cookie names/values vary by site. Export from browser to get actual values.
|
||||
|
||||
### Step 3: Pass Cookies to fetch_news.py
|
||||
|
||||
**Option A: Modify fetch_news.py (not officially supported)**
|
||||
|
||||
Add cookie loading to `fetch_rss()` function (maintains existing signature):
|
||||
|
||||
```python
|
||||
import json
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
def fetch_rss(url: str, limit: int = 10) -> list[dict]:
|
||||
"""Fetch and parse RSS feed with optional cookie authentication."""
|
||||
|
||||
# Load cookies if they exist
|
||||
cookie_file = Path(__file__).parent.parent / "config" / "cookies.json"
|
||||
cookies = {}
|
||||
if cookie_file.exists():
|
||||
with open(cookie_file) as f:
|
||||
all_cookies = json.load(f)
|
||||
# Extract domain from URL (e.g., feeds.a.dj.com)
|
||||
domain = urlparse(url).netloc
|
||||
cookies = all_cookies.get(domain, {})
|
||||
|
||||
# Fetch with cookies and User-Agent
|
||||
req = urllib.request.Request(url, headers={'User-Agent': 'OpenClaw/1.0'})
|
||||
if cookies:
|
||||
cookie_header = "; ".join([f"{k}={v}" for k, v in cookies.items()])
|
||||
req.add_header("Cookie", cookie_header)
|
||||
|
||||
# ... rest of function (unchanged)
|
||||
```
|
||||
|
||||
**Note:** This is a doc-only suggestion, not officially supported by the skill.
|
||||
|
||||
**Option B: Use requests library instead of urllib**
|
||||
|
||||
Replace `urllib` with `requests` for easier cookie handling (maintains API signature):
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
def fetch_rss(url: str, limit: int = 10, cookies_dict: dict = None) -> list[dict]:
|
||||
response = requests.get(url, cookies=cookies_dict, timeout=10)
|
||||
response.raise_for_status()
|
||||
# ... parse with feedparser
|
||||
```
|
||||
|
||||
### Step 4: Security Considerations
|
||||
|
||||
**Critical: Do NOT commit cookies to git**
|
||||
|
||||
1. **`.gitignore` already includes cookie files:**
|
||||
- `config/cookies.json`
|
||||
- `*.cookie`
|
||||
- No action needed (already configured)
|
||||
|
||||
2. **Set restrictive file permissions:**
|
||||
```bash
|
||||
chmod 600 config/cookies.json
|
||||
```
|
||||
|
||||
2. **Set restrictive file permissions:**
|
||||
```bash
|
||||
chmod 600 config/cookies.json
|
||||
```
|
||||
|
||||
3. **Rotate cookies regularly:**
|
||||
- Browser session cookies expire (usually 7-30 days)
|
||||
- Re-export cookies when authentication fails
|
||||
|
||||
4. **Never share cookie files:**
|
||||
- Cookies grant full account access
|
||||
- Treat like passwords
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "HTTP 403 Forbidden" errors
|
||||
|
||||
**Cause:** Cookies expired or invalid
|
||||
|
||||
**Fix:**
|
||||
1. Log in to WSJ/Barron's in browser
|
||||
2. Re-export cookies
|
||||
3. Update `config/cookies.json`
|
||||
|
||||
### "Paywall detected" in articles
|
||||
|
||||
**Cause:** RSS feed doesn't require auth, but full article does
|
||||
|
||||
**Fix:**
|
||||
- Premium sources often provide headlines/snippets in RSS (no auth needed)
|
||||
- Full articles require subscription + cookie auth
|
||||
- If you only need headlines → no cookies needed
|
||||
|
||||
### Cookies not working
|
||||
|
||||
**Debug checklist:**
|
||||
- [ ] Correct domain in cookies.json:
|
||||
- WSJ: Use `feeds.a.dj.com` (not `wsj.com`)
|
||||
- Barron's: Use `www.barrons.com` (not `barrons.com`)
|
||||
- Check `config/config.json` for actual feed URLs
|
||||
- [ ] Cookie values copied completely (no truncation)
|
||||
- [ ] Browser session still active (test by visiting site)
|
||||
- [ ] File permissions correct (chmod 600)
|
||||
|
||||
---
|
||||
|
||||
## Alternative: Use APIs Instead
|
||||
|
||||
Some premium sources offer APIs:
|
||||
- **WSJ API:** Not publicly available
|
||||
- **Barron's API:** Part of Dow Jones API (enterprise only)
|
||||
- **Bloomberg API:** Enterprise only
|
||||
|
||||
**Conclusion:** Cookie-based auth is the only practical option for individual users.
|
||||
|
||||
---
|
||||
|
||||
## Recommendation
|
||||
|
||||
**For most users:** Stick with free sources. They're reliable, no auth needed, and provide comprehensive market coverage.
|
||||
|
||||
**For premium subscribers:** Follow Option 2, but be prepared to maintain cookie files and handle expiration.
|
||||
Reference in New Issue
Block a user